Permutation Tests and Multiple Comparisons in the Linear Models and Mixed Linear Models, with Extension to Experiments using Electroencephalography

Abstract

We present how permutation tests can be applied in experiments using electroencephalography (EEG). First, we present the permuco R package which allows permutation tests on linear model and repeated measures ANOVA with nuisance variables. It uses several permutation methods and, for comparison of signals, it applies multiple comparisons procedures like the cluster-mass test or the threshold-free cluster-enhancement. Second, we show that most of the permutation methods have a geometrical interpretation. Moreover, we present a real data analysis where the cluster-mass test is used for a full-scalp analysis of EEG data. We also show that using the slopes of the EEG signals in combination to the cluster-mass test produces more powerful tests. Third, asymptotic properties of the F statistic of several permutation methods are derived using the moments of the conditional distribution by permutations. Fourth, we explain why experiments in psychology should often be modelised by a cross-random effects mixed effects model (CRE-MEM) and we show that the assumed correlation structure of the data influences tests of fixed effect parameters. Finally, we propose a general re-sampling framework to analyse EEG data when using CRE-MEM.