LIMO-EEG-Toolbox / limo_tools

Hierarchical Linear Modelling for MEEG data
https://limo-eeg-toolbox.github.io/limo_meeg/
Other
59 stars 27 forks source link

Covariate error #43

Closed amisepa closed 4 years ago

amisepa commented 4 years ago

Hi, I am getting a "covariate error" when trying to upload the covariate text file during regression 2nd level. I am using the WLS branch, I first select the Beta_files_design1_GLM_Channels_Frequency_WLS.txt generated during 1st level, then select "1" in pop up text box, and then my covariate file. I have been doing for the past few months successfully but now I get this message: "covariate error - make sure data are in line and column" The data is continuous 1-100 in a text file, in 1 column, no variable name (see attached). limo_variable.txt

I think this is since the last pull but could be wrong.

Thanks in advance! Cedric

CPernet commented 4 years ago

i pushed some changes - let me know if that works now, thx

amisepa commented 4 years ago

It worked on 20 subjects but not on the whole group of 324 subjects. I get this error now:

The logical indices in position 3 contain a true value outside of the array bounds.
Error in limo_random_select (line 320)
                Yr = Yr(:,:,~isnan(sum(X,2)));
Error in limo_random_effect>Regression_Callback (line 245)
    limo_random_select('regression',handles.chan_file,'nboot',handles.b,'tfce',handles.tfce,'type','Channels');
Error in gui_mainfcn (line 95)
        feval(varargin{:});
Error in limo_random_effect (line 29)
    gui_mainfcn(gui_State, varargin{:});
Error in
matlab.graphics.internal.figfile.FigFile/read>@(hObject,eventdata)limo_random_effect('Regression_Callback',hObject,eventdata,guidata(hObject)) 
Error while evaluating UIControl Callback.

Could it be that the 2 digits numbers need to have a 0 as a first digit so that they all are 3 digit numbers (as 100 is the only 3 digits number of the file)?

And another quick question: I get 2 Beta files from 1st level now (I was getting only one file before):

Thanks, Cedric

CPernet commented 4 years ago

'It worked on 20 subjects but not on the whole group of 324 subjects. I get this error now:'

but the analysis is for the group, what do you mean by worked on 20 subjects?
pull the current version just to be sure -- try to put a break point line 320 Yr = tmp_data(:,:,~isnan(sum(X,2)));
it says that ~isnan(sum(X,2)) has a value outside the range of the data, so two possibilities either size(tmp_data,3) is not 324 or size(X,1) is not 324 (since i simply sum the columns of X and look for non nan) - once we know that we can figure where the problem is coming from

CPernet commented 4 years ago

'Beta_files_Gp_STUDY.design 1_GLM_Channels_Frequency_WLS' no difference, have you specified a group in STUDY ; if not (or even a single group) this shouldn't happen - but somehow still does :~(

amisepa commented 4 years ago

Sorry I meant that I ran it on a group of 20 subjects to check quickly and it worked. But then it didn't work on my whole sample of 324 subjects.

I re-ran it with less aggressive ASR and only included files with > 60 sec of data (some of them had less after cleaning as they are short files). Now I have 319 files. I didn't get the error about subjects that didn't get processed in 1st level. Thanks for solving that part. No error message in 2nd level, but following your steps, I get size(tmp_data,3) = 319 and size(X,1) = 315 And the R2 and Betas results generated by the 2nd level regression are filled with NaNs.

Does LIMO work on subjects with different number of channels? I don't have a lot of them with this wearable headset so I can't interpolate them when they are bad. Could that be the problem?

Thanks, Cedric

amisepa commented 4 years ago

no difference, have you specified a group in STUDY ; if not (or even a single group) this shouldn't happen - but somehow still does :~(

I just imported my BIDS folder to STUDY with the plugin and pre-computed the data before LIMO. Maybe it automatically specifies a group?

And I think there used to be a message in command window before saying "regression completed succesfully" or something like that. Whereas now there is nothing:

image

CPernet commented 4 years ago

you need to create/edit a design within study see https://github.com/LIMO-EEG-Toolbox/limo_meeg/wiki

amisepa commented 4 years ago

I think that's automatic with BIDS because it was working before without having to do that, and the variable is visible in the design after importing with BIDS. I only have one continuous variable (1 session, 1 condition). I did it manually in the GUI and recomputed everything just in case, and it still doesn't work. The R2 is filled with NaNs.

I see this in your tuto page: "LIMO EEG uses a Weighted least Squares approach for each trial, which means you must have more trials than time frames (for ERP and ERSP) or frequency frames (for Spectrum)." Just to check, my files have at least 60 trials of 256 samples in my new analysis. So if I do spectrum 1-30 Hz, that means all files must have at least 31 trials, correct?

CPernet commented 4 years ago

Hi Cedric, sorry about that 1) i pushed yesterday a pop up windown specific to this problem, do you mind retrying see if that's come up? 2) spectrum 1-30Hz doesn't necessarily means 30 frames, depends how it's computed but yes at least 31 trials 3) @arnodelorme - do we need to create a design automatically? i thing STUDY yes but leave empty in terms of design

amisepa commented 4 years ago

Hi Cyril,

Ok thanks! I pulled the WLS branch and tried again but it looks like it is still not popping up and’ processing the regression.

CPernet commented 4 years ago

if it's processing it's good no?

amisepa commented 4 years ago

Sorry, "not processing", the results are still empty

arnodelorme commented 4 years ago

This is not a very useful message to help debug. What is empty? Maybe the files got saved somewhere else etc...

On Aug 28, 2020, at 10:11 AM, Cedric Cannard notifications@github.com wrote:

Sorry, "not processing", the results are still empty

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

amisepa commented 4 years ago

Yes this is in the discussion above and addressed to Cyril. The WLS regression results are created but filled with NaN values. When launching the regression, there used to be a popup window to confirm computing the regression after uploading the Beta_design and covariable.txt files, as well as confirmation that the regression was done. But now nothing happens except the creation of results files (i.e. R2.mat, LIMO.mat, covaraite.mat, etc.) containing NaNs. I think Cyril is working on it and the question for you was: When importing from BIDS to STUDY, is the design created automatically with the imported condition (knowing I have only 1 condition and don't need to create specific designs) or do you need to go in STUDY design to add it manually for LIMO to work properly? Or do you need to leave it empty and only upload the covariable.txt later in 2nd level? Or both? I am pretty sure I used to leave it empty and only add the txt file during 2nd level when it was working before, but I could be wrong.

CPernet commented 4 years ago

@arnodelorme we made something to make a design of 'ones(N_trials,1)' so maybe that's what it does and somehow fails to compute? @amisepa would be useful to have access to some data

amisepa commented 4 years ago

Sorry I closed the issue by accident, trying to delete a comment. Ok I'll work on a smaller sized folder to share with you this week.

Cedric

arnodelorme commented 4 years ago

Hi Cedric,

Can you provide a dataset (small) and clear instructions?

Arno

On Aug 31, 2020, at 8:33 AM, Cedric Cannard notifications@github.com wrote:

Sorry I closed the issue by accident, trying to delete a comment. Ok I'll work on a smaller sized folder to share with you this week.

Cedric

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

amisepa commented 4 years ago

Can you access this link? https://drive.google.com/drive/folders/1VzS8Kg_3niSy_E8IkH9qJPwKc9M8Ojh-?usp=sharing

arnodelorme commented 4 years ago

I was on the phone with Cedric. Way to reproduce the problem (WLS branch of LIMO with latest changes):

  1. Load attached study (small files) https://drive.google.com/drive/folders/1VzS8Kg_3niSy_E8IkH9qJPwKc9M8Ojh-?usp=sharing
  2. Already done. Compute spectrum (just in case) [1 30] Hz
  3. Already done. Run limo first level.
  4. Create regression folder; Select channel location file
  5. Call regression. 5.1. Select beta file. 5.2. Select covariable.txt 5.3. Look at design matrix (which is ok)

Then instead of calculating (as it did before), it just returns right away and all files with NaN (look at the R2 file). Note that some subjects do not have all electrodes (but you said it was not a problem).

CPernet commented 4 years ago

ah GUI problem - solved now (pull new limo repo / also eeglab repo to update, you should not have had this gp file)

now of course no need of GUI - update the script with chanlocs = fullfile(STUDY.filepath,'limo_gp_level_chanlocs.mat');
limo_random_select('regression',chanlocs,'LIMOfiles',...
fullfile(STUDY.filepath,'Beta_files_design1_GLM_Channels_Frequency_WLS.txt'), ...
'analysis type','Full scalp analysis', 'type','Channels','nboot',1000,'tfce',0,'regressor',...
fullfile(STUDY.filepath,'covariable.txt'),'parameter',1,'zscore','Yes','skip design check','Yes')

Note: it defaults to OLS for the group analysis, the bootstrap is too conservative with IRLS (working on it)

arnodelorme commented 4 years ago

Glad you fixed it.

On Sep 3, 2020, at 5:09 AM, Cyril Pernet notifications@github.com wrote:

Closed #43.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

amisepa commented 4 years ago

It worked :) Great thanks!

I get this new error message when plotting the results though.

image

Note: it defaults to OLS for the group analysis, the bootstrap is too conservative with IRLS (working on it)

Does this mean it is doing OLS instead of WLS even though I select WLS? Is bootstrap ok with WLS? I am curious as to why you didn't turn ON TFCE in your code above?

CPernet commented 4 years ago

unique(newticks) solved it - pull the update

amisepa commented 4 years ago

Worked thanks!