sccn / eeglab

EEGLAB is an open source signal processing environment for electrophysiological signals running on Matlab and developed at the SCCN/UCSD
https://eeglab.ucsd.edu/
Other
612 stars 247 forks source link

EEGLab EDF+ events misplaced and wrongly named #267

Closed faimai00 closed 3 years ago

faimai00 commented 3 years ago

Description

I am loading EEG files in EDF+ format using the EEG = pop_biosig function (MATLAB_R2020b). When I plot them (pop_eegplot(EEG)) the events appear all grouped at the beginning of the recording, and all have the same name. This should not be the case, as the events should be marking specific events across the file, and should also not all be named the same way. The information contained in the EEG.event matrix reflect this (see screenshot).

Please would you be able to help? How can I import these so the events fall on the correct locations and have correct naming? Many thanks in advance - any advice would be greatly appreciated.

Irene

Screenshot 2021-03-10 at 15 01 58 Screenshot 2021-03-10 at 15 02 49


Steps to Reproduce

EEG = pop_biosig('/Volumes/Ashur DT2/EEG/RKS10/RawData.EDF') pop_eegplot(EEG)


Expected behavior: events are loaded with correct names throughout the file


Actual behavior: events all have the same name, and they are all grouped in the beginning of the recording.


Versions

| OS version | 10.15.7 (Catalina) | | Matlab version | R2020b | | EEGLAB version | eeglab2021.0 |

arnodelorme commented 3 years ago

Would you mind to share the EDF file. Cedric @amisepa is our EDF expert :-). Cedric would you mind having a look?

faimai00 commented 3 years ago

Of course - thank you so much @arnodelorme and @amisepa. Please find the EDF+ file here: https://drive.google.com/drive/folders/1SYpc6UMnbe-ip4lRB_cD_Eq1HCRP9e9I?usp=sharing

This is just one file, but I am having the same problem with all of the recordings I have.

Many thanks. Irene

faimai00 commented 3 years ago

Dear Cedric,

thank you for your email. Sorry - I am not sure where I can find the checkbox. Could you please specify further?

Many thanks. Irene


From: Cedric Cannard @.> Sent: 11 March 2021 17:53 To: sccn/eeglab @.> Cc: Faiman, Irene @.>; Author @.> Subject: Re: [sccn/eeglab] EEGLab EDF+ events misplaced and wrongly named (#267)

You don't often get email from @.*** Learn why this is importanthttp://aka.ms/LearnAboutSenderIdentification Feedbackhttp://aka.ms/SafetyTipsFeedback

Hi,

During import, have you tried using the checkbox entitled "Import using alternative BIOSIG method (mexSLOAD)"?

Cedric

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsccn%2Feeglab%2Fissues%2F267%23issuecomment-796924287&data=04%7C01%7Cirene.faiman%40kcl.ac.uk%7Ca07e796e978b4f2440b608d8e4b69415%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637510820112131588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iHIDOzQP9wCoFp9ytEpzzsDMBCecmKWtotwKXdfnvw8%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPFDXTIPQ6CVTTYUMRDRLCDTDDYQRANCNFSM4Y6HIB5Q&data=04%7C01%7Cirene.faiman%40kcl.ac.uk%7Ca07e796e978b4f2440b608d8e4b69415%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637510820112141576%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hU1o0GB60tbDBTvpFQm5QH%2BZJO2TJ1a2te5tNkS5d98%3D&reserved=0.

amisepa commented 3 years ago

It's in the pop-up window during importation: image

It's not working though I just tried with your file. You seem to have custom event names that contain spaces and are too long. I'll work on an alternative solution asap.

Cedric

faimai00 commented 3 years ago

Thank you so much - I really appreciate it. Irene


From: Cedric Cannard @.> Sent: 11 March 2021 18:21 To: sccn/eeglab @.> Cc: Faiman, Irene @.>; Author @.> Subject: Re: [sccn/eeglab] EEGLab EDF+ events misplaced and wrongly named (#267)

You don't often get email from @.*** Learn why this is importanthttp://aka.ms/LearnAboutSenderIdentification Feedbackhttp://aka.ms/SafetyTipsFeedback

It's in the pop-up window during importation: [image]https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F58382227%2F110835142-449d7b80-8253-11eb-8164-047ecfe52f0a.png&data=04%7C01%7Cirene.faiman%40kcl.ac.uk%7C21c5af0414ab4a189e3d08d8e4ba7b04%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637510836870588648%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1Wf%2BHduj8nvBShd3VI9e9R%2BdCrytUohNmvXwwcHkCfw%3D&reserved=0

It's not working though I just tried with your file. You seem to have custom event names that contain spaces and are too long. I'll work on an alternative solution asap.

Cedric

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsccn%2Feeglab%2Fissues%2F267%23issuecomment-796944349&data=04%7C01%7Cirene.faiman%40kcl.ac.uk%7C21c5af0414ab4a189e3d08d8e4ba7b04%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637510836870598646%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0Yy%2F1orV41unaeDA1Dnpyjq6EkTbh8tgWnOZzbVJsbc%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPFDXTK2Y33INGVDBG7ZSRDTDD3ZJANCNFSM4Y6HIB5Q&data=04%7C01%7Cirene.faiman%40kcl.ac.uk%7C21c5af0414ab4a189e3d08d8e4ba7b04%7C8370cf1416f34c16b83c724071654356%7C0%7C0%7C637510836870598646%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=fcuHncbBVszYFqme8qXfBcxovH5M60LSmLpOtex2d6o%3D&reserved=0.

amisepa commented 3 years ago

Hi,

It should work now if you pull eeglab,

Let me know!

arnodelorme commented 3 years ago

Cedric, I was looking at the 'makeur' command you mention erase the event structure. Would you mind to find an example. This command should not affect at all the event structure except for adding a "urevent" field.

faimai00 commented 3 years ago

Hi @amisepa, thank you for that. However after updating EEGLab I still cannot read the file properly. Nothing has changed. Did it work for you?

I am not sure if it is relevant but I don't have the option of importing using alternative BIOSIG method (mexSLOAD). I am using EEGLab 2021.0:

Screenshot 2021-03-17 at 09 50 44
faimai00 commented 3 years ago

@amisepa may I please also ask if there is anything I can do differently in how I annotate and export the EDF+ files, or whether the problem is purely on the MatLab import end? Many thanks! Irene

amisepa commented 3 years ago

Do you use git? You probably need to install eeglab with this command: git --recurse-submodules https://github.com/sccn/eeglab.git @arnodelorme is that why she doesn't have the new window?

Sorry I forgot to add to my message this piece of information:

Your data is EDF+D (discontinuous) so you need to convert it to EDF+C with EDFBrowser for example. Sorry, I don't have a better option for now so you'll have to do that for each file manually, but at least it works. 1) download EDFBrowser here: https://www.teuniz.net/edfbrowser/ 2) Tools > convert to EDC+C This will create several small files (because your data is discontinuous) image

3) in eeglab > Import > eglab functions > from EDF/EDF+ (biosig toolbox) Untick import EDF+ annotations and tick 'Import using alternative method'

4) Then you'll need to do this for each sub-file and merge them (https://sccn.ucsd.edu/wiki/Set_merge) You do for example: EEG1= EEG; import next file EEG2= EEG; import next file EEG3 = EEG; ... etc

EEG = set_merge(EEG1,EEG2,EEG3)

To save time, you can import each dataset with the command line:

EEG1 = pop_biosig('file_path\RawData_0001.edf', 'importannot','off','importmex','on');
EEG2 = pop_biosig('file_path\RawData_0002.edf', 'importannot','off','importmex','on');
EEG3 = pop_biosig('file_path\RawData_0003.edf', 'importannot','off','importmex','on');
EEG = set_merge(EEG1,EEG2,EEG3)

Let me know if this works for you.

As for your second question, I have never recorded EDF data so I cannot advise on that sorry. You might look if you can change settings in your recording software to record the data as EDF or EDF+C directly to avoid the conversion. And making the annotations simple numbers or codes (e.g T1, T2, etc.) is generally better than characters.

Cedric

faimai00 commented 3 years ago

Dear Cedric, Thank you very much for your help and detailed instructions. I have converted to EDF+C and the events are now positioned in the right place, however they still have all the same name. I have cloned the repository, but I still can't see the new window for some reason so I cannot import using alternative BIOSIG method. Is that probably why the event names keep being incorrect? Would you please have any advice for how to update to the new window? I have re-downloaded the last version of EEGLab + went through git cloning, but I still get the same window as the one in the screenshot I posted above. Many thanks again for your precious help! Irene

amisepa commented 3 years ago

Yes you need the new window to have the fixed annotations.

Did you use this line in your terminal to install eeglab? git clone --recurse-submodules https://github.com/sccn/eeglab.git

I would delete the old eeglab folder, re-install using the line above, and make sure the Matlab path is updated to the new eeglab folder.

I think that is what you need to get the new changes.

faimai00 commented 3 years ago

Dear Cedric,

thank you very much for this. I have done what you said and I am able to see the new changes now. I am so sorry to bother with this, but I am experiencing more problems and I still cannot get to open the files correctly.

Initially I could not tick the box "Import using alternative method" because of an error message saying that mexSLOAD was not installed. I have then downloaded mexSLOAD from https://pub.ist.ac.at/~schloegl/src/mexbiosig/ and pasted it into my eeglab folder. I am now able to tick the box "import using alternative method", but as soon as I press OK this error comes out (see screenshot). I can see that line 238 of pop_biosig() is the following:
237 if strcmpi(g.importmex, 'on') 238 [s,HDR] = mexSLOAD(filename);

I have tried downloading multiple mexSLOAD files from pub.ist.ac.at but no-one seems to work. Does any solution to this come to your mind? Thank you so much again and sorry about this. I spent several hours on it but I could not solve the problem.

Screenshot 2021-03-18 at 15 05 10
amisepa commented 3 years ago

Hi,

You don't need to download the mex files, they are already in the Biosig plugin in eeglab when you install it. It might be due to an update we did yesterday to improve this function.

Are you on Mac or Windows? Did you install the bisoig toolbox via eeglab GUI first?

It is working on my side now, I followed these steps:

1) Convert EDF files to EDF+C. This should give you 5 files right? image 2) clone eeglab with git clone --recurse-submodules link 3) add path to eeglab folder in matlab 4) File > import EDF data with eeglab > pop-up window "do you want to install Biosig toolbox?" > Yes 5) select one of the files created by EDFBrowser 6) UNCHECK the "Import EDF+ annotations" box and CHECK the "Import using alternative Biosig method (mexsload)" box 7) Plot channel data (scroll)

faimai00 commented 3 years ago

Hi. I am using mac (OS version | 10.15.7 (Catalina)). I did install biosig via the eeglab GUI. It shown within matlab plugins as Biosig3.7.5. However no luck unfortunately.... It looks like the mexSLOAD file is not there (see screenshot). Can it be a problem with the files I get from the EDF converter? I get those 5 files with exactly the same naming you have, but EDFBrowser cannot read them (see screenshot). Thanks!

Screenshot 2021-03-19 at 12 51 57 Screenshot 2021-03-19 at 12 47 58
Teuniz commented 3 years ago

I get those 5 files with exactly the same naming you have, but EDFBrowser cannot read them

Sorry for barging in like this. You are probably using an older version of EDFbrowser. Please make sure you are using the latest version of EDFbrowser and convert the files again. At this moment, the latest version is 1.82. Older versions can create invalid EDF+ files when using the EDF+D to EDF+C converter, depending on the content of the recording field. From here it seems you are probably using version 1.77 for Mac which is affected by this bug. Regards, Teunis

faimai00 commented 3 years ago

Dear Teunis, thank you for that. Unfortunately the last version available that I can see for mac is v1.77. I will see if I can find a way around this somehow. So do you think that the problem is actually the way the EDFs are created, rather than anything on eeglab then? Many thanks for your great help @amisepa and @Teuniz, I really appreciate it!

Teuniz commented 3 years ago

So do you think that the problem is actually the way the EDFs are created, rather than anything on eeglab then?

I don't know but it's likely. You can compile EDFbrowser from source, there's a howto at the end of the README file that comes with the sourcecode that describes how to compile on the Mac. Otherwise, run the Linux- or windows version in a virtual machine.

amisepa commented 3 years ago

Hi,

Thanks @Teuniz for helping on this, I don't know the details about EDFBrowser.

1) step 1: @faimai00 I can read them successfully in Windows version 1.82. Can you compile it from source as Teuniz suggested?

2) step 2: @arnodelorme I think you added recently the correct mexSLOAD for Mac. Faimaz gets an error saying it is not installed in her Biosig toolbox (see above), could you please check why or send her the appropriate file for Mac? It should be located with the other ones in ...\plugins\Biosig3.7.5\biosig\t200_FileAccess

3) @faimai00 Do you have the Signal processing toolbox installed on your Matlab 2020b? if so, we can try another approach in the meantime.

Cedric

faimai00 commented 3 years ago

Hi Cedric, many thanks. I have just downloaded the signal processing toolbox so very happy to try any alternative method in the meantime @amisepa . I will try to compile EDF browser from source tomorrow anyways and see what happens. The directory you kindly gave in point 2 does not actually include a file for mac in my folder so if you could point me to the right file @arnodelorme I would be really grateful as I have tried a few but could not get them to work. Thank you so much all for your support!

faimai00 commented 3 years ago

Dear @amisepa and @arnodelorme, just checking if there has been any progress with this? Thank you!

faimai00 commented 3 years ago

I have now converted the files to EDF+C with the latest version of EDFbrowser for Windows. That appears not to be the problem as I get the same error messages regarding mexSLOAD not being installed. If I install the mexSLOAD Mac file myself from https://pub.ist.ac.at/~schloegl/src/mexbiosig/, the error message I get regards the mex file being invalid.
Thank you.

amisepa commented 3 years ago

Hi @faimai00,

Sorry, been very busy!

I have worked on the alternative method when I could, but it was harder than I thought because of the discontinuous nature of your data (mainly with the event latencies). Just curious, what system is this?

I've come up with this script attached that works only with the Signal Processing Toolbox on Matlab 2020b or later. You don't need to convert to EDF+C with this one.

Just add path to this script, and import your file directly with:

eeglab
EEG = import_EDFplusD();   %pop-up 
% EEG = import_EDFplusD(filePath)
pop_eegplot(EEG, 1, 0, 1);
figure; pop_spectopo(EEG, 1, [], 'EEG' , 'freq', [6 10 22], 'freqrange',[1 80],'electrodes','off');

I've just compared the result briefly with what is displayed in EDFBrowser. It looks ok so far, but please run more tests to make sure everything is correct! Especially the markers!

Let me know how it goes.

Cedric

import_EDFplusD.zip

faimai00 commented 3 years ago

Dear Cedric, thank you very much for all your work and for coming up with this! I have tried it. The order of the labels across the recording seem fine, however I think that the events are still getting misplaced. In the past week I have tried an edf import function that a colleague of mine has written, and there is discrepancy between where the events get positioned (see screenshots).

Based on observation of the recordings and where the events fall, I think that the latencies obtained with import_EDFplusD are inaccurate. Also, these differ from the ones obtained by pop_biosig when "import EDF+ annotations" is checked.

My plan I think will be to still use the pop_biosig function (with import EDF+ annotations checked, as I cannot check the alternative method mexSLOAD box). With this, I get the events positioned correctly but with the wrong name. I will then rename them based on the events order that I get with either import_EDFplusD or the experimental edf import function. It is a bit tedious but I think it can work. What do you think? Many thanks again! Irene

PS The system where I am exporting data from is the Natus EEG software

Screenshot 2021-04-01 at 15 13 02 Screenshot 2021-04-01 at 15 14 28
amisepa commented 3 years ago

Hi,

Sorry I got the sampling rate wrong. The latencies are in ms, so you would have the following latencies in seconds with the correct sample rate: 1.998; 2.808; 2.92; etc. Here is it is fixed. import_EDFplusD.zip

Maybe I missed something, but I believe your latencies are incorrect. The durations between events seem to correspond, but I think the start time is wrong in either your method or mine. I adjusted the start time based on the actual time stamps of the EEG data. I see the same ones as yours in info.Annotations. Not sure why they are different. And I corrected latencies for gaps between segments, whereas in your screenshot you still have the gaps (with no EEG data) included that need adjustment.

Comparing my latencies from segment 1 to the ones from EDFBrowser(1st segment converted to C+ with EDFBrowser), which are likely the correct ones: image

If you set a breakpoint line 120 of my function and launch it, you can look at the ev structure containing, event type, correct latency, data segment it belongs to, and corresponding gaps removed. You can then compare the correct latencies with EDFBrowser (after they are converted in each segment of EDF+C). I've looked briefly at a few events and their differences in duration in segments 1 and 2 and they seem to match. But this is based on the assumption that EDFbrowser has correct latencies after conversion to EDF+C.

Cedric

Teuniz commented 3 years ago

I get those 5 files with exactly the same naming you have, but EDFBrowser cannot read them

I just wanted to inform you that some volunteer has released EDFbrowser 1.83 for the Mac. (which solves the described problem) You can find it here: https://gitlab.com/whitone/EDFbrowser/-/releases or here: https://formulae.brew.sh/cask/edfbrowser#default

Best, Teunis

faimai00 commented 3 years ago

Dear Cedric,

Thank you so much for adjusting the sampling rate. You are right, the latencies obtained with your function are correct! I have also managed to compare them with the original files in Natus and they do correspond. I can read most of the files with the correct annotations! Thank you!

Just one thing - I am testing the function with different files I have (all edf+) and it seems like I cannot read the files containing one clip only (Error message appears as per screenshot below: "Unable to perform assignment with 0 elements on the right hand side"). These cannot be read by your function, but can be read by EDFbrowser. I have included an example file in the Drive (named 09RawData1clip): https://drive.google.com/drive/folders/1SYpc6UMnbe-ip4lRB_cD_Eq1HCRP9e9I Is it because the function is tailored to read +D only?

Also, it appears that it cannot read some of the files that contain more than one clip (+D). I haven't been able to pinpoint in what aspect these might differ from the rest of the multi-clip (EDF+D) files that are readable. I have included an example file in the Drive (named 17RawDataUnreadable). This generates a different type of error (see screenshot: "Index exceeds the number of array elements (0)").

Thank you so much Cedric for your continuous help, I am very grateful.

Best wishes, Irene

Screenshot 2021-04-06 at 16 31 57 Screenshot 2021-04-06 at 16 32 42
amisepa commented 3 years ago

Hi,

Is it because the function is tailored to read +D only?

Yes I made the function based on your discontinuous EDF+D file.

Also, it appears that it cannot read some of the files that contain more than one clip (+D). I haven't been able to pinpoint in what aspect these might differ from the rest of the multi-clip (EDF+D) files that are readable.

How is this file different than the other EDF+D file I worked on?

Thanks for sharing the files, I will see if I can adjust the function for these other files when I have some time (could be in a few weeks just so you know), and potentially make a new EEGLAB plugin from it if it works well.

Thanks, Cedric

faimai00 commented 3 years ago

Dear Cedric,

I am not sure how these files differ from the other EDF+D file you worked on. I tried to find out but I cannot pinpoint any specific factors unfortunately. If you could have a look, that would be great thank you.

Re: EDF+C files: would there be an easy way for your function to read these as well? With biosig mexSLOAD not working on mac, I am unable to read these unfortunately.

Thank you.

Irene

amisepa commented 3 years ago

Ok.

Yes, I will add the option in my function, it is much easier than EDF+D as it is continuous.

I'll let you know when it's done.

Cedric

amisepa commented 3 years ago

I've made it compatible for continuous data: import_EDF.zip

However, I realized that there may be DC offset differences between segments of the discontinuous data (EDF+D data) that needs to be corrected. I'll make a plugin soon with this problem fixed when I have some time. In the meantime, you may add boundaries manually with pop_eegplot(EEG,1,0,1) if you can reject sections of data where segments change. This will add boundaries at the right place in the data and then EEGLAB's filter functions will automatically correct DC offset changes if there are any.

The problem with the other EDF+D file comes from the Matlab edfread function for some reason. Could you please submit that specific issue on MathWorks to see if they can fix it?

Cedric

faimai00 commented 3 years ago

Dear Cedric,

thank you so much for making the function compatible with EDF+C. Also, thank you in advance for working on the plugin that will correct the DC offset changes. If you could notify when this is done I would be grateful so I can switch from using the manual method you kindly suggested.

All right - I will look into the problem with the other EDF+D files separately and log an issue.

I am really grateful for all your help! It has been really invaluable, thank you! Irene

arnodelorme commented 3 years ago

Cedric, any more feedback. Can we close this report?

amisepa commented 3 years ago

Yes we can close it, I just need to make a new plugin with this alternative method that uses Signal Processing Toolbox with Matlab > 2020a.

faimai00 commented 3 years ago

Dear Cedric @amisepa , I am experiencing some problems with this import_EDF function as this seems to import only half of the recordings file in MatLab. For example, the file named RKS05 (https://drive.google.com/drive/folders/1SYpc6UMnbe-ip4lRB_cD_Eq1HCRP9e9I?usp=sharing ) consists of 246 seconds of EEG recordings in EDF+, but once I import it using import_EDF, my xmax = 127.999. As you can see, the EEG.event latencies go up to 234 seconds. If you open the file in EDFbrowser, you can see all the 246 seconds of recording. This problem presents consistently across most of the EEG files I have.

Please may I ask if you have any suggestions on how to solve this problem?

I am now working in Matlab2021_a (as MatLab2020b was giving some problems with the matlab edfread function, as described above - which get resolved by updating to 2021_a).

Many thanks. Irene

amisepa commented 3 years ago

Sorry to hear that, I'll take a look when I can

amisepa commented 3 years ago

I have an idea of what the problem could be. Does the sampling rate look correct?

faimai00 commented 3 years ago

Dear Cedric, thank you for your answer. The sampling rate is not correct - it displays as 1000 Hz for these data (RKS05), whilst the original sampling rate is 500 Hz. However, it does import a correct sampling rate for other data I have (e.g RKS07 which I have uploaded in the same folder). Many thanks for your advice. Best wishes, Irene

amisepa commented 3 years ago

Ok thank you. I have an idea of what might be going on. I'll take a look when possible.

faimai00 commented 3 years ago

Hi Cedric, all right. I am looking forward to any suggestions you could give on this. Many thanks! Irene

faimai00 commented 3 years ago

Dear @amisepa I am sorry for the additional message, I was just wondering whether you would have any suggestions on this? I have looked into it but I was not able to find the cause of the problem. Any suggestions would be very welcome as I am very stuck with this problem. Many thanks again. Irene

amisepa commented 3 years ago

Sorry, extremely busy period...

When your RKS05 data are imported with my function line 64, cells contain 1000 data points and timepoints are every 2 s, and the matlab function reads 1000 hz sample rate as a consequence, but the sample rate should actually be 500 hz. My code then takes each cell one by one and converts it to continuous data, and assumes the sample rate is 1000 hz. I believe the solution would be to divide each cell in half instead to get the correct 500 timepoints for each second, and correct the sample rate to 500. Might be tricky to get the correct event latencies though, can't remember how that would go. You can try and see if events adjust automatically. I'll try to take a look maybe next week if I can. Hope this helps and that you can find a solution in the mean time.

Cedric

amisepa commented 3 years ago

Hi Irene,

I think I fixed the issue. Please try this version attached: import_EDF.zip

I compared the events of your file RKS05 with EDFBrowser (accounting for gaps between segments that were removed by my function), and it looks correct. Please run some tests and let me know.

Cedric

faimai00 commented 3 years ago

Dear Cedric,

Thank you so much with that! I ran some tests also with some other files I have and it seems like it is working now! The sampling rates are correct and the whole EEG file is read and visualised correctly. I will let you know if I encounter any problems when I am going to work with it more in a few weeks, but it doesn't seem likely.

I only get an error message regarding EEG.setname (line 239):

Screenshot 2021-07-14 at 10 23 29

However if I comment out line 239 (which just regards the file name anyways) it runs smoothly! Many thanks again for your help, this is extremely useful!

Irene

amisepa commented 3 years ago

Glad it's working.

That setname issue should be fixed now, and I added some code to insert boundaries where DC offsets changes occur between discontinuous segments (e.g. file RKS05). EEGLAB default filters should automatically correct them now by detecting the boundaries.

Here is the new repo for this function: https://github.com/amisepa/import_EDF.git This way you can just download or pull the latest version from there directly in the future.

Hope it works well with the rest of your data.

Cedric

faimai00 commented 3 years ago

This is amazing, thank you so much Cedric for all your help!! Irene

arnodelorme commented 3 years ago

Thanks Cedric,

arnodelorme commented 3 years ago

This one you can submit to the plugin manager. Tx,

On Jul 16, 2021, at 8:14 AM, Arnaud Delorme @.***> wrote:

Thanks Cedric,

amisepa commented 3 years ago

Ok I'll do that when I have some time ;)

faimai00 commented 3 years ago

Hi Cedric, just a note - line 99 should import data if they are continuous I believe, but it refers to discontinuous data instead:

%CONTINUOUS DATA
if ~discontinuous
    disp('Continuous data detected.');
    if (max(nSrate)-min(nSrate))/max(nSrate) > 0.01
        warning('Sampling rate unstable! This can be a serious problem!');
    end
    for iEv = 1:size(annot,1)
        EEG.event(iEv,:).type = char(table2array(annot(iEv,2)));
        latency = datenum(datetime(table2array(annot(iEv,1)), 'Format', 'HH:mm:ss:SSS'));
        latency = latency - datenum(datetime(edfTime(1), 'Format', 'HH:mm:ss:SSS'));
        EEG.event(iEv,:).latency = round(latency*24*60*60*sRate);   %correct latency in ms
        EEG.event(iEv,:).urevent = iEv;
    end
    EEG = eeg_checkset(EEG);

If you would have a chance to check that, it would be really useful. Many thanks! Irene