Open Lychfindel opened 5 months ago
Hi - this is very odd. We are having trouble replicating this. What version of freesurfer are you using?
I'm using FreeSurfer 7.4.1 on Ubuntu22.
The full version name is freesurfer-linux-ubuntu22_x86_64-7.4.1-20230614-7eb8460
It looks like a permission error on the fsaverage folder. It is odd given that you had permission to download fsaverage (??):
PermissionError: [Errno 13] Permission denied: '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/bem'
Just to check that this isnt' using your system SUBJECTS_DIR variable - can you confirm that this freesurfer directory is the one associated with the Enigma processing (MNE will use the environmental var if none is provided - and that would be a bug in our code)
If it is the one associated with the BIDS dir processing the Enigma code, can you do the following:
chmod -R a+rw /mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage
And you will also need to chmod the .../subjects/morph-maps folder (this will be created several steps later).
If you run into the issue again - can you add the full commandline call to process_meg.py in the comment, so we can investigate further. You can replace any sensitive information (paths/subjids) with alternative characters. This will help us better troubleshoot the issue.
Thanks for providing the info.
Hi Jeff, thanks for the explanation.
The permission error arise only after the step proc_mri
: before this step the folder has full read/write permissions, but during this step the directory fsaverage
is replaced with a symlink to my local folder /usr/local/freesurfer/7.4.1/subjects/fsaverage
which has write permissions only for root user.
The command line I'm using is:
process_meg.py -bids_root /mnt/raid/atonin/bids/Protocollone -subject sub-0001 -session 20180410 -mains 50 -n_jobs 32 -run 01 -rest_tag MEGLabRestingEO -emptyroom_tag SysTest
But the $SUBJECTS_DIR variable points to my local directory:
echo $SUBJECTS_DIR
> /home/alessandro.tonin/subjects
Looking into this currently - will try to have a fix for you soon.
Thanks
I tried a full reinstall on a new system (freesurfer 7.4 / conda / mne / enigma_meg + megnet) and am still not able to replicate the error. We did make a change to the code - it was trying ot pull a new version of fsaverage for all subjects even if it was present.
Could you pull the updated code (or just run pip install git+https://github.com/jstout211/enigma_MEG.git
to install the new changes).
If you unlink the fsaverage from your BIDS/freesurfer/subjects folder (linked to your primary SUBEJCTS_DIR) and re-run I'm hoping that this may fix the issue after the code updates. If you still have issues we can try to troubleshoot this over video call. Let me know the outcome.
Thanks
Unfortunately, also with the updated code (commit 373c73) I get an error and the fsaverage folder is replaced with a sym link. However, the error is now different:
INFO:0001_ses_20180410_task_MEGLabRestingEO_run_01:do_beamformer :: COMPLETED
INFO:0001_ses_20180410_task_MEGLabRestingEO_run_01:do_make_aparc_sub :: START
ERROR:0001_ses_20180410_task_MEGLabRestingEO_run_01:do_make_aparc_sub :: [Errno 13] Permission denied: '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/label/tmpzxoa3e11' | Pooch could not write to data cache folder '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/label'. Will not be able to download data files.
Traceback (most recent call last):
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/site-packages/pooch/utils.py", line 264, in make_local_storage
with tempfile.NamedTemporaryFile(dir=path):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/tempfile.py", line 580, in NamedTemporaryFile
file = _io.open(dir, mode, buffering=buffering,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/tempfile.py", line 577, in opener
fd, name = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/tempfile.py", line 257, in _mkstemp_inner
fd = _os.open(file, flags, 0o600)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/label/tmpzxoa3e11'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 88, in wrapper
output = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 819, in do_make_aparc_sub
write_aparc_sub(subjid=f'sub-{self.subject}',
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 1316, in write_aparc_sub
mne.datasets.fetch_aparc_sub_parcellation(subjects_dir=subjects_dir,
File "<decorator-gen-479>", line 10, in fetch_aparc_sub_parcellation
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/site-packages/mne/datasets/utils.py", line 411, in fetch_aparc_sub_parcellation
pooch.retrieve(
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/site-packages/pooch/core.py", line 227, in retrieve
make_local_storage(path)
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/site-packages/pooch/utils.py", line 276, in make_local_storage
raise PermissionError(" ".join(message)) from error
PermissionError: [Errno 13] Permission denied: '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/label/tmpzxoa3e11' | Pooch could not write to data cache folder '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/label'. Will not be able to download data files.
Traceback (most recent call last):
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/site-packages/pooch/utils.py", line 264, in make_local_storage
with tempfile.NamedTemporaryFile(dir=path):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/tempfile.py", line 580, in NamedTemporaryFile
file = _io.open(dir, mode, buffering=buffering,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/tempfile.py", line 577, in opener
fd, name = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/tempfile.py", line 257, in _mkstemp_inner
fd = _os.open(file, flags, 0o600)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/label/tmpzxoa3e11'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/bin/process_meg.py", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 1773, in <module>
process_subject(args.subject, args) # process the single specified subject
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 1438, in process_subject
proc.do_proc_allsteps()
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 1070, in do_proc_allsteps
self.do_make_aparc_sub()
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 88, in wrapper
output = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 819, in do_make_aparc_sub
write_aparc_sub(subjid=f'sub-{self.subject}',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/alessandro.tonin@rc.fhsc.net/PycharmProjects/Enigma_MEG/enigma_MEG/enigmeg/process_meg.py", line 1316, in write_aparc_sub
mne.datasets.fetch_aparc_sub_parcellation(subjects_dir=subjects_dir,
File "<decorator-gen-479>", line 10, in fetch_aparc_sub_parcellation
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/site-packages/mne/datasets/utils.py", line 411, in fetch_aparc_sub_parcellation
pooch.retrieve(
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/site-packages/pooch/core.py", line 227, in retrieve
make_local_storage(path)
File "/home/alessandro.tonin@rc.fhsc.net/.conda/envs/enigma_meg/lib/python3.11/site-packages/pooch/utils.py", line 276, in make_local_storage
raise PermissionError(" ".join(message)) from error
PermissionError: [Errno 13] Permission denied: '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/label/tmpzxoa3e11' | Pooch could not write to data cache folder '/mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage/label'. Will not be able to download data files.
I think the errors are coming from MNE trying to download (using pooch) the template annotation (aparc-sub) that is not included with Freesurfer. The fsaverage folder will have to be writeable. Can you run the following - its a hack, but should remove this linking issue:
unlink /mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage
cp -R ${SUBJECTS_DIR}/fsaverage /mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/
chmod -R 770 /mnt/raid/atonin/bids/Protocollone/derivatives/freesurfer/subjects/fsaverage
And then rerun.
Also - sorry for all the back and forth with this. Appreciate your patience with this issue.
Hi, while running the code I'm receiving the following error:
As far as I understand the error is due to the fact that
fsaverage
directory is a symlink. I investigated a little bit and I noticed these lines inprocess_meg.py
:However in my case
fsaverage
is downloaded at the beginning of the script, and it is a new real folder as expected. I run the script putting a breakpoint after each single step ofdo_proc_allsteps
and I found out that the folder is replaced with a sym link offsaverage
installed in my computer during the stepproc_mri
.