Closed iamzoltan closed 2 months ago
This can probably be fixed by using a directory relative to the current directory for /temp
If I understand that should be fixable with an os.mkdir("/temp") before the following lines:
data_storage_directory = "/temp" # Note: this path must exist on your local drive cache = VisualBehaviorOphysProjectCache.from_s3_cache(cache_dir=data_storage_directory)
Can you check @yavorska-iryna? Thanks a lot in advance.
We have not seen this issue when we ran the notebook. Steeve's fix may work. I need to test it.
@iamzoltan I dont run into the same permission error when I test the notebook in Jupyter lab or colab. I changed the name of the folder and the code still ran, so I dont think os.makedir
would fix it. Since I can't replicate this error, it's hard for me to fix it.
Yes I understand. We are talking about the processing environment on GH. Can you change the location to ./temp
and lets see if that fixes the issue
Let me look into it.
@iamzoltan I changed the path to "./temp" and merged it to my forked branch. Let me know if that works.
I just tested the notebook on the allen-hot-fix
branch with the following change to the cache loading and it worked as expected:
data_storage_directory = "./temp" cache = VisualBehaviorOphysProjectCache.from_s3_cache(cache_dir=data_storage_directory)
A folder called 'temp' was created in the same directory as the notebook and the cache loaded properly.
Looks like the process is failing after adding in the fMRI fixes
actually I cant get the allen books to run locally. I get this error:
41 # query on valid_roi if exclude_invalid_rois == True
42 if exclude_invalid_rois:
---> 43 cell_specimen_table = ophys_experiment.cell_specimen_table.query('valid_roi').reset_index() # noqa E501
44 else:
45 cell_specimen_table = ophys_experiment.cell_specimen_table.reset_index() # noqa E501
Cell In[16], line 20, in <lambda>(self, expr, **kwargs)
18 pd.set_option('display.max_columns', 500)
19 # this line may be needed if you run into Error in pandas query function
---> 20 pd.DataFrame.query = lambda self, expr, **kwargs: self.query(expr, engine='python', **kwargs)
Cell In[16], line 20, in <lambda>(self, expr, **kwargs)
18 pd.set_option('display.max_columns', 500)
19 # this line may be needed if you run into Error in pandas query function
---> 20 pd.DataFrame.query = lambda self, expr, **kwargs: self.query(expr, engine='python', **kwargs)
TypeError: __main__.<lambda>() got multiple values for keyword argument 'engine'
@iamzoltan this line can be removed: pd.DataFrame.query = lambda self, expr, **kwargs: self.query(expr, engine='python', **kwargs)
It was added because in some instances pandas used numpy engine and query function didn't work. It can also be fixed by adding engine='python'
when using pd.query.
I am sorting it out. now we are running out of space, which is a good error. I will clear some space, and try again.
@steevelaquitaine @yavorska-iryna - its the same issue as before.