NeuromatchAcademy / course-content

NMA Computational Neuroscience course
https://compneuro.neuromatch.io
Creative Commons Attribution 4.0 International
2.65k stars 983 forks source link

Dataset hot fix #1109

Closed iamzoltan closed 2 months ago

iamzoltan commented 2 months ago

@AdiBish

we are getting some odd errors while trying to process your changes.

------------------
data_storage_directory = "/temp"  # Note: this path must exist on your local drive
cache = VisualBehaviorOphysProjectCache.from_s3_cache(cache_dir=data_storage_directory)
============================== Failure ==============================
------------------
---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
Cell In[3], line 2
      1 data_storage_directory = "/temp"  # Note: this path must exist on your local drive
----> 2 cache = VisualBehaviorOphysProjectCache.from_s3_cache(cache_dir=data_storage_directory)
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/brain_observatory/behavior/behavior_project_cache/project_cache_base.py:77, in ProjectCacheBase.from_s3_cache(cls, cache_dir, bucket_name_override)
     50 @classmethod
     51 def from_s3_cache(
     52         cls,
     53         cache_dir: Union[str, Path],
     54         bucket_name_override: Optional[str] = None
     55 ) -> "ProjectCacheBase":
     56     """instantiates this object with a connection to an s3 bucket and/or
     57     a local cache related to that bucket.
     58 
   (...)
     74 
     75     """
---> 77     fetch_api = cls.cloud_api_class().from_s3_cache(
     78 cache_dir,
     79 bucket_name=(
     80 bucket_name_overrideifbucket_name_overrideisnotNone
     81 elsecls.BUCKET_NAME),
     82 project_name=cls.PROJECT_NAME,
     83 ui_class_name=cls.__name__)
     85     return cls(fetch_api=fetch_api)
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/brain_observatory/behavior/behavior_project_cache/project_apis/data_io/project_cloud_api_base.py:108, in ProjectCloudApiBase.from_s3_cache(cls, cache_dir, bucket_name, project_name, ui_class_name)
     78 @classmethod
     79 def from_s3_cache(cls, cache_dir: Union[str, Path],
     80                   bucket_name: str,
     81                   project_name: str,
     82                   ui_class_name: str) -> "ProjectCloudApiBase":
     83     """instantiates this object with a connection to an s3 bucket and/or
     84     a local cache related to that bucket.
     85 
   (...)
    106 
    107     """
--> 108     cache = S3CloudCache(cache_dir,
    109 bucket_name,
    110 project_name,
    111 ui_class_name=ui_class_name)
    112     return cls(cache)
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/api/cloud_cache/cloud_cache.py:1066, in S3CloudCache.__init__(self, cache_dir, bucket_name, project_name, ui_class_name)
   1063 self._manifest = None
   1064 self._bucket_name = bucket_name
-> 1066 super().__init__(cache_dir=cache_dir,project_name=project_name,
   1067 ui_class_name=ui_class_name)
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/api/cloud_cache/cloud_cache.py:391, in CloudCacheBase.__init__(self, cache_dir, project_name, ui_class_name)
    390 def __init__(self, cache_dir, project_name, ui_class_name=None):
--> 391     super().__init__(cache_dir=cache_dir,project_name=project_name,
    392 ui_class_name=ui_class_name)
    394     # what latest_manifest was the last time an OutdatedManifestWarning
    395     # was emitted
    396     self._manifest_last_warned_on = None
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/allensdk/api/cloud_cache/cloud_cache.py:63, in BasicLocalCache.__init__(self, cache_dir, project_name, ui_class_name)
     57 def __init__(
     58     self,
     59     cache_dir: Union[str, Path],
     60     project_name: str,
     61     ui_class_name: Optional[str] = None
     62 ):
---> 63     os.makedirs(cache_dir,exist_ok=True)
     65     # the class users are actually interacting with
     66     # (for warning message purposes)
     67     if ui_class_name is None:
File /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/os.py:225, in makedirs(name, mode, exist_ok)
    223         return
    224 try:
--> 225     mkdir(name,mode)
    226 except OSError:
    227     # Cannot rely on checking for EEXIST, since the operating system
    2[28](https://github.com/NeuromatchAcademy/course-content/actions/runs/9813753170/job/27100280023#step:10:29)     # could give priority to other errors like EACCES or EROFS
    229     if not exist_ok or not path.isdir(name):
PermissionError: [Errno 13] Permission denied: '/temp'
Error: Process completed with exit code 1.

I do think this is unrelated to your changes, but it is preventing our pipeline from running. Any ideas?

marius10p commented 2 months ago

Let's wait on this one. The original creator of the notebook is supposed to upload a fixed version very soon...

steevelaquitaine commented 2 months ago

@iamzoltan @marius10p @yavorska-iryna, I think that @yavorska-iryna has addressed the issues in her pull request (if she is the original creator of the notebook you mentioned @marius10p). @iamzoltan, do you think we can now accept the pull request? What about the other pull requests ?

iamzoltan commented 2 months ago

Alright, I will close this PR.