ONE params get cache dir failing if old parameters are no longer accessible

[seankmartin]

Hi,

Summary

I had the following situation.

1. Use ONE with some data on a temporary drive (e.g. a USB or mounted drive).
2. Remove said temporary drive, and try to use ONE from a new location.
3. Run into a crash because in one.params.get_cache_dir() and one.params.setup(silent=True) try to call mkdir on an inaccessible directory.

Cache retrieval crashing

I've checked the parameters cache file (HOME\AppData\Roaming\.one) and it contains the following:

{
    "CLIENT_MAP": {
        "openalyx.internationalbrainlab.org": "D:\\example-data\\ibl-data\\openalyx.internationalbrainlab.org"
    },
    "DEFAULT": "openalyx.internationalbrainlab.org"
}

D is the inaccessible directory that I was previously using. So calling one.params.get_cache_dir() will crash as it calls mkdir on the CLIENT_MAP value. I don't think it should, I'd expect the code to return the currently inaccessible directory without crashing.

Resolving the issue

I can call one.setup(silent=False) to fix the parameter file. However, it was a bit hard to figure out what the issue was in the first place, because the stack trace for the mkdir error is quite confusing.

Environment information

I'm on Windows 10 with python 3.8 and ibllib==2.19.0, one-api==1.17.0

[k1o0]

Hello, thanks for taking the time to create an issue on this! I agree the error should be informative and easily traceable. It's unclear to me whether it should instantiate without any error though. At some point an error in this situation is unavoidable. How and when would you want to be made aware that the drive is inaccessible?

[seankmartin]

Hey @k1o0,

For sure, I badly worded that in hindsight. I'd also expect a crash to happen at some point. However, I'd expect something like this:

1. Call one.params.get_cache_dir() or one.params.setup(silent=True).
2. The code would crash because the original parameter file is not available, but caught by a try except around the mkdir.
3. An error message is shown, something like "The current cache directory of {value} set in the ONE parameter file at {location} is not accessible. You can establish a new cache directory by modifying {location} or create a new parameter file by calling one.setup(silent=False)".
4. The code crashes as before, but now it is clear how to proceed.

3 could be a simple error message, or throwing a custom exception etc.

What do you think? Does that seem reasonable?

Cheers!

[k1o0]

Are you happy with this?

from one.api import ONE
one = ONE(cache_dir='I:\\Downloads', base_url='https://openalyx.internationalbrainlab.org')

FileNotFoundError: Cache directory not accessible: I:\Downloads
Please provide valid tables_dir / cache_dir kwargs or run ONE.setup to update the default directory.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:/Users/User/AppData/Roaming/JetBrains/PyCharm2021.2/scratches/scratch_32.py", line 2, in <module>
    one = ONE(cache_dir='I:\\Downloads', base_url='https://openalyx.internationalbrainlab.org')
  File "C:\Users\User\Documents\Github\ONE\one\api.py", line 1347, in ONE
    return OneAlyx(mode=mode, wildcards=wildcards, **kwargs)
  File "C:\Users\User\Documents\Github\ONE\one\api.py", line 1398, in __init__
    super(OneAlyx, self).__init__(
  File "C:\Users\User\Documents\Github\ONE\one\api.py", line 76, in __init__
    self.load_cache()
  File "C:\Users\User\Documents\Github\ONE\one\api.py", line 1504, in load_cache
    raise ex from FileNotFoundError(
  File "C:\Users\User\Documents\Github\ONE\one\api.py", line 1453, in load_cache
    cache_info = self.alyx.get(f'cache/info/{tag or ""}'.strip('/'), expires=True)
  File "C:\Users\User\Documents\Github\ONE\one\webclient.py", line 879, in get
    rep = self._generic_request(requests.get, rest_query, **kwargs)
  File "C:\Users\User\Documents\Github\ONE\one\webclient.py", line 133, in wrapper_decorator
    rest_cache.mkdir(parents=True)
  File "C:\Users\User\Anaconda3\envs\ONE\lib\pathlib.py", line 1291, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "C:\Users\User\Anaconda3\envs\ONE\lib\pathlib.py", line 1291, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "C:\Users\User\Anaconda3\envs\ONE\lib\pathlib.py", line 1287, in mkdir
    self._accessor.mkdir(self, mode)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'I:\\'

[seankmartin]

Looks great, thanks so much @k1o0 !