LariDG
commented
2 weeks ago Windows 11
I have set up a Windows 11 VM with UTM. When using the new version of level1 the cache goes to:
C:\\Users\\larisa\\AppData\\Local\\Temp
the file that was added: tmppz185vqr
When I use the current config/level1 I can't find where the cache goes and hence I imagine it goes to RAM as it can't follow the default cache path and is using simplecache as default. However, when trying to do it from mastapp.site it didn't have the SSL verification. I added the website's certificate to the windows system and that did the job, as I am not sure if this is a windows thing or a virtual machine thing.
Mac OS 5.0.1 | M3 chip
With the new level1 the cache happens in:
/var/folders/rg/35kdzt3j3nldj_1bkcy76_gw0000gp/T
which is the default cache folder, you can check yours via:
import tempfile
tempfile.gettempdir()where it should be the same but a different random string.
the file that was added was a similar looking file to the one in windows: tmphlf2e46y
The current config goes to /tmp as instructed by the yaml but has a different name, a longer random string: 6927a9a4251a9626a60b960b9eeb79f82950c29d893ed8e5f13486086499476
To solve issues: #73 and #74
Issue is where windows users fails due to the catalog file, which takes in level1.yml setting cache location to /tmp. Assumption was made by group that taking out cache_dir from yaml file would make os choose it's default temp file. Test by taking out and seeing it cache is working. When loaded first, takes some time, when loaded second is instantaneous. This worked for me on Mac OS, need to test with Windows OS to ensure the same results.
Way I tested:
catalog_local = intake.open_catalog('src/api/static/intake/catalog.yml')
url = "s3://mast/level1/shots/30467.zarr/amc" dataset_local = catalog_local.level1.sources(url=url) dataset_local.read()
All fairly quick, slight delay on the dataset.read()
After doing again, dataset_local.read() and dataset_local.to_dask()
Happened instantaneously, hence cache is fine.
Compared to normal way with mastapp catalog files:
catalog_mastapp = intake.open_catalog('https://mastapp.site/intake/catalog.yml')
dataset_mastapp = catalog_mastapp.level1.sources(url = url) dataset_mastapp.read()
All happened the same as local, fairly quick apart from .read()
After doing again, dataset_mastapp.read() and dataset_mastapp.to_dask()
Happened instantaneously, exactly the same as the local version.