search

Thomas-Moore-Creative / ACDtools

Australian Climate Data Tools
MIT License
0 stars 0 forks source link

[NCI `fs38` catalog] what does `file_type` = `f` vs `l` mean and which is the correct one to use? #2

Open Thomas-Moore-Creative opened 2 weeks ago

Thomas-Moore-Creative commented 2 weeks ago

on Intake-ESM on CMIP5/6 datasets the metadata file_type shows up but then just vanishes without explanation?

What does "l" vs "f" mean? And what to use? And why?

Thomas-Moore-Creative commented 5 days ago

response from NCI:

The ‘f’ and ‘l’ values of ‘file_type’ indicate whether the file is a regular file or a symbolic link. In some cases, a new revision may create a link to a file from a previous revision to save storage space.

Thomas-Moore-Creative commented 4 days ago

further response from NCI:

Soft links may exist in multiple versions, so we can’t rely on ‘l’ to identify the latest version, requiring several steps to determine it. I agree that the catalog file should offer a more straightforward way to access the ‘latest’ dataset version. We’re working on adding ‘latest’ to the ‘versions’ column to make it easier to filter. I’ll let you know as soon as the updated catalog file is available for you to try.

Thomas-Moore-Creative commented 4 days ago

Documented some of this here: https://forum.access-hive.org.au/t/cmip6-data-analysis-at-nci-file-type-issue/3842