While doing testing for #175 , I've discovered that the build function is computing the minimum (and presumably the maximum) catalog version from the live catalog location on g/data, even when the build is taking place somewhere else. The system should be looking for min and max catalog versions in the build directory.
To Reproduce
Running a test build of #175 with MOM6 sources only into a blank test directory on Gadi /scratch yields the following catalog.yaml:
sources:
access_nri:
args:
columns_with_iterables:
- model
- realm
- frequency
- variable
mode: r
name_column: name
path: /scratch/tm70/mcw120/intake-cat-test/{{version}}/metacatalog.csv
yaml_column: yaml
description: ACCESS-NRI intake catalog
driver: intake_dataframe_catalog.core.DfFileCatalog
metadata:
storage: gdata/None+gdata/ik11
version: '{{version}}'
parameters:
version:
default: v2024-11-12
description: Catalog version
max: v2024-11-12
min: v2023-07-10 # HERE - This should be equal to max and default, as there were no other existing catalogs here
type: str
The min value returned corresponds to the current minimum value present in /g/data/xp65/public/apps/access-nri-intake-catalog (i.e., the live catalog directory).
Additional context
Doesn't block #175, but should be fixed ASAP. Should also include a test to prevent this from happening again.
Describe the bug
While doing testing for #175 , I've discovered that the build function is computing the minimum (and presumably the maximum) catalog version from the live catalog location on
g/data
, even when the build is taking place somewhere else. The system should be looking for min and max catalog versions in the build directory.To Reproduce
Running a test build of #175 with MOM6 sources only into a blank test directory on Gadi
/scratch
yields the followingcatalog.yaml
:The min value returned corresponds to the current minimum value present in
/g/data/xp65/public/apps/access-nri-intake-catalog
(i.e., the live catalog directory).Additional context
Doesn't block #175, but should be fixed ASAP. Should also include a test to prevent this from happening again.