leap-stc / LEAP_template_feedstock

A Prototype feedstock that implements independent metadata and data updates using pangeo forge
Apache License 2.0
1 stars 2 forks source link

Remove catalog zarr logic #19

Closed jbusecke closed 3 months ago

jbusecke commented 3 months ago

As discussed with @andersy005 earlier I will remove all the logic to create a separate 'catalog zarr' here and we will work with the feedstock/catalog.yaml instead.

andersy005 commented 3 months ago

@jbusecke, i finally got a chance to look at this and and i'm considering the changes necessary to align it with https://github.com/leap-stc/data-management/pull/98

catalog.yaml:

"ncviewjs:meta_yaml_url": "https://github.com/leap-stc/proto_feedstock/blob/main/feedstock/meta.yaml"
tags:
  - my-custom-tag
  - zarr
stores:
  - name: small.zarr
    url: "gs://leap-scratch/jbusecke/proto_feedstock/latest/data/small.zarr"
    "ncviewjs:rechunking":
      - path: "gs://some-bucket/small.zarr"
        use_case: "multiscales"

  - name: large.zarr # no pyramids
    url: "gs://leap-scratch/jbusecke/proto_feedstock/latest/data/large.zarr"

i'm curious to hear your feedback on these changes.

jbusecke commented 3 months ago

That seems perfectly sensible to me.

Let me modify this accordingly.

andersy005 commented 3 months ago

great... @jbusecke, let me know when this ready

jbusecke commented 3 months ago

Ok I just ran the recipe and produced this store:

jbusecke commented 3 months ago
image
jbusecke commented 3 months ago

@andersy005 Ready to go from my end. Do you prefer I merge this or is there some reason to wait until we have tested the catalog?

andersy005 commented 3 months ago

we are missing maintainers in meta.yaml

maintainers:
  - name: "Julius Busecke"
    orcid: "0000-0001-8571-865X"
    github: jbusecke
andersy005 commented 3 months ago

Ready to go from my end. Do you prefer I merge this or is there some reason to wait until we have tested the catalog?

@jbusecke, once we've added maintainers, feel free to merge.

jbusecke commented 3 months ago

Ah nice. Wondering if we can port some of the schema validation to this repo (which will be a template going forward).