STAC Implementation

[Alexander-Barth]

I have some early prototype implementation of the STAC spec (SpatioTemporal Asset Catalog, https://stacspec.org/) which allows to make a catalog of data products (such as model data or satellite data). The STAC spec is relatively large (the python implementation pystac has about 14 000 lines of code excluding tests). I think it would be good to have it directly in a Julia organization to channel efforts. Would it be OK to host such an experimental repo here?

What works so far is traversing the catalog hierarchy, recursively searching all items and sub-catalogs and getting the URLs of the individual assets (i.e. URLs to data files).

Here is an initial discussion: https://discourse.julialang.org/t/ann-geodata-jl/45502/39

As a name I would propose STACatalogs (as STAC is too short). If you know about any available Julia implementation of the STAC spec I would be glad to hear about it.

[Alexander-Barth]

I put my preliminary implementation here: https://github.com/Alexander-Barth/STACatalogs.jl

I still think it would be the best to transfer it to an organization. Maybe some of you are interested: @juliohm @visr @gaelforget @natgeo-wong

[visr]

Nice work @Alexander-Barth! STAC looks like a great new spec. I've been aware of its development, but haven't used it yet so far. But it's probably a matter of time until I need it, so great to have STACatalogs.jl. JuliaClimate could be a fine home for it, though since the spec is for geospatial data in general, perhaps JuliaGeo would also be good?

[Alexander-Barth]

JuliaGeo could be fine too, but I would expect to find more users (= potential future contributors :-)) of the STAC protocol among JuliaClimate (than JuliaGeo) given the type of data distributed (satellite data, atmospheric/ocean model data) which seem not to be quite in the scope of JuliaGeo.

There are already quite similar projects here in JuliaClimate as far as I know: ClimateSatellite.jl, INMET.jl, CDSAPI.jl . So in case that one of these data providers would adopt the STAC protocol, it could simplify the adoption.

(but sometimes I am asking myself, if we should not join different organizations and widening the scopes of the organizations :-) )

[juliohm]

I agree with Alexander that JuliaClimate seems like a perfect fit. Awesome contribution as usual.

On Thu, Oct 14, 2021, 05:48 Alexander Barth @.***> wrote:

JuliaGeo could be fine too, but I would expect to find more users (= potential future contributors :-)) of the STAC protocol among JuliaClimate (than JuliaGeo) given the type of data distributed (satellite data, atmospheric/ocean model data) which seem not to be quite in the scope of JuliaGeo https://github.com/ax1ine/GSW.jl/issues/7#issuecomment-557767800.

There are already quite similar projects here in JuliaClimate as far as I know: ClimateSatellite.jl, INMET.jl, CDSAPI.jl . So in case that one of these data provider would adopt the STAC protocol, it could simplify the adoption.

(but sometimes I am asking myself, if we should not join different organizations and widening the scopes of the organizations :-) )

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JuliaClimate/meta/issues/15#issuecomment-943148422, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZQW3NTCBALY5RW3TTKHU3UG2KOPANCNFSM5FYMFARQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[visr]

Ah yes I understand. The link to the comment on the scope of JuliaGeo you sent may be confusing though. In general you can read it as "geospatial". I added a bit of text to https://juliageo.org/, see "GitHub organizations", to attempt to clarify scopes.

[juliohm]

@Alexander-Barth did you consider using the shorter acronym STAC.jl for the package? I know acronyms aren't the standard in Julia but these datasets and catalogs like INMET.jl, CDSAPI.jl, GADM.jl all follow this acronym convention.

[Alexander-Barth]

@visr Thanks for the clarification. @juliohm Yes, STAC.jl would certainty be nicer. So far we have the type STACatalog, but there should also be a collection type. Naming it STACCollection does not seem to be be quite consistent. STAC.Catalog and STAC.Collection (module + unexported type) would be indeed much nicer.

[gaelforget]

Nice! I did not know of STAC but this seems very useful and a great fit indeed for JuliaClimate (time dimension vs geo). Regarding the name STACatalogs.jl or STACatalog.jl (without the s) might seem best to me.

[Balinus]

Nice!

Regarding name, it is hard to tell. I think I prefer STACatalogs.jl : At least it gives me an indication that it is a catalog of something but on the other hand, we use seamlessly NetCDF.jl and not NetworkCommonDataForm.jl 😄

For hosting, I also think that JuliaClimate and JuliaGeo has an overlap. More like JuliaClimate is somewhat an application of JuliaGeo anmd perhaps someday they will merge, who knows.