radiantearth / stac-spec

SpatioTemporal Asset Catalog specification - making geospatial assets openly searchable and crawlable
https://stacspec.org
Apache License 2.0
779 stars 178 forks source link

Link rel type for links to run an example script #992

Closed schwehr closed 1 year ago

schwehr commented 3 years ago

What should the rel type be for a link that loads/runs a preexisting script on a system? I don't know if other systems have this capability, but Earth Engine does. This link will load up an example script in the Earth Engine code editor:

https://code.earthengine.google.com/?scriptPath=Examples:Datasets/NOAA_GOES_16_FDCC

Projects might want to link to a Jupyter Notebook or similar that demonstrate using a STAC catalog entry.

I don't see a rel type that is obvious to use here:

https://github.com/radiantearth/stac-spec/blob/master/catalog-spec/catalog-spec.md#relation-types

Maybe script? But what if there should be multiple links for different ways of using the STAC entry? e.g. If there was also an example notebook for noaa goes that would get the user going using python in Jupyter and another for julia in Jypter?

{
  // SNIP
  "links": [
    {
      "rel": "self",
      "href": "https://earthengine-stac.storage.googleapis.com/catalog/NOAA_GOES_16_FDCC.json"
    },
    {
      "rel": "parent",
      "href": "https://earthengine-stac.storage.googleapis.com/catalog/catalog.json"
    },
    {
      "rel": "script",  // <--
      "href": "https://code.earthengine.google.com/?scriptPath=Examples:Datasets/NOAA_GOES_16_FDCC"
    },
    // SNIP
  ],
  // SNIP
}

So people can see what it looks like on Earth Engine if they don't have an account:

Screen Shot 2021-02-16 at 10 42 32 AM

m-mohr commented 3 years ago

We just used the rel type related with a title:

    {
      "title": "Run example on Google Playground",
      "rel": "related",
      "href": "https://code.earthengine.google.com/?scriptPath=Examples:Datasets/NOAA_GOES_16_FDCC"
    },

I think titles are more useful than rel-types for users. Most clients don't handle non-core rel types anyway and thus for the user-experience it is more useful to provide good titles. (I'm not opposed to a rel type, but I'm not sure it belongs into the core spec.)

schwehr commented 3 years ago

I have yet to implement this for Earth Engine. It would be nice to have a way to specify what programming language is going to be at the other end of the link. We currently have just JavaScript examples, but there may be Python examples in the future.

m-mohr commented 3 years ago

We just ran into this issue as well in the last days.

Some thoughts:

  1. There are multiple types of examples that should be handled differently: A. Examples you can run using a web page directly, e.g. https://code.earthengine.google.com/?scriptPath=Examples:Datasets/NOAA_GOES_16_FDCC B. Just code, e.g. a Python script.
  2. Exposing different languages could be useful (we have Python, R and JS). While for B you can just try to use media types (e.g. application/javascript), for A it's usually text/html due to the website running the script. Would it help to add a field "environment" or "language" to the link? Or are title enough in that case?
schwehr commented 3 years ago

The division in 1. sounds right. One question... if the link goes to a web hosted jupyter notebook, is that the case A or B? And some links to notebooks are active things (e.g. google colab, while others are just static renderings.

For 2. above, I am guessing that users are going to want to be able to filter on language(s), so maybe a separate field for language? I envision someone creating an app that provides a space to execute scripts and they will want to be able to populate the execution environment with the contents at the end of that link. That's what the Earth Engine code editor is doing behind the scenes then it gets the scriptPath.

m-mohr commented 3 years ago

The division in 1. sounds right. One question... if the link goes to a web hosted jupyter notebook, is that the case A or B?

A, it's also text/html as media type for them. For B it's really only plain scripts (.py, .js).

The difference is that clients may want to request the files for B and display them nicely with code highlighting. For A, you can't request the pure script and it's just a link people can follow.

And some links to notebooks are active things (e.g. google colab, while others are just static renderings.

I don't think we need to distinguish them, or?

m-mohr commented 3 years ago

Potential example @schwehr:

var links = [
  // Type A
  {
    "title": "Run NOAA_GOES_16_FDCC example on Google Playground",
    "rel": "related",
    "type": "text/html",
    "code": "JavaScript", // We need to figure out a definitive list of languages, see also https://twitter.com/james_santucci/status/1374711136910770176
    "href": "https://code.earthengine.google.com/?scriptPath=Examples:Datasets/NOAA_GOES_16_FDCC"
  },
  // Type B
  {
    "title": "NOAA_GOES_16_FDCC example",
    "rel": "code",
    "type": "application/javascript",
    "code": "JavaScript",
    "href": "https://code.earthengine.google.com/codes/examples/datasets/NOAA_GOES_16_FDCC.js"
  }
];
schwehr commented 3 years ago

Maybe https://github.com/github/linguist/blob/master/lib/linguist/languages.yml? It's what GitHub uses for the language string in markdown. Definitely open to other sources of language definitions. Linquist is the only one I know of.

JavaScript:
  type: programming
  tm_scope: source.js
  ace_mode: javascript
  codemirror_mode: javascript
  codemirror_mime_type: text/javascript
  color: "#f1e05a"
  aliases:
  - js
  - node
  extensions:
  - ".js"
  - "._js"
// SNIP

and

Python:
  type: programming
  tm_scope: source.python
  ace_mode: python
  codemirror_mode: python
  codemirror_mime_type: text/x-python
  color: "#3572A5"
  extensions:
  - ".py"
  - ".cgi"
  - ".fcgi"
// SNIP
cholmes commented 3 years ago

+1 for related, with proper types and descriptive titles.

schwehr commented 3 years ago

My first go at the links to the Code Editor / Playground: https://storage.googleapis.com/earthengine-stac-experimental/catalog-v018/catalog.json

stac-browser

rocket

jjrom commented 3 years ago

@schwehr Hi Kurt - thanks for point this out ! I published a new rocket version with the issue corrected. Be sure to clear the cache before reload the client. See here https://rocket.snapplanet.io/map?u=https:%2F%2Fearthengine-stac-experimental.storage.googleapis.com%2Fcatalog-v018%2Fcatalog.json&c=0,0,2.55&f=%7B%22__ep%22:%7B%7D%7D

schwehr commented 3 years ago

@jjrom Awesome! That works great.

m-mohr commented 3 years ago

We may have a clash related to the relation types. In this issue https://github.com/radiantearth/stac-browser/issues/50 it's discussed to use related for catalog/item links. @emmanuelmathot @ycespb

I guess we should try to have distinct rel types as the use cases here are completely different and may lead to issues in implementations. Or we need to clearly document somewhere how to distinguish the use cases.

jjrom commented 3 years ago

@m-mohr @emmanuelmathot @ycespb I also use the rel=related relation type to mimic SKOS relations (https://www.w3.org/2009/08/skos-reference/skos.html)

schwehr commented 3 years ago

I'm happy to go with what ever guidance folks have.

@jjrom, I'm also totally new to SKOS. If there is a controlled vocabulary for tags, I'd be interested and let's start a separate bug. The Earth Engine tags have grown organically and already has clashes like gfw meaning either Global Fishing Watch or Global Forest Watch.

emmanuelmathot commented 3 years ago

We are limited by using only the rel attribute. A link must also rely on the content-type of the target. A content-ype is not only composed of a IANA convention string (e.g. application/geo+json) but also parameters such as profile, subtype... That is the purpose of an previous issue I raised some times ago about specific profiled types for stac items. This profiled type allows the software to know in advance the expected content of the link and render the UI or the functions accordingly. There is maybe already a specific mime type for GEE scripts (e.g. application/vnd.google-apps.gee)

schwehr commented 3 years ago

There is not an application media type for GEE code editor scripts. After some research, it doesn't seem that there should be one.

jjrom commented 3 years ago

@schwehr SKOS is a common data model for sharing and linking knowledge organization. In particular it defines how to link concepts between them with the notion of "broader", "narrower" and "related" relations. From a STAC point of view a "broader" relation is similar to a "parent" relation and a "narrower" relation to a "child" relation. For the remaining relation, I used rel="related". You can try this endpoint to have a better view of this approach - https://rocket.snapplanet.io/map?u=http:%2F%2Fdataterra.snapplanet.io (click on the "Concepts" catalog)

schwehr commented 3 years ago

I don't understand what I'm looking at with the concepts. e.g. I see OCEAN, but that doesn't see how that connects to other things. I did see this link in one asset as an example... https://w3id.org/ozcar-theia/surfaceWater I think I need a good tutorial on SKOS. It's used internally in Google, but it's a big investment to come up to speed on it.

m-mohr commented 3 years ago

Me neither, but from what I read here it seems like it could be a good idea to use another rel type for the examples. Probably something easy as "example" and then with an additional code property as discussed. Would be great to write this up into a very minimal extension or so.

cboettig commented 1 year ago

Thanks @m-mohr for pointing me to this thread. I agree that SKOS "example" property seems reasonable here, e.g. rel="http://www.w3.org/2008/05/skos#example". Some colleagues deeply involved in academic linked data in earth sciences also suggested a few other properties defined in RDF vocabularies depending on the intent, e.g. http://purl.org/spar/cito/isDocumentedBy.

Schema.org also defines the property/inverse property https://schema.org/workExample and https://schema.org/exampleOfWork

m-mohr commented 1 year ago

I've created a draft proposal for an "example links" extension: https://github.com/stac-extensions/example-links Please post your comments, feedback, ... in the corresponding issue tracker. I'm not really into all the related specs like SKOS, RDF etc, so haven't taken them into account, but happy to accept PRs etc.