As a user, I want to search for past versions of a product where the LID changed during the product history

jordanpadams commented 2 years ago

💪 Motivation

...so that I can browse through the provenance of a product

📖 Additional Details

Per the parent epic, and the design we need to update the API to implement these changes so we can sufficiently understand the version history.

For example, Product A changes its logical identifier from version "1" of the product (lid_x) to version "2" (lid_y), we want to know that these are actually revisions of the same product.

NOTE: The versions may not be in appropriate ascending order, e.g. you could have lid_x::1.0 and lid_y::1.0, but really lid_x::1 == Product A v1 and lid_y::1.0 == Product A v2

⚖️ Acceptance Criteria

Given the context products urn:nasa:pds:context:instrument:crs.vg1::1.0 and urn:nasa:pds:context:instrument:vg1.crs::1.0 ingested into the registry When I perform a query of the API for products/urn:nasa:pds:context:instrument:crs.vg1/all Then I expect to see 2 records returned for urn:nasa:pds:context:instrument:crs.vg1::1.0 and urn:nasa:pds:context:instrument:vg1.crs::1.0

Given the context products urn:nasa:pds:context:instrument:crs.vg1::1.0 and urn:nasa:pds:context:instrument:vg1.crs::1.0 ingested into the registry When I perform a query of the API for products/urn:nasa:pds:context:instrument:vg1.crs/all Then I expect to see 2 records returned for urn:nasa:pds:context:instrument:crs.vg1::1.0 and urn:nasa:pds:context:instrument:vg1.crs::1.0

Here are some more specific examples, let's say we have: urn:nasa:pds:context:instrument:crs.vg1::1.0 (actually version 1 of this product) urn:nasa:pds:context:instrument:vg1.crs::1.0 (actually version 2 of this product) urn:nasa:pds:context:instrument:vg1.crs::2.0 (actually version 3 of this product)

As a user, when I search for /products/urn:nasa:pds:context:instrument:vg1.crs I will get urn:nasa:pds:context:instrument:vg1.crs::2.0 (API defaults to latest version)
As a user, when I search for /products/urn:nasa:pds:context:instrument:crs.vg1 I will get urn:nasa:pds:context:instrument:vg1.crs::2.0 (API defaults to latest version)
As a user, when I search /products/urn:nasa:pds:context:instrument:crs.vg1/latest, I want to get back urn:nasa:pds:context:instrument:vg1.crs::2.0
As a user, when I search /products/urn:nasa:pds:context:instrument:crs.vg1/all, I want to get back all 3 versions (ordered by creation date time).
As a user, when I search /products/urn:nasa:pds:context:instrument:crs.vg1::1.0, I want to get back 1 product urn:nasa:pds:context:instrument:crs.vg1::1.0

⚙️ Engineering Details

jordanpadams commented 1 year ago

merged per https://github.com/NASA-PDS/registry-mgr/pull/46

jordanpadams commented 1 year ago

this was not actually closed out. this is only supported via registry, not the API

tloubrieu-jpl commented 1 year ago

Discussed at the management council:

Ann: should be handled as superseded relationship
Mark Bentley: we have cases like this where LIDs include times (bad idea, I know) and SPICE kernel updates changed the times. So how do we tell the registry that two products are connected?
alias could be used or reference_list

matthewtiscareno commented 1 year ago

It is very distressing to learn that examples of this issue exist. The first job of a LID is never to change. We have some formation rules for LIDs, but we need to make it more clear for future cases that those formation rules do not take precedence over the more important rule that a LID should never change.

How many superseded LIDs are there? The issue report mentions only one. How easy would it be to obtain a complete list?

Furthermore, how easy would it be to obtain a list of the data products that use superseded LIDs?

Most importantly, we need an SCR to make it more clear for future cases that any LID formation rules do not take precedence over the more important rule that a LID should never change. This may involve a change to the Standards Reference and possibly other things as well. Comments regarding this?

rchenatjpl commented 1 year ago

We've only changed (deprecated then replaced) LIDs that 1) have incorrect format based on the current formation rules, and 2) have no known referrers. The full list of deprecated LIDs (252 total):

urn:nasa:pds:context:facility:observatory.ctio
urn:nasa:pds:context:facility:observatory.eso
urn:nasa:pds:context:facility:observatory.irtf
urn:nasa:pds:context:facility:observatory.mauna_kea
urn:nasa:pds:context:instrument:a-star.clem1
urn:nasa:pds:context:instrument:acp.hp
urn:nasa:pds:context:instrument:alice.nh
urn:nasa:pds:context:instrument:alice.ro
urn:nasa:pds:context:instrument:apxs.mpfr
urn:nasa:pds:context:instrument:asi.gp
urn:nasa:pds:context:instrument:aspera-3.mex
urn:nasa:pds:context:instrument:b-star.clem1
urn:nasa:pds:context:instrument:cfi.con
urn:nasa:pds:context:instrument:cida.con
urn:nasa:pds:context:instrument:cida.sdu
urn:nasa:pds:context:instrument:cosima.ro
urn:nasa:pds:context:instrument:cospin-at.uly
urn:nasa:pds:context:instrument:cospin-het.uly
urn:nasa:pds:context:instrument:cospin-hft.uly
urn:nasa:pds:context:instrument:cospin-ket.uly
urn:nasa:pds:context:instrument:cospin-let.uly
urn:nasa:pds:context:instrument:crat.lro
urn:nasa:pds:context:instrument:crispimag.con
urn:nasa:pds:context:instrument:crispspec.con
urn:nasa:pds:context:instrument:crs.vg1
urn:nasa:pds:context:instrument:crs.vg2
urn:nasa:pds:context:instrument:ctio.2mass1m3.2masssouth
urn:nasa:pds:context:instrument:ctio.smarts1m0.fruttispec
urn:nasa:pds:context:instrument:ctio.smarts1m5.cassegrainspec
urn:nasa:pds:context:instrument:ctio.soar4m1.ghts
urn:nasa:pds:context:instrument:dfmi.sdu
urn:nasa:pds:context:instrument:did.gio
urn:nasa:pds:context:instrument:disr.hp
urn:nasa:pds:context:instrument:dlre.lro
urn:nasa:pds:context:instrument:ducma.vega1
urn:nasa:pds:context:instrument:ducma.vega2
urn:nasa:pds:context:instrument:dwe.gp
urn:nasa:pds:context:instrument:dwe.hp
urn:nasa:pds:context:instrument:dynsci.sdu
urn:nasa:pds:context:instrument:epa.gio
urn:nasa:pds:context:instrument:epac.uly
urn:nasa:pds:context:instrument:epas.ice
urn:nasa:pds:context:instrument:epd.go
urn:nasa:pds:context:instrument:epi.gp
urn:nasa:pds:context:instrument:epps.mess
urn:nasa:pds:context:instrument:eso.3m6.timmi2
urn:nasa:pds:context:instrument:eso.ntt.efosc2
urn:nasa:pds:context:instrument:eso.ntt.eso_multi
urn:nasa:pds:context:instrument:eso.ntt.eso_rild
urn:nasa:pds:context:instrument:eso.ntt.sofi
urn:nasa:pds:context:instrument:eso.photometric1m.denis
urn:nasa:pds:context:instrument:esp.suisei
urn:nasa:pds:context:instrument:fpa.iras
urn:nasa:pds:context:instrument:gas.uly
urn:nasa:pds:context:instrument:gcms.hp
urn:nasa:pds:context:instrument:gdds.go
urn:nasa:pds:context:instrument:giada.ro
urn:nasa:pds:context:instrument:gpms.gp
urn:nasa:pds:context:instrument:grav.jno
urn:nasa:pds:context:instrument:grb.uly
urn:nasa:pds:context:instrument:gre.gio
urn:nasa:pds:context:instrument:grs.ody
urn:nasa:pds:context:instrument:gwe.uly
urn:nasa:pds:context:instrument:had.gp
urn:nasa:pds:context:instrument:hasi.hp
urn:nasa:pds:context:instrument:hic.go
urn:nasa:pds:context:instrument:hires.clem1
urn:nasa:pds:context:instrument:hiscale.uly
urn:nasa:pds:context:instrument:hmc.gio
urn:nasa:pds:context:instrument:hrii.dif
urn:nasa:pds:context:instrument:hriv.dif
urn:nasa:pds:context:instrument:hrsc.mex
urn:nasa:pds:context:instrument:hstacs.hst
urn:nasa:pds:context:instrument:huygens_hk.hp
urn:nasa:pds:context:instrument:ici.ice
urn:nasa:pds:context:instrument:ids.ds1
urn:nasa:pds:context:instrument:iks.vega1
urn:nasa:pds:context:instrument:imf.sakig
urn:nasa:pds:context:instrument:ims.gio
urn:nasa:pds:context:instrument:iris.vg1
urn:nasa:pds:context:instrument:iris.vg2
urn:nasa:pds:context:instrument:irtf.3m0.cvf
urn:nasa:pds:context:instrument:irtf.3m0.mirac
urn:nasa:pds:context:instrument:irtf.3m0.mirac2
urn:nasa:pds:context:instrument:irtf.3m0.mirli
urn:nasa:pds:context:instrument:irtf.3m0.mirlin
urn:nasa:pds:context:instrument:irtf.3m0.mirsi
urn:nasa:pds:context:instrument:irtf.3m0.primo_1
urn:nasa:pds:context:instrument:issn.vg1
urn:nasa:pds:context:instrument:issn.vg2
urn:nasa:pds:context:instrument:issw.vg1
urn:nasa:pds:context:instrument:issw.vg2
urn:nasa:pds:context:instrument:its.dii
urn:nasa:pds:context:instrument:jpa.gio
urn:nasa:pds:context:instrument:lamp.lro
urn:nasa:pds:context:instrument:lecp.vg1
urn:nasa:pds:context:instrument:lecp.vg2
urn:nasa:pds:context:instrument:leisa.nh
urn:nasa:pds:context:instrument:lend.lro
urn:nasa:pds:context:instrument:lgrs-a.grail-a
urn:nasa:pds:context:instrument:lgrs-b.grail-b
urn:nasa:pds:context:instrument:lidar.clem1
urn:nasa:pds:context:instrument:lola.lro
urn:nasa:pds:context:instrument:lorri.nh
urn:nasa:pds:context:instrument:lrd.gp
urn:nasa:pds:context:instrument:lroc.lro
urn:nasa:pds:context:instrument:lwir.clem1
urn:nasa:pds:context:instrument:lwp.iue
urn:nasa:pds:context:instrument:lwr.iue
urn:nasa:pds:context:instrument:mag.gio
urn:nasa:pds:context:instrument:mag.go
urn:nasa:pds:context:instrument:mag.ice
urn:nasa:pds:context:instrument:mag.m10
urn:nasa:pds:context:instrument:mag.vg1
urn:nasa:pds:context:instrument:mag.vg2
urn:nasa:pds:context:instrument:marsis.mex
urn:nasa:pds:context:instrument:mauna_kea.2m24.snifs
urn:nasa:pds:context:instrument:mauna_kea.2m24.tinsley_phot
urn:nasa:pds:context:instrument:mauna_kea.2m24.uht2kccd
urn:nasa:pds:context:instrument:mcdonald.smith2m7.p45a_phot
urn:nasa:pds:context:instrument:micas.ds1
urn:nasa:pds:context:instrument:midas.ro
urn:nasa:pds:context:instrument:miro.ro
urn:nasa:pds:context:instrument:mischa.vega1
urn:nasa:pds:context:instrument:mischa.vega2
urn:nasa:pds:context:instrument:mrflro.lro
urn:nasa:pds:context:instrument:mri.dif
urn:nasa:pds:context:instrument:mrs.mex
urn:nasa:pds:context:instrument:mvic.nh
urn:nasa:pds:context:instrument:navcam.ro
urn:nasa:pds:context:instrument:navcam.sdu
urn:nasa:pds:context:instrument:nep.gp
urn:nasa:pds:context:instrument:nfr.gp
urn:nasa:pds:context:instrument:ngims.con
urn:nasa:pds:context:instrument:nims.go
urn:nasa:pds:context:instrument:nir.clem1
urn:nasa:pds:context:instrument:omega.mex
urn:nasa:pds:context:instrument:ope.gio
urn:nasa:pds:context:instrument:osinac.ro
urn:nasa:pds:context:instrument:osiwac.ro
urn:nasa:pds:context:instrument:pepe.ds1
urn:nasa:pds:context:instrument:pepssi.nh
urn:nasa:pds:context:instrument:pfs.mex
urn:nasa:pds:context:instrument:pia.gio
urn:nasa:pds:context:instrument:plawav.ice
urn:nasa:pds:context:instrument:pls.go
urn:nasa:pds:context:instrument:pls.m10
urn:nasa:pds:context:instrument:pls.vg1
urn:nasa:pds:context:instrument:pls.vg2
urn:nasa:pds:context:instrument:pm1.vega1
urn:nasa:pds:context:instrument:pm1.vega2
urn:nasa:pds:context:instrument:ppr.go
urn:nasa:pds:context:instrument:pps.vg2
urn:nasa:pds:context:instrument:pra.vg1
urn:nasa:pds:context:instrument:pra.vg2
urn:nasa:pds:context:instrument:puma.vega1
urn:nasa:pds:context:instrument:puma.vega2
urn:nasa:pds:context:instrument:pws.go
urn:nasa:pds:context:instrument:pws.vg1
urn:nasa:pds:context:instrument:pws.vg2
urn:nasa:pds:context:instrument:radwav.ice
urn:nasa:pds:context:instrument:rclt.mpfr
urn:nasa:pds:context:instrument:rcrr.mpfr
urn:nasa:pds:context:instrument:rcrt.mpfr
urn:nasa:pds:context:instrument:rdrs.mgn
urn:nasa:pds:context:instrument:roe.v15
urn:nasa:pds:context:instrument:roe.v16
urn:nasa:pds:context:instrument:rosina.ro
urn:nasa:pds:context:instrument:rpcica.ro
urn:nasa:pds:context:instrument:rpcies.ro
urn:nasa:pds:context:instrument:rpclap.ro
urn:nasa:pds:context:instrument:rpcmag.ro
urn:nasa:pds:context:instrument:rsi.ro
urn:nasa:pds:context:instrument:rss-vg1s.vg1
urn:nasa:pds:context:instrument:rss-vg2s.vg2
urn:nasa:pds:context:instrument:rss-vg2u.vg2
urn:nasa:pds:context:instrument:rss.clem1
urn:nasa:pds:context:instrument:rss.dif
urn:nasa:pds:context:instrument:rss.go
urn:nasa:pds:context:instrument:rss.lro
urn:nasa:pds:context:instrument:rss.mgn
urn:nasa:pds:context:instrument:rss.mo
urn:nasa:pds:context:instrument:rss.vg2
urn:nasa:pds:context:instrument:sce.uly
urn:nasa:pds:context:instrument:sdc.nh
urn:nasa:pds:context:instrument:sow.sakig
urn:nasa:pds:context:instrument:sp1.vega1
urn:nasa:pds:context:instrument:sp1.vega2
urn:nasa:pds:context:instrument:sp2.vega1
urn:nasa:pds:context:instrument:sp2.vega2
urn:nasa:pds:context:instrument:spicam.mex
urn:nasa:pds:context:instrument:spirit3.msx
urn:nasa:pds:context:instrument:src.sdu
urn:nasa:pds:context:instrument:ssd.go
urn:nasa:pds:context:instrument:ssi.go
urn:nasa:pds:context:instrument:ssp.hp
urn:nasa:pds:context:instrument:swap.nh
urn:nasa:pds:context:instrument:swics.uly
urn:nasa:pds:context:instrument:swoops.uly
urn:nasa:pds:context:instrument:swp.ice
urn:nasa:pds:context:instrument:swp.iue
urn:nasa:pds:context:instrument:telescope.spex.irtf3m0
urn:nasa:pds:context:instrument:tnm.vega1
urn:nasa:pds:context:instrument:tvs.vega1
urn:nasa:pds:context:instrument:tvs.vega2
urn:nasa:pds:context:instrument:udds.uly
urn:nasa:pds:context:instrument:uleca.ice
urn:nasa:pds:context:instrument:urap.uly
urn:nasa:pds:context:instrument:uvs.go
urn:nasa:pds:context:instrument:uvs.vg1
urn:nasa:pds:context:instrument:uvs.vg2
urn:nasa:pds:context:instrument:uvvis.clem1
urn:nasa:pds:context:instrument:vex.aspera4-els
urn:nasa:pds:context:instrument:vex.mag
urn:nasa:pds:context:instrument:vex.rss
urn:nasa:pds:context:instrument:vex.virtis
urn:nasa:pds:context:instrument:vhm-fgm.uly
urn:nasa:pds:context:instrument:virtis.ro
urn:nasa:pds:context:instrument:wfpc2.hst
urn:nasa:pds:context:instrument_host:spacecraft.gio
urn:nasa:pds:context:instrument_host:spacecraft.hp
urn:nasa:pds:context:instrument_host:spacecraft.iue
urn:nasa:pds:context:instrument_host:spacecraft.mex
urn:nasa:pds:context:instrument_host:spacecraft.rl
urn:nasa:pds:context:instrument_host:spacecraft.ro
urn:nasa:pds:context:instrument_host:spacecraft.uly
urn:nasa:pds:context:instrument_host:spacecraft.vex
urn:nasa:pds:context:investigation:mission.giotto
urn:nasa:pds:context:investigation:mission.giotto_extended_mission
urn:nasa:pds:context:investigation:mission.international_rosetta_mission
urn:nasa:pds:context:investigation:mission.international_ultraviolet_explorer
urn:nasa:pds:context:investigation:mission.iue
urn:nasa:pds:context:investigation:mission.mars_express
urn:nasa:pds:context:investigation:mission.ulysses
urn:nasa:pds:context:investigation:mission.venus_express
urn:nasa:pds:context:target:star.1swasp_j175207.01+373246.3
urn:nasa:pds:context:target:star.bd+36_2593
urn:nasa:pds:context:target:star.bd+47_2846
urn:nasa:pds:context:target:sun.sun
urn:nasa:pds:context:telescope:ctio.2mass1m3
urn:nasa:pds:context:telescope:ctio.smarts1m0
urn:nasa:pds:context:telescope:ctio.smarts1m5
urn:nasa:pds:context:telescope:ctio.soar4m1
urn:nasa:pds:context:telescope:eso.1m52
urn:nasa:pds:context:telescope:eso.3m6
urn:nasa:pds:context:telescope:eso.ntt
urn:nasa:pds:context:telescope:eso.photometric1m
urn:nasa:pds:context:telescope:irtf.3m0
urn:nasa:pds:context:telescope:mauna_kea.0m61
urn:nasa:pds:context:telescope:mauna_kea.2m24
urn:nasa:pds:context:telescope:mcdonald.smith2m7
urn:nasa:pds:context:telescope:siding_spring.att2m3

jordanpadams commented 1 year ago

@rchenatjpl can we actually have a CSV for this?

previous_lid,new_lid

This is so we can track the mapping between the 2.

rchenatjpl commented 1 year ago

Two of the old LIDs converted into multiple LIDs, so the .csv actually has 4 comma-separated columns lidChange.csv

matthewtiscareno commented 1 year ago

Thank you, @rchenatjpl !!

You say that all deprecated LIDs have no known referrers, though I think this statement is confined to context product LIDs. Be that as it may, shouldn't it be true (at least for domains within which it's true that deprecated LIDs have no referrers) that the issue in the initial request is not needed?

matthewtiscareno commented 1 year ago

Funny. If you type :telescope:, GitHub turns it into 🔭 . This explains the icons within some of @rchenatjpl's LIDs.

NASA-PDS / registry-api