search

SciCrunch / sparc-curation

code and files for SPARC curation workflows
MIT License
14 stars 12 forks source link

curation export releases tracking #25

Open tgbugs opened 4 years ago

tgbugs commented 4 years ago

This is the issue for tracking ttl file releases. If you want/need to receive notifications when a new ttl file release is complete subscribe to this issue.

The usual location is https://cassava.ucsd.edu/sparc/exports/. Archives are located at https://cassava.ucsd.edu/sparc/archive/exports/.

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-03-02T01%3A48%3A43%2C640068-08%3A00/

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-03-17T14%3A08%3A06%2C838475-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

bandrow commented 4 years ago

thanks Leo

On Tue, Mar 17, 2020 at 5:59 PM Philip Leo Pascual notifications@github.com wrote:

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-600371423, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNM6IPBXOXDWEFAUIVDRIAMFRANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-03-23T12%3A16%3A56%2C102146-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

bandrow commented 4 years ago

Thank you very much.

On Mon, Mar 23, 2020 at 9:12 PM Philip Leo Pascual notifications@github.com wrote:

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-603003576, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNOQNQGSW4POP2ANVY3RJAXLJANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-03-30T11%3A55%3A38%2C421500-07%3A00/

bandrow commented 4 years ago

thanks!!! I posted it

On Tue, Mar 31, 2020 at 11:45 AM Tom Gillespie notifications@github.com wrote:

New release. Archive url:

https://cassava.ucsd.edu/sparc/archive/exports/2020-03-30T11%3A55%3A38%2C421500-07%3A00/

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-606804525, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNIQFQO6UGV5AK5RBITRKI223ANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

bandrow commented 4 years ago

thanks Leo!

On Tue, Mar 31, 2020 at 8:39 PM Philip Leo Pascual notifications@github.com wrote:

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-607008975, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNKQ2BBB5NK37OGHCDTRKKZOLANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-04-05T16%3A02%3A18%2C946065-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

bandrow commented 4 years ago

Thank you Leo!

On Mon, Apr 6, 2020 at 1:56 PM Philip Leo Pascual notifications@github.com wrote:

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-610032403, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNKPCZMBICES7QUEUGLRLI6YNANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-04-13T17%3A31%3A27%2C070426-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

bandrow commented 4 years ago

thank you Leo!

On Tue, Apr 14, 2020 at 1:24 PM Philip Leo Pascual notifications@github.com wrote:

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-613662355, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNNRGS6NAS4W4BJQYTLRMTA7HANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-04-20T01%3A02%3A39%2C264047-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-04-27T03%3A16%3A37%2C215794-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-04-29T16%3A00%3A58%2C288483-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-05-11T12%3A00%3A40%2C064667-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

bandrow commented 4 years ago

thank you both, I pinged the appropriate people

On Mon, May 11, 2020 at 7:12 PM Philip Leo Pascual notifications@github.com wrote:

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-627065609, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNOEVPLSMCMOVAAG4NTRRCWAXANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-05-18T12%3A32%3A02%2C210968-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

tgbugs commented 4 years ago

Manually patched release of ttl file to include metadata for N:dataset:ca9afa19-b616-41a9-a532-3ae5aaf4088f. This release should NOT be ingested into disco since there has been no change to the other files. @bandrow this is a temporary fix for that dataset's metadata since it has been published on discover. Once the new version of pipeline is up and running in production this dataset won't be missing from the usual process. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-05-19T20%3A06%3A14%2C237707-07%3A00/

bandrow commented 4 years ago

Pretty sure the train left the station already. Besides, there is no harm done.

On Tue, May 19, 2020 at 8:42 PM Tom Gillespie notifications@github.com wrote:

Manually patched release of ttl file to include metadata for N:dataset:ca9afa19-b616-41a9-a532-3ae5aaf4088f. This release should NOT be ingested into disco since there has been no change to the other files. @bandrow https://github.com/bandrow this is a temporary fix for that dataset's metadata since it has been published on discover. Once the new version of pipeline is up and running in production this dataset won't be missing from the usual process. Archive url:

https://cassava.ucsd.edu/sparc/archive/exports/2020-05-19T20%3A06%3A14%2C237707-07%3A00/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-631217708, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNLU623H5D5WB4JN6ITRSNGTHANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

tgbugs commented 4 years ago

New release. NEW FORMAT Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-05-22T03%3A41%3A32%2C756852-07%3A00/

This release is the first that includes the new information content for identifiers and provides unnormalized units. A list of changes:

  1. New units structure. The normalized base units are still present under an additional predicate, but can be ignored if they are not needed. @jwagenaar 
    <subjects/sub-2013-05-21> a owl:NamedIndividual,
    sparc:Subject ;
sparc:animalSubjectHasWeight [ a sparc:Measurement ;
        TEMP:hasUnit unit:kilogram ;
        rdf:value 5.5e+00 ;
        TEMP:asBaseUnits [ a ilxtr:Quantity ;
                TEMP:hasUnit unit:gram ;
                rdf:value 5.5e+03 ] ] ;
sparc:animalSubjectIsOfSpecies "felis catus" ;
sparc:animalSubjectIsOfStrain "domestic short hair" ;
TEMP:hasAge "UNKNOWN" ;
TEMP:hasAgeCategory "Adult" ;
TEMP:hasBiologicalSex "female" ;
TEMP:localId "sub-2013-05-21" .
  2. Minimal ontology identifier metadata is included in the ttl file. For example. 
    UBERON:0001155 a owl:Class ;
rdfs:label "colon" ;
definition: "Last portion of the large intestine before it becomes the rectum." .
  3. Identifier metadata is embedded in the json output. @jgrethe I'm sure there are still bugs lurking here (e.g. #32).
    • ORCiD 
      {
  "contributor_affiliation": "https://ror.org/034xvzb47",
  "contributor_name": "Monroy, Guillermo L",
  "contributor_orcid_id": {
    "id": "https://orcid.org/0000-0002-3669-8514",
    "label": "Guillermo Monroy",
    "system": "Orcid",
    "type": "identifier"
  },
  "contributor_role": [
    "DataCollector",
    "DataCurator",
    "DataManager",
    "ProjectLeader",
    "Researcher"
  ],
  "first_name": "Guillermo",
  "id": {
    "id": "https://orcid.org/0000-0002-3669-8514",
    "label": "Guillermo Monroy",
    "system": "Orcid",
    "type": "identifier"
  },
  "is_contact_person": false,
  "last_name": "Monroy",
  "middle_name": "L"
},
    • DOI 
      {"category": "ArticleJournal",
"id": "https://doi.org/10.3389/fphys.2018.00785",
"label": "Central Respiration and Mechanical Ventilation in the Gating of Swallow With Breathing",
"system": "Doi",
"type": "identifier"
}
    • ror 
        "affiliation": {
    "id": "https://ror.org/0156zyn36",
    "label": "California Medical Innovations Institute",
    "system": "Ror",
    "type": "identifier"
  },
    • RRID 
      "rrid_for_strain": {
"id": "RRID:IMSR_JAX:000664",
"label": "C57BL/6J",
"system": "Rrid",
"type": "identifier"
}
    • protocols.io 
      "protocol_url_or_doi": [
  {
    "id": "https://www.protocols.io/private/AC95D1B52DAFBB09536341A0B6668D33",
    "label": "Pseudorabies Virus (PRV) injection into interscapular brown adipose tissue",
    "system": "Pio",
    "type": "identifier"
  },
  {
    "id": "https://www.protocols.io/private/B7569F5C0050B3198C8BBC721EC210BC",
    "label": "iDISCO protocol for whole-mount immunostaining and volume imaging",
    "system": "Pio",
    "type": "identifier"
  },
  {
    "id": "https://www.protocols.io/private/C57C52C4FAD26F259F15FB2CEC099B5D",
    "label": "Light sheet microscopy",
    "system": "Pio",
    "type": "identifier"
  }
],
Phileodontist commented 4 years ago

Does the latest release require DISCO's interop file to change at all?

tgbugs commented 4 years ago

Not intentionally, but there is a chance that things have drifted a bit over time.

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-05-25T03%3A48%3A13%2C026297-07%3A00/

jwagenaar commented 4 years ago

Hi Tom, Thanks for sending this over. I created a wrike ticket that reflects the required modifications to the TTL import script and marked the importance as high.

https://www.wrike.com/open.htm?id=511238746

Best, Joost

Joost Wagenaar, Ph.D. | VP, Scientific Applications | Blackfynn Inc.

joost@blackfynn.com | Phone: (412) 726-9859

On Mon, May 25, 2020 at 6:56 AM Tom Gillespie notifications@github.com wrote:

New release. Archive url:

https://cassava.ucsd.edu/sparc/archive/exports/2020-05-25T03%3A48%3A13%2C026297-07%3A00/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-633513906, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMICQJP5CAZZ3BSTNUQ7VLRTJFFBANCNFSM4K7QPT3Q .

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-06-01T10%3A22%3A43%2C419491-07%3A00/

bandrow commented 4 years ago

Hey Leo, It looks like the data for SPARC on scicrunch was blank yesterday, can you please take a look? Wonder if the schema changed in some subtle way

On Mon, Jun 1, 2020 at 10:29 AM Tom Gillespie notifications@github.com wrote:

New release. Archive url:

https://cassava.ucsd.edu/sparc/archive/exports/2020-06-01T10%3A22%3A43%2C419491-07%3A00/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-637001626, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNNFDW5SBMETL5JAJGDRUPQOFANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

It looks like the data for SPARC on scicrunch was blank yesterday

@bandrow what URL are you seeing this? The above links seem to show the dataset fine.

bandrow commented 4 years ago

https://scicrunch.org/SPARC/Any/search?q=heart&l=heart Looks good today thank you for checking

On Wed, Jun 3, 2020 at 9:58 AM Philip Leo Pascual notifications@github.com wrote:

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

It looks like the data for SPARC on scicrunch was blank yesterday

@bandrow https://github.com/bandrow what URL are you seeing this? The above links seem to show the dataset fine.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SciCrunch/sparc-curation/issues/25#issuecomment-638325444, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNSWNNHL43OMRPHE4VWPVDRUZ6K3ANCNFSM4K7QPT3Q .

-- All key biological entities deserve an #RRID! orcid.org/0000-0002-5497-0243

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-06-05T00%3A40%3A12%2C675547-07%3A00/

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-06-08T09%3A55%3A21%2C606334-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-06-22T17%3A08%3A18%2C531924-07%3A00/

Phileodontist commented 4 years ago

Both views are now up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

tgbugs commented 4 years ago

New release. Archive url: https://cassava.ucsd.edu/sparc/archive/exports/2020-06-30T02%3A23%3A58%2C995134-07%3A00/

Phileodontist commented 4 years ago

Both views are up-to-date.

https://scicrunch.org/sparc/data/source/SCR_017041-1/search?q= https://scicrunch.org/sparc/data/source/SCR_017041-3/search?q=

bandrow commented 4 years ago

Where is the new data Tom?? Last I see is from 2 weeks ago?

bandrow commented 4 years ago

If you will be re-running, please wait until a little later tonight; Gabi is working on checking a few things and will be done in a few hours