Hackathon project: NIDM-Results in Datalad

[cmaumet]

This issue is open to document work done at the Montreal Hackathon on the NIDM-Results in Datalad project w/ @yarikoptic @mih @djarecka.

[cmaumet]

• Draft of the extractor is now available at: https://github.com/datalad/datalad-neuroimaging/pull/41
• Dataset on which we tested: https://github.com/incf-nidash/demo-datalad-nidmresults

[cmaumet]

(@mih to add more info in here about the commands we ran).

[cmaumet]

For @mih: https://raw.githubusercontent.com/cmaumet/nidmresults-spm/export_from_json/test/spmexport/ex_spm_full_example001/nidm_minimal.json

[mih]

Turn a neurovault download into a datalad dataset

datalad download-url --archive https://neurovault.org/collections/2210/download

Trim to scope of demo:

git mv NIDM-Results\ 1.3.0\ examples\ -\ nidmfsl\ 1.2.01\ and\ SPM12\ r6906/ex_spm_full_example001.nidm.zip .
git rm -rf NIDM-Results\ 1.3.0\ examples\ -\ nidmfsl\ 1.2.01\ and\ SPM12\ r6906/
datalad save -m "Remove all stuff not needed for the demo"

Enable nidmresults extractor:

datalad run-procedure cfg_metadatatypes nidmresults

[cmaumet]

At the end of day 2, we agreed on looking at an alternative representation for the minimal JSON API (that would be fully JSON-LD compliant). More at: https://github.com/cmaumet/nidmresults-spm/pull/4.

[cmaumet]

@mih: here is a small example in "JSON-LD" and what we would like as turtle: https://gist.github.com/cmaumet/bcf93daec19f974aa27be38bfc0a876e

[cmaumet]

For the record I am pasting here draft proposals

by @nicholsn:

{
  "@context": {
    "@version": 1.1,
    "skos": "http://www.w3.org/2004/02/skos/core#",
    "CovMatrix": "@nest",
    "type": {"@id": "@type"},
    "value": {"@id": "xsd:integer"}
  },
  "CovMatrix": {
     "type": "CovMatriz",
     "value": 1
  }
}

Turtle

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

[]
    a <https://json-ld.org/playground-dev/CovMatriz> ;
    <xsd:integer> 1 .

JSON-LD to turtle conversion was done using the rapper utility (brew install raptor) and the NQuads representation from https://json-ld.org/playground-dev.

and by @mih:

{
   "@context": {
        "madeof": {"@id": "madeof", "@container": "@index"},
        "DesignMatrix": "http://purl.org/nidash/nidm#DesignMatrix",
        "SPMsDriftCutoffPeriod":
"http://purl.org/nidash/spm#spm_SPMsDriftCutoffPeriod",
        "name": "dcterms:name"
    },
    "madeof": {
      "DesignMatrix":
          {
              "@type": "DesignMatrix",
              "SPMsDriftCutoffPeriod": 128
          },
      "Software":
          {
              "@type": "AnalysisSoftware",
              "name": "SPM"
          }
    }
}

[] <madeof> [ a <http://purl.org/nidash/nidm#DesignMatrix> ;
           ns2:spm_SPMsDriftCutoffPeriod 128 ],
       [ a <file:///tmp/bcf93daec19f974aa27be38bfc0a876e/AnalysisSoftware> ;
           ns1:name "SPM" ] .

Note: both drafts are not compatible with rdflib, but we could look into pyld.

[satra]

@cmaumet and i worked on this version of the context.

{
  "@context": {
    "@version": 1.1,
    "dc": "http://purl.org/dc/elements/1.1/",
    "dct": "http://purl.org/dc/terms/",
    "ex": "http://example.org/vocab#",
    "xsd": "http://www.w3.org/2001/XMLSchema#",
    "nidm": "http://purl.org/nidash/nidm#",
    "niiri": "http://iri.nidash.org/",
    "spm": "http://purl.org/nidash/spm#",
    "prov": "http://www.w3.org/ns/prov#",
    "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
    "nfo": "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#",
    "location": {
      "@id": "prov:atLocation",
      "@type": "xsd:anyURI"
    },
    "hasDriftModel": {
      "@id": "nidm:NIDM_0000088",
      "@type": "@id"
    },
    "hasHRFBasis": {
      "@id": "nidm:NIDM_0000102",
      "@type": "@id"      
    },
    "description":
    {  "@id": "dc:description",
     "@type": "@id"      
    },
    "design_matrix": {
      "@id": "nidm:NIDM_0000019"
    },
    "records": {
      "@id": "@graph",
      "@container": "@type"
    }
  }
}

this results in one record looking like:

{
  "records": {
    "design_matrix": {
      "@id": "niiri:afa4c8588ca68bae27f3cf6d2f15df1f",
      "description": "niiri:609241dc958fde5cfbd997d646a86357",
      "dct:format": "text/csv",
      "nidm:NIDM_0000021": "[\"Sn(1) active*bf(1)\", \"Sn(1) constant\"]",
      "hasDriftModel": "niiri:2f08d4b34a6da28bbdf5dd25a5df6574",
      "hasHRFBasis": "spm:SPM_0000004",
      "nfo:fileName": "DesignMatrix.csv",
      "rdfs:label": "Design Matrix",
      "location": "http://neurovault.org/collections/2210/ex_spm_full_example001.nidm/DesignMatrix.csv"
    }
  },
  "@id": "file:///software/work/test_mb/ex_spm_full_example001.nidm.ttl"
}

[cmaumet]

More at: https://github.com/datalad/datalad/issues/2753#issuecomment-420330982

[cmaumet]

This issue was used as a report of our datalad/NIDM project at the neuroinformatics Brainhack in Montréal, QC | 7-8 August, 2018.

I'm closing this issue now, cf. https://github.com/datalad/datalad/issues/2753 and #479 for follow ups on this project.