YARRRML supported?

[valentinoli]

:question: Question Does this library support YARRRML out of the box? Or do I need yarrrml-parser?

[ThibaultGerrier]

Hi! Currently there is no build-in yarrrml support. (Although I could see adding it in the future, as I am basically only writing yarrrml)

You should be able to use the yarrrml-parser (https://github.com/RMLio/yarrrml-parser) with this lib.

I have been doing something like this to convert yarrrml to rml/turtle:

import { Writer as N3Writer } from 'n3';
import YarrrmlParser from '@rmlio/yarrrml-parser/lib/rml-generator.js';
import rocketrml from 'rocketrml';

const yarrrmlParse = (yaml) =>
  new Promise((resolve) => {
    const y2r = new YarrrmlParser();
    const yamlQuads = y2r.convert(yaml);
    let prefixes = {
      rr: 'http://www.w3.org/ns/r2rml#',
      rml: 'http://semweb.mmlab.be/ns/rml#',
      xsd: 'http://www.w3.org/2001/XMLSchema#',
      schema: 'http://schema.org/',
      rdf: 'http://www.w3.org/1999/02/22-rdf-syntax-ns#',
      rdfs: 'http://www.w3.org/2000/01/rdf-schema#',
      fnml: 'http://semweb.mmlab.be/ns/fnml#',
      fno: 'http://w3id.org/function/ontology#',
      mex: 'http://mapping.example.com/',
    };
    prefixes = { ...prefixes, ...y2r.getPrefixes() };

    const writer = new N3Writer({ prefixes });
    writer.addQuads(yamlQuads);
    writer.end((_, result) => {
      resolve(result);
    });
  });

(async () => {

    const yarrrml = `
    prefixes:
      ex: "http://example.com/"

    mappings:
      person:
        sources:
          - ['data.json~jsonpath', '$.persons[*]']
        s: http://example.com/$(firstname)
        po:
          - [a, foaf:Person]
          - [ex:name, $(firstname)]
      `

    const rml = await yarrrmlParse(yarrrml);

    const jsonld = await rocketrml.parseFileLive(rml, {
        'data.json': JSON.stringify({
            "persons": [
                {
                    "firstname": "John",
                    "lastname": "Doe"
                },
                {
                    "firstname": "Jane",
                    "lastname": "Smith"
                },
                {
                    "firstname": "Sarah",
                    "lastname": "Bladinck"
                }
            ]
        })
    })

    console.log(jsonld);

})();

[valentinoli]

Great, thank you! This is exactly what I want to do... except I want to use the library in the browser. However, I see that it uses the Node.js fs module so it is not client javascript compatible. Do you have any advice if I need to run this in the browser?

[ThibaultGerrier]

It is possible to run rocktrml&yarrrml-parser in the browser, that is what we do here: https://github.com/semantifyit/rml-editor You can see the result of that project hosted here: https://semantifyit.github.io/rml/

The rocketrml.parseFile function does use the filesystem (to read the mappingfile/data-input files and write the output to the FS), while rocketrml.parseFileLive does not. parseFileLive takes the mappingfile as a string (turtle syntax) and input files as strings (as key value pairs where key is the name of the file and value the actual file as a string) and its return value will be the jsonld. Just like in the code example I posted in my previous comment.

So, you should be able to use parseFileLive in the browser (just as we do here: https://github.com/semantifyit/rml-editor/blob/e15e5964023ef5d88e9faa485ea5d0d58d54538d/src/rmlmapper.ts#L139)

[valentinoli]

Thanks for the reply. I see what you did there and parseFileLive is what I want to use. However, I am using Vue and my application fails to compile due to the fs module not being found:

ERROR Failed to compile with 2 errors
This dependency was not found:
* fs in ./node_modules/rocketrml/src/index.js, ./node_modules/rocketrml/src/input-parser/helper.js

[ThibaultGerrier]

Ah I see, yes that's tricky. I'm afraid there is no easy way to fix that.

Depending on your exact vue (or webpack) setup, it might still be possible. I tried a bit around with nuxt and the following worked for me:

in nuxt.config.js

...
  build: {
    extend (config, { isDev, isClient }) {
        config.node = {
             fs: 'empty', // to ignore fs
         }
     },
     plugins: [
        new webpack.IgnorePlugin({ resourceRegExp: /^/u, contextRegExp: /xpath-iterator/u }) // to ignore xpath-iterator package, which is a optional packages that uses nodejs c++ addon
    ],
  },
...

With this I am able to run the code sample from https://github.com/semantifyit/RocketRML/issues/20#issuecomment-879920233 in the browser with vue/nuxt. (Note, that the version of yarrrml-parser needs to be < 1.2.2, so e.g. "@rmlio/yarrrml-parser": "1.2.1", otherwise you might need some more webpack/babel configuration setup, since they introduced some advanced javascript syntax in the latest version that seems to break the build)

[valentinoli]

Incredible, it worked like a charm! Thank you. I will see if I can get it to work with the latest version of yarrrml-parser

[valentinoli]

I got it working with Nuxt by adding the following to nuxt.config.js:

build: {
  transpile: ['yarrrml-parser']
}

In case it might help anyone, the full config to get both yarrrml-parser and RocketRML working with Nuxt on the client is:

build: {
  transpile: ['yarrrml-parser'],
  extend(config, { isDev, isClient }) {
    config.node = {
      fs: 'empty'
    }
  },
  plugins: [
    new webpack.IgnorePlugin({
      resourceRegExp: /^/u,
      contextRegExp: /xpath-iterator/u
    })
  ]
}

[Peeja]

@ThibaultGerrier It feels a bit silly and wasteful to turn Yarrrml's quads into Turtle and have RocketRML re-parse it. Is there a lower-level API that could accept RML as rdf-js Quads instead of a Turtle string?

[rickadkins]

@Peeja I agree. RocketRML spends a lot of cycles parsing the RML. I often thought it would be useful if I could pre-parse the RML, and present the "compiled" RML to "the rocket" and skip the parsing altogether.