The goal of this repository is to help in transforming raw location data to RDF and storing it on a Solid pod.The location data is stored on the pod as an Event Source, as this allows other applications to build views based on their needs.
Several key components used to achieve this goal:
To save your gpx data to your solid pod, a pipeline was created that is executable with script.py
.
In this flow, we assume the Solid pod is local hosted at http://localhost:3000/
.
Before running the script, some dependencies have to be installed first. This can be done using the following command:
sh ./install.sh
Next, we take our input gpx file (example_run.gpx
) and store it to our Solid pod as an LDES in LDP.
./script.py gpx -i example_run.gpx -l http://localhost:3000/lil/ -V http://example.org/location -u https://data.knows.idlab.ugent.be/person/woslabbi/#me -d http://device.be -ss http://sensor.be -t tm:Walking
In the script, subcommand gpx
is used with following options:
More options can be found using the -h
command (./script.py gpx -h
).
If you are new to Solid, you can use the following commands to set up your own local pod to test the flow.
cd CSS/
npx @solid/community-server -c "@css:config/file-no-setup.json" -f './data'
This command sets up a Solid pod without the set up with a file back-end.
It is also possible to set up a Solid pod with an in memory back-end for rapid prototyping:
npx @solid/community-server -c "memory-no-setup.json"
The code itself uses @inrupt/solid-client-authn Session
objects to deal with authenticated requests.
A small utility script EventSource/loginCreds.ts
is created to create credentials (stored in config.json
) to be used in script.py
.
The following command allows you to run this utility script:
npx ts-node EventSource/loginCreds.ts -i http://localhost:3000
Where the -i parameter is the IDP used to authenticate your WebID. In this case, a CSS was local hosted.
Transforming raw gpx data to an Event Source in solid follows several steps:
First the gpx file is cleaned to allow this file to be processed. Next a custom RML file is being generated based on the input of the script. For this generation, the yarrrml-parser is used.
Now, using this cleaned gpx and RML file, the actual mapping is executed, which is achieved using the RML Mapper (the mapper is already downloaded during the install script).
Finally, an Event Source is created using a Naive algorithm on the solid pod. This Naive algorithm stores all the location data points as an LDES in LDP.
This whole process is visualised in the diagram below: (generated using dataflow-visualisation)
The script has 5 subcommands in total which do the following:
sub command | Description |
---|---|
gpx | Transform a gpx file to RDF using RML and store it on a solid server using LDES in LDP |
container | Use a solid container with location data resources and store it as an LDES in LDP in another container |
css | Use an RDF file of location data and store it on a solid server (all in one container) |
gpxToCss | Transform a gpx file to RDF using RML and store it on a solid server (all in one container) |
linestr | Use an LDES using the location model and aggregate the location points to a wkt line string |
All these flows can also be executed manual without the use of the script. More information on how to run them can be found in the development documentation.
Currently the only type of resources this pipeline works on is location data. Due to this, several parts of this pipeline are hard coded with the location data in mind.
Future work on this project will include extracting resources in the pipeline in a general way. This allows for creating an Event Source of any sort of data such as sensor time-series, financial transactions, ... .
Do not hesitate to report a bug.
Further questions can also be asked to Wout Slabbinck (developer and maintainer of this repository).