search

pietercolpaert / Blue-Bike-to-Linked-GBFS

MIT License
2 stars 3 forks source link

location of example blue bike data #17

Open bblfish opened 1 year ago

bblfish commented 1 year ago

Hi,

this repository could be very useful to help understand how an RML to LDES mapping works if there were just a bit more documentation. Here are a couple of problems I came across.

  1. it would be useful if there were a small extract of blue-bike-data so that one can see what the structure of the input data being transformed is. This would just make it a lot easier to read the transformation RML.
  2. I could find links to bluebike web sites from this repo, but not to the data being used or to documentation about the data. Searching the web gave me some links such as https://www.bluebikes.com/system-data but I found it difficult to work out which data set was being used, especially because there is no clear example here (see point 1 above).
  3. I found a link to https://datapiloten.be/bluebike/availabilities.geojson in mapping-webapi-official.rml.ttl but I am not sure what that file is meant to be.

That would be just a matter of adding some links to the README explaining perhaps what some of the main files are doing, and what the roles of the directories are. Also a link to the helpful github workflows action would help to understand how to run things.

pietercolpaert commented 1 year ago

@DylanVanAssche ↑ can you take this?

DylanVanAssche commented 1 year ago

it would be useful if there were a small extract of blue-bike-data so that one can see what the structure of the input data being transformed is. This would just make it a lot easier to read the transformation RML.

There's such a file in the root of the repository: bluebike.jsonld. I used this file to write the RML mappings.

I could find links to bluebike web sites from this repo, but not to the data being used or to documentation about the data. Searching the web gave me some links such as https://www.bluebikes.com/system-data but I found it difficult to work out which data set was being used, especially because there is no clear example here (see point 1 above).

I'm afraid this data is not Open Data, @pietercolpaert knows the details about the license and usage. Because of that, we cannot share the live URL with the world.

I found a link to https://datapiloten.be/bluebike/availabilities.geojson in mapping-webapi-official.rml.ttl but I am not sure what that file is meant to be.

datapiloten.be provides live data about BlueBike but it is not the official data, it is scraped from BlueBike.

I hope this helps? Feel free to ask if you're stuck with something.

bblfish commented 1 year ago

@DylanVanAssche wrote in response to my remark:

it would be useful if there were a small extract of blue-bike-data so that one can see what the structure of the input data being transformed is. This would just make it a lot easier to read the transformation RML.

There's such a file in the root of the repository: bluebike.jsonld. I used this file to write the RML mappings.

Is that not the output of the RML transformation? (I don't see how you would need RML for JSON-LD input.)

Also: is there a version of the 'availabilities.shacl' around?

Btw. I am trying to use this to document how one can do this whole transformation using some open CSV data that we have.

DylanVanAssche commented 1 year ago

Oh this is the original datasource: https://datapiloten.be/bluebike/availabilities.geojson Not sure what the JSON-LD file was then anymore, the output from the RMLMapper is writting with a Logical Target as Turtle.

I don't have the SHACL shape, my repo was forked by @pietercolpaert and he applied some changes on it. Maybe he can shine light on this.

bblfish commented 1 year ago

Mhh, both of the files in the bluebike container https://datapiloten.be/bluebike/ are geojson files with "@context" indicating they are in json-ld format already. So that also does not look like the original data.... (Unless during Covid all devs suddenly had a json-ld enlightening that I missed)

DylanVanAssche commented 1 year ago

@bblfish It is scraped from BlueBike, the official JSON API is proprietary. JSON-LD can be read by RML as regular JSON.

bblfish commented 1 year ago

Note: you should warn folks that this github repo is 1.9 GB large...

DylanVanAssche commented 1 year ago

Yeah it will keep growing indefinitely because each time a new commit is made, a new version is added to the repo.

bblfish commented 1 year ago

Was the answer I was looking for not simply that some example BlueBike data in its original form is available on the RML-LDES-mapping-rules repo ?