search

DigitalPiranesi / Manifest-Generator

Apache License 2.0
0 stars 0 forks source link

Summer/Fall - 2023-2024 - Automated RDF Cloner #23

Closed waltster closed 1 year ago

waltster commented 1 year ago

The RDF currently being used for experimentation is/will be stale by the time production is ready. The original RDF was downloaded via the API for Scalar with the following query:

https://scalar.usc.edu/works/piranesidigitalproject/rdf/instancesof/content?format=json&rec=1&ref=1&start=<START INDEX>&results=2500

By incrementing the <START INDEX> by 2500 in sequence or parallel, the entirety of the RDF JSON can be downloaded. The results field can be modified to speed up the download.

A Bash/Python script could automate this, making it simple to download the entirety of the site quickly.

adamskeefer commented 1 year ago

Solved with Scalar API feature.