Implement Gemeente Ede as a source

openstate / open-cultuur-data

The back- and front-end code that powers the Open Cultuur Data API

http://opencultuurdata.nl/

28 stars 18 forks source link

Implement Gemeente Ede as a source #72

Closed breyten closed 9 years ago

breyten commented 10 years ago

Implement Gemeente Ede as a source

The CSV file was converted to JSON, then uploaded to the static opencultuurdata endpoint
We now have a station JSON importer as well (It's different from the dump importer)
There's a slight issue with get_all_text() which I wasn't sure how to solve. I've added the call in the item transformer to make sure the tests pass

bartdegoede commented 9 years ago

Looks good to me. Not sure what the "Gemeente Ede" collection is exactly, and if that should be the collection name though, but I guess that's something that you guys should decide.

breyten commented 9 years ago

I'll change the naming to Gemeentearchief Ede :-)

breyten commented 9 years ago

As far as the collection name goes :P

justinvw commented 9 years ago

The documentation for the dataset is missing (docs/user/datasets.rst) :wink:. I briefly looked at the code, and it looks good to me. Tomorrow I wil try do an actual import before we merge this in.

bartdegoede commented 9 years ago

Mentioned docs and test shortly in CONTRIBUTING.rst :-)

justinvw commented 9 years ago

@breyten I just ran the importer without any problems.

Apart from the missing docs and the two comments in staticfile.py there is another small issue with the items that should be fixed before we can merge. The URL to the photograph in media_urls currently points to a Wikimedia Commons HTML page that includes the photo. However, media_urls should only include permanent links to the actual file, so that API users can directly request the images inside their applications.

breyten commented 9 years ago

Ok, fixed that. Thanks for the feedback!

bartdegoede commented 9 years ago

For future reference: there is a Wikimedia Tools Lab which hosts (externally) developed tools that support users of the Wikimedia platform. We use one of these for the Tropenmuseum items, that allows a developer to request author and metadata from the Wikimedia Commons in XML. For example: http://tools.wmflabs.org/magnus-toolserver/commonsapi.php?image=File:GA32001.jpg

For now, this seems to be a bit overkill and the regex solution works fine. For some data on Commons there may be more metadata available that we want to store in the OCD API as well (for enrichment purposes, for instance).