Review Wikidata processing

Living-with-machines / station-to-station

This repository provides underlying code and materials for the paper 'Station to Station: Linking and Enriching Historical British Railway Data'.

https://ceur-ws.org/Vol-2989/long_paper29.pdf

MIT License

5 stars 1 forks source link

Review Wikidata processing #8

Closed mcollardanuy closed 3 years ago

mcollardanuy commented 3 years ago

Instructions:

Go to branch 8-review-wikidata and open the PlaceLinking/wikidata/wd_feature_exploration.ipynb notebook.
The main function to review is parse_record(record). This function is called in two sections in the notebook:
- Parse just one Wikidata record (from client): This parses one wikidata entry that you can specify (change Q609161 in wikidata_entity = "Q609161"). Could you check different locations (of different types and characteristics), and check that we do parse them correctly and capture all the important fields?
- Parse all Wikidata: this is commented, because it applies the same function to all wikidata entries. You don't need to review this cell for now.

mcollardanuy commented 3 years ago

@kmcdono2 please let me know if something is not clear! Thank you!

mcollardanuy commented 3 years ago

Ah @kmcdono2, you actually don't need to download the Wikidata dump file unless you run the last cell ("Parse all wikidata").

kmcdono2 commented 3 years ago

just starting. there is an error in importing pydash. do you want me to comment in a pull request or here?

(I just had to add pip install pydash - but I assume this is an update to the py37deezy env that I've missed)

kmcdono2 commented 3 years ago

also !pip install wikidata

mcollardanuy commented 3 years ago

Hi Katie, here it's fine, thanks!

kmcdono2 commented 3 years ago

notes from parsed record:

described by source field e.g. https://www.wikidata.org/wiki/Property:P1343 seems to be left out. It's not particularly interesting in Abergavenny case. But this is the field we could use to link Topographical Dic entries to wikidata entries perhaps?
Post code? could be useful for disambiguation?
grab EMLO location ID, Encyclopædia Britannica Online ID, OpenStreetMap relation ID, Who's on First ID

kmcdono2 commented 3 years ago

And now I'm having a look at some other places to see re: other fields

https://www.wikidata.org/wiki/Q209055

different from seems useful

https://www.wikidata.org/wiki/Q205679

geoshape
Getty Thesaurus of Geographic Names ID

https://www.wikidata.org/wiki/Q9679

part of
contains administrative territorial entity
shares border with
Pleiades ID
Digital Atlas of the Roman Empire ID

Scottish records: https://www.wikidata.org/wiki/Q2015758

heritage designation
Canmore ID
Historic Scotland ID

Rail-related properties Euston station example https://www.wikidata.org/wiki/Q800751

named after
owned by
operated by
date of official opening
connecting service -UK railway station code

Street- or building-related properties https://www.wikidata.org/wiki/Q6939080

located on street (in Euston record)
inception

kmcdono2 commented 3 years ago

I'm tempted to include 1 of the following IDs: VIAF, Library of Congress, or WorldCat. Simply bc sometimes those are all that are listed.

kmcdono2 commented 3 years ago

OK all done! Happy to discuss if needed.

mcollardanuy commented 3 years ago

Hi @kmcdono2 sorry for the delay, I've addressed this here: 141480276370a4a17f0af6e351271ccf2dac940b

I think we already discussed that some months ago, but I can't find it: I haven't added fields that link to external data or datasets that we don't and won't have (since they won't help in the disambiguation and we can always add this if we need it at some later point), I think I have covered everything else that wasn't already there!

Thanks!