#135 crawler implementation completed - Githubissues

VirtualFlyBrain / vfb-pipeline-collectdata

Apache License 2.0

0 stars 0 forks source link

#135 crawler implementation completed #20

Closed hkir-dev closed 3 years ago

hkir-dev commented 3 years ago

A bibliographic data crawler integrated to the pipeline collectdata.

Feature can be enabled/disabled by a config: -- https://github.com/VirtualFlyBrain/vfb-pipeline-config/blob/652dd12e41d73d52cf98a41d93cdd292d552551f/config/collectdata/config.env#L5 -- https://github.com/obophenotype/bds-pipeline-config/blob/dd2b6ebcbc4020b525529ceb84f32531ac4f4784/config/collectdata/config.env#L5
For each ontology, extracts hasDbXref relations.
Queries DOI and PMIDs in the europmc. Persists bibliographic data in DublinCore format.
Packages bibliographic data along with the existing ontologies.