Added openalex and dimensions publication links

sul-dlss-labs / rialto-airflow

Airflow for harvesting data for research intelligence and open access analysis

Apache License 2.0

1 stars 0 forks source link

Added openalex and dimensions publication links #98

Closed edsu closed 2 months ago

edsu commented 2 months ago

It might be useful to include links to the article for (possibly) getting full-text. These could be helpful when using the dataset as a source for testing machine learning techniques that use the full text as an input.

OpenAlex: best_oa_location https://docs.openalex.org/api-entities/works/work-object#best_oa_location
Dimensions: linkout https://docs.dimensions.ai/dsl/datasource-publications.html

lwrubel commented 2 months ago

Approved, pending seeing how memory usage goes when you run the DAG.

edsu commented 2 months ago

Approved, pending seeing how memory usage goes when you run the DAG.

It looks like it ran ok, but I would like to do some follow on work to unpack the JSON objects into separate columns.