Capture link to N-Grams

metaphacts / semopenalex

43 stars 6 forks source link

Capture link to N-Grams #49

Open VladimirAlexiev opened 1 year ago

VladimirAlexiev commented 1 year ago

Capture link to N-Grams from the Internet Archive General Index.
See https://docs.openalex.org/api-entities/works/get-n-grams
About 50M paper have this
It can be used for NLP analysis in lieu of full text

linnaung commented 1 year ago

We will amend the mapping and include these in the next data generation.

linnaung commented 1 year ago

When looking N-grams, it's not included in the OpenAlex snapshot, data dump and only available via API. Therefore, to integrate this, we might virtualize the data using https://help.metaphacts.com/resource/Help:EphedraRESTService.

VladimirAlexiev commented 1 year ago

I meant to only capture these API links. If you capture the data in RDF, that may be even better: but first estimate how many triples. I don't remember but I think it was 2-grams to 5-grams, so a lot of data.

linnaung commented 11 months ago

@VladimirAlexiev , we will capture the link in the next data generation.