using code to pull metadata from WOS using scrapped data as reference

[amchagas]

this one is most likely me doing something silly:

But when trying to use get_wos_records_from_api.py I do not manage to get the metadata and keep getting errors dumped into the log file.

Here is what I tried so far:

• import code
• initiate class using the inner DATA_PATHS variable as source_files , a local folder as output_dir, and the api_key from WOS.
• run this:

  data = wosTtr.scraped_data()
  record = wosTtr.get_records_from_scraped_data(data)

so effectively:

import get_wos_records_from_api as gwr

source_files =  gwr.DATA_PATHS 
output_dir ="./test_wos"
api_key = "wos_api"

wosTtr = gwr.TitlesToRecords(source_files,output_dir,api_key)

data = wosTtr.scraped_data()
record = wosTtr.get_records_from_scraped_data(data)

but no success so far... welp!

[amchagas]

uploaded a notebook with examples of what I have been trying, but do not seem to get things working. https://github.com/amchagas/open-hardware-supply/blob/master/code/collect_wos_entries.ipynb

[solstag]

Ni! Ah... my bad, no doc. What you need to do is call wosTtr.download_records() to download all matches into 'multi_records.json', then wosTtr.dump_records() to clean the downloaded data and dump it into the final 'records.jsonl'. Sorry!!! :P

Everything else in the class is there to be used by these two methods. I'll add some documentation today.

[amchagas]

cool, thanks for the clarification! I ran the code you mentioned above, but keep getting no data... maybe something with the API key? Would you mind trying to run the jupyter notebook I created on your side?

Also, had to update the code a bit so that it would run on a windows machine (just the naming of the log files... substituting space and : for _ )

[solstag]

api_key = "wos_api" api_key = "wos_api" api_key = "wos_api"

[amchagas]

ha! sorry for that.... :see_no_evil: too sleep deprived I suppose... Maybe this is me making another stupid mistake, but after correcting the above code blooper, I am still not getting anywhere...

[solstag]

You actually did the (kind of) right thing, the key should not be in a public notebook. The right thing™ would be for your code to read the key from a file in a non-shared space. So it turns out you had the right key but was including some separator they used when they sent it tot us. Glad we got that figured out (=