NIH Funding Opportunities Data Processing and MeSH tagging

innovation-growth-lab / alphafold-impact

0 stars 0 forks source link

Closed Jack-0-0 closed 4 months ago

Jack-0-0 commented 4 months ago

Closes #18.

This PR:

Adds utils functions for MeSH tagging
Adds processing pipeline for NIH funding opportunities to clean the data and produce MeSH tags
Fixes the Crunchbase entries in the data catalog. Using ${diss.crunchbase} in the catalog.yml and having diss: crunchbase: s3://discovery-iss/data/crunchbase/Crunchbase_20240109_000941 in the catalog_globals.yml file does not work as kedro expects the items to be datasets.

Jack-0-0 commented 4 months ago

@ampudia19 Regarding the Crunchbase data catalog entries, I tried having

diss:
  crunchbase: s3://discovery-iss/data/crunchbase/Crunchbase_20240109_000941

in globals.yml and using ${diss.crunchbase} in the catalog.yml but it still results in an error. I've opened an issue to look into this.

ampudia19 commented 4 months ago

@Jack-0-0 I've merged because I need the MeSH tagger, but feel free to address the comments above.

Jack-0-0 commented 4 months ago

@ampudia19 I don't see any comments!