Drugs and GC scrappers update along minor improvements

epfLLM / meditron

Meditron is a suite of open-source medical Large Language Models (LLMs).

https://huggingface.co/epfl-llm

Apache License 2.0

1.77k stars 159 forks source link

Closed jpcorb20 closed 3 months ago

jpcorb20 commented 3 months ago

Updated DrugsScrapper and GCScrapper to avoid issues.
Optimization of Scrapper to use the cached links already fetched by previous runs to skip the link gathering.
Optimization of the chrome driver and handling of more options, of which some are necessary on windows (binary and driver locations) and others optional (headless, page loading mode, and pdf download folder).
One missing requirement: scipdf.

AGBonnet commented 3 months ago

Thanks for contributing to the open-source community!