ctrltz
commented
5 years ago Summary about working with bioRxiv:
- no API or bulk download option, only email alerts, Twitter account and RSS feeds with 30 most recent publications.
- Website - HTML page source contains various useful metadata about corresponding article in
<meta>tags, but references are presented as text, e.g.:<meta name="citation_reference" content="Pickersgill H, Kalverda B, de Wit E, Talhout W, Fornerod M, van Steensel B: Characterization of the Drosophila melanogaster genome at the nuclear lamina. Nat Genet 2006, 38:1005–1014." />It may be possible to look for papers with the same title in the local database to declare ID for the paper.
- by posting on bioRxiv, authors explicitly consent to text mining of their work (e.g., by search engines or researchers).
About bioRxiv in general:
- is intended to complement arXiv.
- no peer review for articles, some articles can cause fake hype.
- articles in bioRxiv are categorized as New Results, Confirmatory Results, or Contradictory Results.
- a new branch called medRxiv (https://www.medrxiv.org/) contains preprints about clinical trials.
olegs
If everything fails, we can go with beatiful soup crawling, see: https://github.com/OmnesRes/prepub