I am trying to analyze around ~100,000 PubMed case reports and tried to get the data using entrez_fetch as shown below:
for(i in seq(1,10000,25)){ info <- entrez_fetch(db="pubmed", web_history=pubmed_search$web_history, rettype="xml", retmax=25, retstart=i, parsed = TRUE) cat(saveXML(info), "\n", file="sample.xml", append=TRUE) cat(i+24, "sequences downloaded\r") }
This code snippet is very similar to the one found in the documentation. However, after a few thousand files, an error occurs and I discovered it's because the database won't allow me to download it - I think it's because it might be more than 3 requests per second. Does anyone know a workaround for this?
I am trying to analyze around ~100,000 PubMed case reports and tried to get the data using entrez_fetch as shown below:
for(i in seq(1,10000,25)){ info <- entrez_fetch(db="pubmed", web_history=pubmed_search$web_history, rettype="xml", retmax=25, retstart=i, parsed = TRUE) cat(saveXML(info), "\n", file="sample.xml", append=TRUE) cat(i+24, "sequences downloaded\r") }This code snippet is very similar to the one found in the documentation. However, after a few thousand files, an error occurs and I discovered it's because the database won't allow me to download it - I think it's because it might be more than 3 requests per second. Does anyone know a workaround for this?