ContentMine / quickscrape

A scraping command line tool for the modern web
MIT License
259 stars 43 forks source link

Multiple URLs: accumulating event listeners #33

Closed blahah closed 9 years ago

blahah commented 9 years ago

With multiple URLs, the further through the list the scrape gets, the more event listeners that accumulate and print mulitple messages:

fulltext.pdf
info: [scraper]. download started. fulltext.pdf.
info: [scraper]. download started. fulltext.pdf.
info: [scraper]. download started. fulltext.pdf.
info: [scraper]. download started. fulltext.pdf.
fulltext.pdf
info: [scraper]. download started. fulltext.pdf.
info: [scraper]. download started. fulltext.pdf.
info: [scraper]. download started. fulltext.pdf.
info: [scraper]. download started. fulltext.pdf.
fig-2-full.png
info: [scraper]. download started. fig-2-full.png.
info: [scraper]. download started. fig-2-full.png.
info: [scraper]. download started. fig-2-full.png.
info: [scraper]. download started. fig-2-full.png.
fulltext.html
info: [scraper]. download started. fulltext.html.
info: [scraper]. download started. fulltext.html.
info: [scraper]. download started. fulltext.html.
info: [scraper]. download started. fulltext.html.
NMR_spectra.zip
info: [scraper]. download started. NMR_spectra.zip.
info: [scraper]. download started. NMR_spectra.zip.
info: [scraper]. download started. NMR_spectra.zip.
info: [scraper]. download started. NMR_spectra.zip.
fulltext.xml
info: [scraper]. download started. fulltext.xml.
info: [scraper]. download started. fulltext.xml.
info: [scraper]. download started. fulltext.xml.
info: [scraper]. download started. fulltext.xml.
fulltext.html
info: [scraper]. download started. fulltext.html.
info: [scraper]. download started. fulltext.html.
info: [scraper]. download started. fulltext.html.