-
Could we change the crawler to download PDF from "Defesa civil" instead of Vale website?
I found this [PDF](http://www.vale.com/brasil/PT/aboutvale/servicos-para-comunidade/minas-gerais/atualizacoe…
-
Scrapy provides a potentially good foundation, but to function as a archival crawler, we need to add a few features:
- [x] Start with generic spider that reads seeds from a file.
- [x] By default,…
-
Hi all,
I've been experimenting with making an AWS lambda function for browsertrix-crawler and I've gone some distance but hit a snag that the maintainers are probably better equipped to help with.…
-
-
This will be really useful if our nodes become unsynchronized from the blockchain for any reason.
It will also be useful to verify data.
This shouldn't live in `mooncrawl`. Maybe we should write…
-
Hallo Max,
habe gerade (erneut) festgestellt, dass der Crawler manchmal am Ende des Crawls festhängt und wiederholt die Info bzgl. der "Crawled x pages (x pages/min)" ausgibt. Anbei eine Kopie des …
-
# Feed crawler
Feed crawler – service which posts the best (under multiple criteria) news from media services and social networks.
**Problem**: There is too much information on the Internet. You…
n0str updated
4 years ago
-
-
Maybe this should be opened against the crawler component, but this looks like an application-wide feature.
We could trigger application-wide events during the crawler's lifecycle when a request is i…
-
Avem nevoie de un crawler de texte românești. Deocamdată ne trebuie pentru:
1. Exemple de folosire a cuvintelor (împărțim textele în fraze, apoi construim un index lexem -> frază, apoi la căutarea unu…