-
I am extracting whole page source as html using raw html as item type.
But after extraction I am getting only partial HTML tags within the content and also start and end tags are missing.
Junks like "…
-
Hello,
Could you please help me figure out what I'm doing wrong ? Here are the steps:
i followed the portia install manual - all ok
i created a new project, entered an url, tagged an item - all ok
cl…
-
One scrapy job is running forever, can't be stopped.
1) I have set downloader timeout to 30
2) In my code, I checked the running periodlly, if longer than some time, use 'http://localhost:6800/cancel…
-
Hi, guys!
Thanks for that awesome project. It's hard to implement or you can say the way to achive it (have time for making PR) saving to MongoDB as storage?
-
You can add new feature of notifying a user when the job is complete. For example, using job id person should be able to know if that job is completed or not. Put it in another way on the basis of job…
-
[Scrapy](http://scrapy.org/) seems to be more developed and feature rich project. Have you ever considered switching to it from [scrapelib](https://github.com/sunlightlabs/scrapelib/)?
One thing that…
-
has always these are problably "dumb" issues from someone who only used scrapinghub.com and never used scrapy/scrapyd
i downloaded my project from scrapinghub and upload to data/projects however port…
-
How can i use scrapyd with portia ?
Thanks
-
Today, job data in scrapyd is only kept in memory?
-
Would be great to have a way to set a flag that tells portiacrawl to crawl all spiders instead of just doing one at a time.