-
I'm finally ready to crawl but when executing
`portiacrawl /Users//Software/90\ Web\ Scraping/portia/slyd/slyd/data/projects/`
to get all spiders listed for the project, I always receive the follow…
-
Sorry, my English is not very well, but I'm working on it.
I'm flowing the doc to study how to use DDS, everything works fine just before the 'Scheduling scrapers/checkers' section in the advanced to…
jszsj updated
9 years ago
-
I just pull recent master - looks totally great and I really enjoy all those cool new features.
There is one thing that doens't work for me though. When I try to deploy to some specific target I get …
-
### Context
I am running scrapyd 1.1 + scrapy 0.24.6 with a single spider that crawls over many domains according to parameters. The development machine that host scrapyd's instance is an OSX Yosemit…
-
Using the pre-built tarball for revision 0.2, I cannot run hyphe unter Ubuntu 14.04 (Trusty). I have removed all instance of scrapy/scrapyd before running the installation script. I have also tried to…
kai17 updated
9 years ago
-
When I run container-transform on my docker-compose.yml, I only get the array for _containerDefinitions_ generated.
**docker-compose.yml**
```
runner:
image: ceroic/scrapyd
ports:
- …
-
I am evaluating portia and run in to an issue deploying to scrapyd.
I have one docker container running portia and another running scrapyd.
When I try to deploy my portia project using
```
scrapyd-d…
-
Hi,
I'm trying to use [Hyphe](https://github.com/medialab/hyphe). Hyphe is using Scrapy but I failed to connect scrapy.
Here is my log file from `scrapyd.log`
> 2015-06-02 14:29:22+0200 [-] "127.0.0…
-
I am deploying my Portia spider in scrapyd. I have given a pattern to be followed in Crawling section in Portia.
While deploying the spider, links are not following the link pattern which I have give…
-
Hi Medialab,
We just discovered that our hyphe instance had created around 15 gb of log files after being used for 2 months. The files are located in subfolders in /var/log/scrapyd/
Can these files …