scrapinghub / portia

Visual scraping for Scrapy
BSD 3-Clause "New" or "Revised" License
9.22k stars 1.41k forks source link

Spiders generated by Portia don't work with Scrapy 1.8.0 #895

Open acrossoffwest opened 4 years ago

acrossoffwest commented 4 years ago

I don't know why projects created by Protia don't working

$ scrapyd-deploy -p Test

spiders are not deploying

{"status": "ok", "project": "Test", "version": "1580255071", "spiders": 0, "node_name": "7f4967535546"}

I tried this: https://github.com/scrapy/quotesbot It's okay.

My docker-compose file

version: '3'
services:
  app:
    build: ./portia
    volumes:
      - ./data/projects:/app/data/projects:rw
      - ./scripts:/app/data/scripts
    ports:
      - 9001:9001
    restart: always
    networks:
        - scrapy
  scrapyd:
    build: ./scrapyd
    ports:
      - 6800:6800
    environment:
      - USERNAME=scrap
      - PASSWORD=test1234
      - PACKAGES=simplejson,requests,redis,elasticsearch,certifi 
    networks:
      - scrapy
networks:
  scrapy:

./portia/Dockerfile

FROM scrapinghub/portia:latest

WORKDIR /app/data/projects

RUN pip install scrapyd-client

./scrapyd/Dockerfile

FROM cdrx/scrapyd-authenticated

RUN pip install --upgrade pip
RUN pip install --upgrade scrapy

Can anyone help me?