search

scrapinghub / portia

Visual scraping for Scrapy
BSD 3-Clause "New" or "Revised" License
9.3k stars 1.41k forks source link

Cannot scrape website with language other that english. #743

Open hackrush01 opened 7 years ago

hackrush01 commented 7 years ago

This is in extension to the issue label #742.

hackrush01 commented 7 years ago

I am running portia using the development method. But I am still unable to scrape the website mentioned in #742. Does something else needs to be done?

ruairif commented 7 years ago

Can you provide more information. We don't know which sites or if you're getting any errors

hackrush01 commented 7 years ago

Yes, Sure. These are the ones I tried

  1. france.fr
  2. http://www.patrika.com/recipes/
  3. http://blog.csdn.net/u013007900

Error is the same as posted in #742 OP.

ruairif commented 7 years ago

The error occurs when you try to run the spider by pressing the green button?

hackrush01 commented 7 years ago

Yes, should I take a screenshot and send it?

ruairif commented 7 years ago

Portia is not used for scheduling/running spiders directly, that is handled by scrapyd or Scrapy Cloud. It is possible to run this as a separate service on portia 6800 so when you run your spider from portia it will be scheduled and run on scrapyd

hackrush01 commented 7 years ago

Ok, thanks I'll try it out. Is there anything I should know before setting it up? It can be setup inside docker, right?