scrapinghub / portia

Visual scraping for Scrapy
BSD 3-Clause "New" or "Revised" License
9.28k stars 1.41k forks source link

Portia install on windows #711

Closed devensonas closed 7 years ago

devensonas commented 7 years ago

Hi, i tried Vagrant and docker to install Portia on windows but always get errors, maybe somebody have step by step guide to install correctly? Or ready to use VMware or Virtualbox image?

ruairif commented 7 years ago

Which instructions did you follow? The most up to date instructions and scripts can be found here

devensonas commented 7 years ago

I tried this

  1. mkdir portia
  2. cd portia
  3. git clone https://github.com/scrapinghub/portia.git
  4. cd portia
  5. git checkout update_installation
  6. mkdir ~/data
  7. cd portiaui
  8. npm install && bower install
  9. cd node_modules/ember-cli && npm install && cd ../../ && ember build
  10. cd ..
  11. docker build -t portia:v1 .
  12. docker run -i -t --rm -p 9001:9001 -v ~/data:/app/data/projects ~/portia/portia/portiaui/dist:/app/portiaui/dist portia:v1
  13. Point a browser to localhost:9001
ruairif commented 7 years ago

Did it work? Did you try to use the vagrant instructions?

devensonas commented 7 years ago

Yes i tried this. But i get an error.

On 21 Feb 2017 1:07 p.m., "Ruairi Fahy" notifications@github.com wrote:

Did it work? Did you try to use the vagrant instructions?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/scrapinghub/portia/issues/711#issuecomment-281312862, or mute the thread https://github.com/notifications/unsubscribe-auth/AGlayxW5-cEiz42GgmAvhmVE3GDQGEpZks5resWAgaJpZM4MG670 .

devensonas commented 7 years ago

Maybe you can sent me step by step instruction with "vagrant" maybe i do something wrong?

devensonas commented 7 years ago

Sorry for writing again, but maybe somebody can help me install PORTIA on Windows or on Ubuntu? I am trying 3 days, nothing helped. Please sent me step by step instructions ar contact me by skype:devensonas for helping me...

ruairif commented 7 years ago

Vagrant instructions are here

devensonas commented 7 years ago

With Vagrant nothing helper... so I installed Ubuntu. So trying do run with docker...

Install using docker i get error: devensonas@ubuntu:~$ sudo docker run -i -t --rm -v projects:/app/data/projects:rw -p 9001:9001 scrapinghub/portia /app/slyd/bot.py:40: ScrapyDeprecationWarning: scrapy.spider.BaseSpider is deprecated, instantiate scrapy.spider.Spider instead. spider = BaseSpider('slyd') /app/slyd/tap.py:59: ScrapyDeprecationWarning: scrapy.settings.CrawlerSettings is deprecated, instantiate scrapy.settings.Settings instead. crawler_settings = CrawlerSettings(settings_module=slyd.settings) /usr/local/lib/python2.7/dist-packages/scrapy/settings/__init__.py:170: ScrapyDeprecationWarning:Settings.overridesattribute is deprecated and won't be supported in Scrapy 0.26, useSettings.set(name, value, priority='cmdline')instead if opt_name in self.overrides: /usr/local/lib/python2.7/dist-packages/scrapy/settings/__init__.py:174: ScrapyDeprecationWarning:Settings.defaultsattribute is deprecated and won't be supported in Scrapy 0.26, useSettings.set(name, value, priority='default')` instead if opt_name in self.defaults: /usr/local/lib/python2.7/dist-packages/twisted/internet/_sslverify.py:184: UserWarning: You do not have the service_identity module installed. Please install it from https://pypi.python.org/pypi/service_identity. Without the service_identity module and a recent enough pyOpenSSL tosupport it, Twisted can perform only rudimentary TLS client hostnameverification. Many valid certificate/hostname mappings may be rejected. verifyHostname, VerificationError = _selectVerifyImplementation() 2017-02-23 06:23:09+0000 [-] Log opened. 2017-02-23 06:23:09+0000 [-] twistd 14.0.0 (/usr/bin/python 2.7.3) starting up. 2017-02-23 06:23:09+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor. 2017-02-23 06:23:09+0000 [-] Site starting on 9001 2017-02-23 06:23:09+0000 [-] Starting factory <twisted.web.server.Site instance at 0x467f518>

`

in firefox:

`No Such Resource

No such child resource.`

What is wrong?

ruairif commented 7 years ago

You need to mount the dist directory too so that assets are available in the docker image

devensonas commented 7 years ago

How to do this?

On 23 Feb 2017 11:15 a.m., "Ruairi Fahy" notifications@github.com wrote:

You need to mount the dist directory too so that assets are available in the docker image

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/scrapinghub/portia/issues/711#issuecomment-281937385, or mute the thread https://github.com/notifications/unsubscribe-auth/AGlay1gkuboOgtAXRNBIxAMWIqMcoUf7ks5rfU4fgaJpZM4MG670 .

arcontechnologies commented 7 years ago

logfile.txt Sorry but Vagrant isn't work either. I tried several time and still the issue there. Is there a stable version. I follwed up the updated installtion and still getting portia not working.

I logged into ssh ssh session and slyd service is not recognized.

vagrant@portia:~$ sudo service slyd start slyd: unrecognized service

I attached the logfile of the installtion maybe you can let me know what is going wrong.

==> default: Installing collected packages: slybot ==> default: Running setup.py develop for slybot ==> default: ==> default: warning: no files found matching 'slybot/splash-script-combined.js' ==> default: Creating /usr/local/lib/python2.7/dist-packages/slybot.egg-link (link to .) ==> default: Adding slybot 0.13.0b28 to easy-install.pth file ==> default: Installing slybot script to /usr/local/bin ==> default: Installing portiacrawl script to /usr/local/bin ==> default: ==> default: Installed /vagrant/slybot ==> default: Successfully installed slybot ==> default: Cleaning up... ==> default: Executing command: configure_nginx ==> default: Executing command: configure_initctl ==> default: Starting slyd service ==> default: ===================== The SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed. The output for this command should be in the log above. Please read the output to determine what went wrong.

ruairif commented 7 years ago

The correct install instructions can be found here