scrapinghub / portia

Visual scraping for Scrapy
BSD 3-Clause "New" or "Revised" License
9.3k stars 1.4k forks source link

Running Portia 2.0 UI on portia-orm-on-django branch #635

Closed eng1neer closed 7 years ago

eng1neer commented 8 years ago

I'm having trouble running vagrant on portia-orm-on-django branch. Everything's ok on the master branch, but after switching to portia-orm-on-django and running vagrant up, I couldn't connect to localhost:9001.

Is there anything else that needs to be done in order to run Portia 2.0 UI?

ruairif commented 8 years ago

Hi @eng1neer. I need to re add support for vagrant in that branch. Can you use docker? If so you can launch portia using:

cd portia
docker build -t portia .
cd portiaui
npm install
bower install
ember build -e production
docker run -i -t --rm -v ~/scrapinghub/data:/app/slyd/slyd/data/projects:rw -p 9001:9001 portia
eng1neer commented 7 years ago

Hi @ruairif , thank you for the instructions. I tried to run it and got 502 Bad Gateway from nginx. Here is an output of docker run -i -t --rm -v ~/scrapinghub/data:/app/slyd/slyd/data/projects:rw -p 9001:9001 portia:

+ action=
+ shift
+ '[' -z '' ']'
+ _run
+ service nginx start
+ _set_env
+ path=/app/portia_server:/app/slyd:/app/slybot
+ export PYTHONPATH=/app/portia_server:/app/slyd:/app/slybot
+ PYTHONPATH=/app/portia_server:/app/slyd:/app/slybot
+ echo /app/portia_server:/app/slyd:/app/slybot
/app/portia_server:/app/slyd:/app/slybot
+ /app/portia_server/manage.py runserver
+ /app/slyd/bin/slyd -p 9002 -r /app/portiaui/dist
2016-11-21 14:11:51+0000 [-] Log opened.
2016-11-21 14:11:51.827932 [-] Splash version: 2.2.2
2016-11-21 14:11:51.829984 [-] WARNING: Lua scripting is not available because 'lupa' Python package is not installed
2016-11-21 14:11:51.831010 [-] Qt 5.5.1, PyQt 5.5.1, WebKit 538.1, sip 4.17, Twisted 16.1.1
2016-11-21 14:11:51.831963 [-] Python 2.7.6 (default, Jun 22 2015, 17:58:13) [GCC 4.8.2]
2016-11-21 14:11:51.832378 [-] Open files limit: 1048576
2016-11-21 14:11:51.833043 [-] Can't bump open files limit
2016-11-21 14:11:52.058786 [-] Xvfb is started: ['Xvfb', ':1', '-screen', '0', '1024x768x24']
2016-11-21 14:11:53.225306 [-] Traceback (most recent call last):
2016-11-21 14:11:53.225712 [-]   File "/app/slyd/bin/slyd", line 28, in <module>
2016-11-21 14:11:53.226297 [-]     splash.server.main(server_factory=make_server, argv=[])
2016-11-21 14:11:53.226420 [-]   File "/usr/local/lib/python2.7/dist-packages/splash/server.py", line 372, in main
2016-11-21 14:11:53.226563 [-]     server_factory=server_factory,
2016-11-21 14:11:53.226641 [-]   File "/usr/local/lib/python2.7/dist-packages/splash/server.py", line 291, in default_splash_server
2016-11-21 14:11:53.226763 [-]     argument_cache_max_entries=argument_cache_max_entries,
2016-11-21 14:11:53.226834 [-]   File "/app/slyd/bin/slyd", line 25, in make_server
2016-11-21 14:11:53.226915 [-]     'docroot': opts.root}))
2016-11-21 14:11:53.227175 [-]   File "/app/slyd/slyd/tap.py", line 88, in makeService
2016-11-21 14:11:53.228513 [-]     root = create_root(config, settings_module)
2016-11-21 14:11:53.229513 [-]   File "/app/slyd/slyd/tap.py", line 61, in create_root
2016-11-21 14:11:53.229848 [-]     for file_name in listdir(config['docroot']):
2016-11-21 14:11:53.230597 [-] OSError: [Errno 2] No such file or directory: '/app/portiaui/dist'
Performing system checks...

System check identified no issues (0 silenced).

You have 12 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): auth, contenttypes, db_repo, sessions.
Run 'python manage.py migrate' to apply them.

November 21, 2016 - 14:11:54
Django version 1.10.1, using settings 'portia_server.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

I see that there are 2 errors regarding inexisting /app/portiaui/dist directory and unapplied migrations. Could you please advise on how to best deal with these?

sagelliv commented 7 years ago

@eng1neer Portia works with Django and Ember. In this case, the migration error is referring to database migrations that have to be run to properly setup the Django database. For the /app/portiaui/dist directory error this means you are not building the Ember assets correctly. Use ember build -w within the portiaui directory to build Ember.

eng1neer commented 7 years ago

@sagelliv Thanks, got it running, but Portia displays a page without projects. How can I create one?

screen shot 2016-12-08 at 14 40 44
eng1neer commented 7 years ago

This is fixed in #670 and I think this can be closed