Closed ElCep closed 9 years ago
Which browser are you using, and can you check in the browser's console if there's any javascript error ?
Hi, of course in firefox dev tool :
SyntaxError: in strict mode code, functions may be declared only at top level or immediately within another function prepareCrawls.js:296:17
Error: [$injector:modulerr] http://errors.angularjs.org/1.3.0-beta.17/$injector/modulerr?p0=hyphe&p1=%5B%24injector%3Amodulerr%5D%20http%3A%2F%2Ferrors.angularjs.org%2F1.3.0-beta.17%2F%24injector%2Fmodulerr%3Fp0%3Dhyphe.controllers%26p1%3D%255B%2524injector%253Amodulerr%255D%2520http%253A%252F%252Ferrors.angularjs.org%252F1.3.0-beta.17%252F%2524injector%252Fmodulerr%253Fp0%253Dhyphe.preparecrawlsController%2526p1%253D%25255B%252524injector%25253Anomod%25255D%252520http%25253A%25252F%25252Ferrors.angularjs.org%25252F1.3.0-beta.17%25252F%252524injector%25252Fnomod%25253Fp0%25253Dhyphe.preparecrawlsController%25250AS%25252F%25253C%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A6%25253A457%25250Asd%25252Fb.module%25253C%25252F%25253C%25252Fb%25255Be%25255D%25253C%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A21%25253A48%25250Asd%25252Fb.module%25253C%25252F%25253C%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A20%25253A1%25250Af%25252F%25253C%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A34%25253A94%25250Aq%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A7%25253A294%25250Af%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A33%25253A445%25250Af%25252F%25253C%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A34%25253A111%25250Aq%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A7%25253A294%25250Af%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A33%25253A445%25250Af%25252F%25253C%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A34%25253A111%25250Aq%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A7%25253A294%25250Af%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A33%25253A445%25250AHb%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A37%25253A174%25250Aic%25252Fd%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A17%25253A352%25250Aic%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A18%25253A65%25250Apd%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A17%25253A97%25250A%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fangularjs%25252F1.3.0-beta.17%25252Fangular.min.js%25253A224%25253A252%25250An.Callbacks%25252Fj%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fjquery%25252F1.11.0%25252Fjquery.min.js%25253A2%25253A27131%25250An.Callbacks%25252Fk.fireWith%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fjquery%25252F1.11.0%25252Fjquery.min.js%25253A2%25253A27949%25250A.ready%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fjquery%25252F1.11.0%25252Fjquery.min.js%25253A2%25253A29781%25250AK%252540http%25253A%25252F%25252Fajax.googleapis.com%25252Fajax%25252Flibs%25252Fjquery%25252F1.11.0%25252Fjquery.min.js%25253A2%25253A30126%25250A%250AS%252F%253C%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A6%253A457%250Af%252F%253C%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A34%253A370%250Aq%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A7%253A294%250Af%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A33%253A445%250Af%252F%253C%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A34%253A111%250Aq%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A7%253A294%250Af%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A33%253A445%250Af%252F%253C%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A34%253A111%250Aq%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A7%253A294%250Af%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A33%253A445%250AHb%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A37%253A174%250Aic%252Fd%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A17%253A352%250Aic%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A18%253A65%250Apd%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A17%253A97%250A%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fangularjs%252F1.3.0-beta.17%252Fangular.min.js%253A224%253A252%250An.Callbacks%252Fj%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fjquery%252F1.11.0%252Fjquery.min.js%253A2%253A27131%250An.Callbacks%252Fk.fireWith%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fjquery%252F1.11.0%252Fjquery.min.js%253A2%253A27949%250A.ready%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fjquery%252F1.11.0%252Fjquery.min.js%253A2%253A29781%250AK%2540http%253A%252F%252Fajax.googleapis.com%252Fajax%252Flibs%252Fjquery%252F1.11.0%252Fjquery.min.js%253A2%253A30126%250A%0AS%2F%3C%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A6%3A457%0Af%2F%3C%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A34%3A370%0Aq%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A7%3A294%0Af%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A33%3A445%0Af%2F%3C%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A34%3A111%0Aq%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A7%3A294%0Af%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A33%3A445%0AHb%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A37%3A174%0Aic%2Fd%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A17%3A352%0Aic%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A18%3A65%0Apd%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A17%3A97%0A%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fangularjs%2F1.3.0-beta.17%2Fangular.min.js%3A224%3A252%0An.Callbacks%2Fj%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.11.0%2Fjquery.min.js%3A2%3A27131%0An.Callbacks%2Fk.fireWith%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.11.0%2Fjquery.min.js%3A2%3A27949%0A.ready%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.11.0%2Fjquery.min.js%3A2%3A29781%0AK%40http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.11.0%2Fjquery.min.js%3A2%3A30126%0A
Thank you @boogheta
@jacomyma can you check what this stack refers to (prepareCrawls.js:296:17)?
@ELCep looks like something broken with your version of firefox in recent developments of the code. Can you try with another browser like Chromium or Chrome? Otherwise you can try maybe to pull the code back to when docker was added until proper fix:
git checkout 2df97413b2031db234f2bbd919a97ac3c06c97f4
Then rebuild the docker container.
This function is not strict mode compliant. The inner getStartPagesSuggestions
function should not be declared within the if
statement.
@ElCep: @jacomyma pushed a commit that should fix your firefox issue. If you git pull then rebuild the docker you should be good
Hi all, Ok I have take time this morning for try :-) but something is wrong . I'm quiet new with docker so I don't know is the error come from docker or your modification :
>git pull
>docker rm $(docker ps -a -q)
> docker-compose up
Creating hyphe_memorystructure_1...
Creating hyphe_frontend_1...
Creating hyphe_mongo_1...
Creating hyphe_crawler_1...
Creating hyphe_backend_1...
Attaching to hyphe_memorystructure_1, hyphe_frontend_1, hyphe_mongo_1, hyphe_crawler_1, hyphe_backend_1
frontend_1 | Starting up http-server, serving ./ on port: 8000
frontend_1 | Hit CTRL-C to stop the server
mongo_1 | 2015-10-02T06:38:00.843+0000 I JOURNAL [initandlisten] journal dir=/data/db/journal
mongo_1 | 2015-10-02T06:38:00.843+0000 I JOURNAL [initandlisten] recover : no journal files present, no recovery needed
mongo_1 | 2015-10-02T06:38:05.570+0000 I JOURNAL [initandlisten] preallocateIsFaster=true 41.64
crawler_1 | 2015-10-02 06:38:03+0000 [-] Log opened.
crawler_1 | 2015-10-02 06:38:03+0000 [-] twistd 15.4.0 (/usr/bin/python 2.7.9) starting up.
crawler_1 | 2015-10-02 06:38:03+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
crawler_1 | 2015-10-02 06:38:03+0000 [-] Site starting on 6800
crawler_1 | 2015-10-02 06:38:03+0000 [-] Starting factory <twisted.web.server.Site instance at 0x7f69c4019320>
crawler_1 | 2015-10-02 06:38:03+0000 [Launcher] Scrapyd 1.0.1 started: max_proc=12, runner='scrapyd.runner'
backend_1 | Unhandled Error
backend_1 | Traceback (most recent call last):
backend_1 | File "/opt/virtualenvs/hyphe/lib/python2.7/site-packages/twisted/application/app.py", line 617, in run
backend_1 | runApp(config)
backend_1 | File "/opt/virtualenvs/hyphe/lib/python2.7/site-packages/twisted/scripts/twistd.py", line 25, in runApp
backend_1 | _SomeApplicationRunner(config).run()
backend_1 | File "/opt/virtualenvs/hyphe/lib/python2.7/site-packages/twisted/application/app.py", line 348, in run
backend_1 | self.application = self.createOrGetApplication()
backend_1 | File "/opt/virtualenvs/hyphe/lib/python2.7/site-packages/twisted/application/app.py", line 413, in createOrGetApplication
backend_1 | application = getApplication(self.config, passphrase)
backend_1 | --- <exception caught here> ---
backend_1 | File "/opt/virtualenvs/hyphe/lib/python2.7/site-packages/twisted/application/app.py", line 424, in getApplication
backend_1 | application = service.loadApplication(filename, style, passphrase)
backend_1 | File "/opt/virtualenvs/hyphe/lib/python2.7/site-packages/twisted/application/service.py", line 411, in loadApplication
backend_1 | passphrase)
backend_1 | File "/opt/virtualenvs/hyphe/lib/python2.7/site-packages/twisted/persisted/sob.py", line 207, in loadValueFromFile
backend_1 | eval(codeObj, d, d)
backend_1 | File "/app/hyphe_backend/core.tac", line 28, in <module>
backend_1 | from hyphe_backend.lib.utils import *
backend_1 | File "/app/hyphe_backend/lib/utils.py", line 8, in <module>
backend_1 | config = load_config()
backend_1 | File "/app/hyphe_backend/lib/config_hci.py", line 82, in load_config
backend_1 | db = MongoClient(os.environ.get('HYPHE_MONGODB_HOST', mongoconf['host']), int(os.environ.get('HYPHE_MONGODB_PORT', mongoconf['mongo_port'])))[mongoconf.get('db_name', mongoconf.get('project'))]
backend_1 | File "/opt/virtualenvs/hyphe/lib/python2.7/site-packages/pymongo/mongo_client.py", line 366, in __init__
backend_1 | raise ConnectionFailure(str(e))
backend_1 | pymongo.errors.ConnectionFailure: [Errno 111] Connection refused
backend_1 |
backend_1 |
backend_1 | Failed to load application: [Errno 111] Connection refused
backend_1 |
hyphe_backend_1 exited with code 1
Gracefully stopping... (press Ctrl+C again to force)
Stopping hyphe_crawler_1... done
Stopping hyphe_mongo_1... done
Stopping hyphe_frontend_1... done
This is unfortunately where we reach the limits of my Docker knowledge so far :/ This should not be linked in any way to the fix. Apparently this is the mongodb container which is unaccessible to the backend for some reason, @oncletom might have an idea?
Hi all, I'm trying at home to install your last release... it still doesn't work with firefox. With chrome I've got something but not a good message : YOU ARE DISCONNECTED FROM HYPHE :-(
In ma terminal :
frontend_1 | [Sat, 03 Oct 2015 11:03:50 GMT] "GET /" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
backend_1 | 2015-10-03 11:03:50+0000 [-] "172.17.42.1" - - [03/Oct/2015:11:03:50 +0000] "POST /hyphe-api/ HTTP/1.1" 200 247 "http://192.168.122.1:8000/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
backend_1 | 2015-10-03 11:03:50+0000 [-] "172.17.42.1" - - [03/Oct/2015:11:03:50 +0000] "POST /hyphe-api/ HTTP/1.1" 200 35 "http://192.168.122.1:8000/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
backend_1 | 2015-10-03 11:03:53+0000 [-] "172.17.42.1" - - [03/Oct/2015:11:03:52 +0000] "POST /hyphe-api/ HTTP/1.1" 200 247 "http://192.168.122.1:8000/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
backend_1 | 2015-10-03 11:03:53+0000 [-] "172.17.42.1" - - [03/Oct/2015:11:03:52 +0000] "POST /hyphe-api/ HTTP/1.1" 200 35 "http://192.168.122.1:8000/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
E.
Hi, according to the error trace, the connection to the MongoDB database can't be properly made.
The backend reports this error so eventually do docker inspect backend_1
and check if the value of HYPHE_MONGODB_HOST
makes sense.
Hi @oncletom
docker inspect backend_1
give me nothing :-S
Error: No such image or container: backend_1
[]
However docker ps -a
give me
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
01d36b3d4bf6 hyphe_backend "/bin/sh -c '/bin/bas" 3 days ago Exited (1) 2 minutes ago hyphe_backend_1
1cda1e275c6d hyphe_crawler "/usr/bin/scrapyd" 3 days ago Exited (0) 2 minutes ago hyphe_crawler_1
e9346bc210e4 mongo:3.0 "/entrypoint.sh mongo" 3 days ago Exited (0) 2 minutes ago hyphe_mongo_1
db8dead2459f hyphe_frontend "/app/node_modules/.b" 3 days ago Exited (143) 2 minutes ago hyphe_frontend_1
9876d21b82bc hyphe_memorystructure "true" 3 days ago Exited (0) 2 minutes ago hyphe_memorystructure_
any ideas E.
Ah, it is because the stack is not up, obviously.
You should be able to get the state of an exited container anyway: docker inspect 01d36b3d4bf6
. Out of curiosity, do the same for the crawler: docker inspect 1cda1e275c6d
. The mongodb env variable should be the same – and aiming at the hostname the container e9346bc210e4
was labeled with.
Thank you @oncletom ! so docker inspect 01d36b3d4bf6
give me "HYPHE_MONGODB_HOST=mongo"
it's seem be ok ?
[
{
"Id": "01d36b3d4bf6cc5c0ab10a33ad678c8ac2893f38b1a36ec48bff026bbd9654e0",
"Created": "2015-10-02T06:38:03.878025849Z",
"Path": "/bin/sh",
"Args": [
"-c",
"/bin/bash -c \"source $(which virtualenvwrapper.sh) \u0026\u0026 workon hyphe \u0026\u0026 twistd -y /app/hyphe_backend/core.tac --nodaemon\""
],
"State": {
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 1,
"Error": "",
"StartedAt": "2015-10-05T08:56:41.35350344Z",
"FinishedAt": "2015-10-05T08:56:41.894873924Z"
},
"Image": "c2ff944c46e667de38f59fe720cb6bb52690124c69bc42ccee9480329dcc1852",
"NetworkSettings": {
"Bridge": "",
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"HairpinMode": false,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"MacAddress": "",
"NetworkID": "",
"PortMapping": null,
"Ports": null,
"SandboxKey": "",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null
},
"ResolvConfPath": "/var/lib/docker/containers/01d36b3d4bf6cc5c0ab10a33ad678c8ac2893f38b1a36ec48bff026bbd9654e0/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/01d36b3d4bf6cc5c0ab10a33ad678c8ac2893f38b1a36ec48bff026bbd9654e0/hostname",
"HostsPath": "/var/lib/docker/containers/01d36b3d4bf6cc5c0ab10a33ad678c8ac2893f38b1a36ec48bff026bbd9654e0/hosts",
"LogPath": "/var/lib/docker/containers/01d36b3d4bf6cc5c0ab10a33ad678c8ac2893f38b1a36ec48bff026bbd9654e0/01d36b3d4bf6cc5c0ab10a33ad678c8ac2893f38b1a36ec48bff026bbd9654e0-json.log",
"Name": "/hyphe_backend_1",
"RestartCount": 0,
"Driver": "devicemapper",
"ExecDriver": "native-0.2",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [],
"ContainerIDFile": "",
"LxcConf": null,
"Memory": 0,
"MemorySwap": 0,
"CpuShares": 0,
"CpuPeriod": 0,
"CpusetCpus": "",
"CpusetMems": "",
"CpuQuota": 0,
"BlkioWeight": 0,
"OomKillDisable": false,
"MemorySwappiness": null,
"Privileged": false,
"PortBindings": {
"6978/tcp": [
{
"HostIp": "",
"HostPort": "6978"
}
]
},
"Links": [
"/hyphe_mongo_1:/hyphe_backend_1/mongo",
"/hyphe_mongo_1:/hyphe_backend_1/mongo_1",
"/hyphe_crawler_1:/hyphe_backend_1/crawler",
"/hyphe_crawler_1:/hyphe_backend_1/crawler_1",
"/hyphe_crawler_1:/hyphe_backend_1/hyphe_crawler_1",
"/hyphe_mongo_1:/hyphe_backend_1/hyphe_mongo_1"
],
"PublishAllPorts": false,
"Dns": null,
"DnsSearch": null,
"ExtraHosts": [],
"VolumesFrom": [
"9876d21b82bc8a1ff4cbd86648bc00f7383453862247f5ef5324d181bdd7d453"
],
"Devices": null,
"NetworkMode": "default",
"IpcMode": "",
"PidMode": "",
"UTSMode": "",
"CapAdd": null,
"CapDrop": null,
"GroupAdd": null,
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"SecurityOpt": null,
"ReadonlyRootfs": false,
"Ulimits": null,
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"CgroupParent": "",
"ConsoleSize": [
0,
0
]
},
"GraphDriver": {
"Name": "devicemapper",
"Data": {
"DeviceId": "214",
"DeviceName": "docker-253:1-1062480-01d36b3d4bf6cc5c0ab10a33ad678c8ac2893f38b1a36ec48bff026bbd9654e0",
"DeviceSize": "107374182400"
}
},
"Mounts": [
{
"Name": "503a9927043a83ad6c08aa62c349275eeb35da09a5aa476c9b4709b284ba15e8",
"Source": "/var/lib/docker/volumes/503a9927043a83ad6c08aa62c349275eeb35da09a5aa476c9b4709b284ba15e8/_data",
"Destination": "/app/memory_structure",
"Driver": "local",
"Mode": "",
"RW": true
},
{
"Name": "82ce29476842999a1de5e874f85db126c3411347d73c7b7ff7a8c837b359fe8f",
"Source": "/var/lib/docker/volumes/82ce29476842999a1de5e874f85db126c3411347d73c7b7ff7a8c837b359fe8f/_data",
"Destination": "/app/hyphe_backend/memorystructure",
"Driver": "local",
"Mode": "",
"RW": true
}
],
"Config": {
"Hostname": "01d36b3d4bf6",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"6978/tcp": {}
},
"PublishService": "",
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"HYPHE_MONGODB_HOST=mongo",
"HYPHE_CRAWLER_PORT=6800",
"HYPHE_CRAWLER_HOST=crawler",
"HYPHE_MONGODB_PORT=27017",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8",
"PYTHON_VERSION=2.7.10",
"PYTHON_PIP_VERSION=7.1.2",
"WORKON_HOME=/opt/virtualenvs"
],
"Cmd": [
"/bin/sh",
"-c",
"/bin/bash -c \"source $(which virtualenvwrapper.sh) \u0026\u0026 workon hyphe \u0026\u0026 twistd -y /app/hyphe_backend/core.tac --nodaemon\""
],
"Image": "hyphe_backend",
"Volumes": null,
"VolumeDriver": "",
"WorkingDir": "/app",
"Entrypoint": null,
"NetworkDisabled": false,
"MacAddress": "",
"OnBuild": null,
"Labels": {
"com.docker.compose.config-hash": "ca9a1ac82c07000611a8d3fbd25150a01b91b5efa620724b211efd78f6f073d1",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "hyphe",
"com.docker.compose.service": "backend",
"com.docker.compose.version": "1.4.2"
}
}
}
]
for the crawler, docker inspect 1cda1e275c6d
, I see nothing with HYPHE_MONGODB_HOST
[
{
"Id": "1cda1e275c6dbcba6a26310bcc1b9c5bc17bc7f71aac4154385dd4c0ad91b202",
"Created": "2015-10-02T06:38:00.9005942Z",
"Path": "/usr/bin/scrapyd",
"Args": [],
"State": {
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 0,
"Error": "",
"StartedAt": "2015-10-05T08:56:41.236482523Z",
"FinishedAt": "2015-10-05T08:56:43.202877854Z"
},
"Image": "f64bed7c1b74d337facab9a37eb87f72f4fafffc58c0cf7932bcdda45cb6763a",
"NetworkSettings": {
"Bridge": "",
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"HairpinMode": false,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"MacAddress": "",
"NetworkID": "",
"PortMapping": null,
"Ports": null,
"SandboxKey": "",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null
},
"ResolvConfPath": "/var/lib/docker/containers/1cda1e275c6dbcba6a26310bcc1b9c5bc17bc7f71aac4154385dd4c0ad91b202/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/1cda1e275c6dbcba6a26310bcc1b9c5bc17bc7f71aac4154385dd4c0ad91b202/hostname",
"HostsPath": "/var/lib/docker/containers/1cda1e275c6dbcba6a26310bcc1b9c5bc17bc7f71aac4154385dd4c0ad91b202/hosts",
"LogPath": "/var/lib/docker/containers/1cda1e275c6dbcba6a26310bcc1b9c5bc17bc7f71aac4154385dd4c0ad91b202/1cda1e275c6dbcba6a26310bcc1b9c5bc17bc7f71aac4154385dd4c0ad91b202-json.log",
"Name": "/hyphe_crawler_1",
"RestartCount": 0,
"Driver": "devicemapper",
"ExecDriver": "native-0.2",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/home/delaye/github/hyphe/config/scrapyd.config:/etc/scrapyd/conf.d/100-hyphe:rw"
],
"ContainerIDFile": "",
"LxcConf": null,
"Memory": 0,
"MemorySwap": 0,
"CpuShares": 0,
"CpuPeriod": 0,
"CpusetCpus": "",
"CpusetMems": "",
"CpuQuota": 0,
"BlkioWeight": 0,
"OomKillDisable": false,
"MemorySwappiness": null,
"Privileged": false,
"PortBindings": {},
"Links": [
"/hyphe_mongo_1:/hyphe_crawler_1/hyphe_mongo_1",
"/hyphe_mongo_1:/hyphe_crawler_1/mongo",
"/hyphe_mongo_1:/hyphe_crawler_1/mongo_1"
],
"PublishAllPorts": false,
"Dns": null,
"DnsSearch": null,
"ExtraHosts": [],
"VolumesFrom": [],
"Devices": null,
"NetworkMode": "default",
"IpcMode": "",
"PidMode": "",
"UTSMode": "",
"CapAdd": null,
"CapDrop": null,
"GroupAdd": null,
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"SecurityOpt": null,
"ReadonlyRootfs": false,
"Ulimits": null,
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"CgroupParent": "",
"ConsoleSize": [
0,
0
]
},
"GraphDriver": {
"Name": "devicemapper",
"Data": {
"DeviceId": "212",
"DeviceName": "docker-253:1-1062480-1cda1e275c6dbcba6a26310bcc1b9c5bc17bc7f71aac4154385dd4c0ad91b202",
"DeviceSize": "107374182400"
}
},
"Mounts": [
{
"Source": "/home/delaye/github/hyphe/config/scrapyd.config",
"Destination": "/etc/scrapyd/conf.d/100-hyphe",
"Mode": "rw",
"RW": true
},
{
"Name": "d5d65e766f8a780ad384c5ebb4df579ecaf35cc5233ee41f1249f8c6d20cb17f",
"Source": "/var/lib/docker/volumes/d5d65e766f8a780ad384c5ebb4df579ecaf35cc5233ee41f1249f8c6d20cb17f/_data",
"Destination": "/var/lib/scrapyd",
"Driver": "local",
"Mode": "",
"RW": true
},
{
"Name": "fcfbd6ab607303aa1e552743519861c9848e3d28ad5a020610aa11573812ec01",
"Source": "/var/lib/docker/volumes/fcfbd6ab607303aa1e552743519861c9848e3d28ad5a020610aa11573812ec01/_data",
"Destination": "/var/log/scrapyd",
"Driver": "local",
"Mode": "",
"RW": true
}
],
"Config": {
"Hostname": "1cda1e275c6d",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"6800/tcp": {}
},
"PublishService": "",
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8"
],
"Cmd": [
"/usr/bin/scrapyd"
],
"Image": "hyphe_crawler",
"Volumes": {
"/etc/scrapyd/conf.d/100-hyphe": {},
"/var/lib/scrapyd": {},
"/var/log/scrapyd": {}
},
"VolumeDriver": "",
"WorkingDir": "",
"Entrypoint": null,
"NetworkDisabled": false,
"MacAddress": "",
"OnBuild": null,
"Labels": {
"com.docker.compose.config-hash": "5094f8d7dda18f04d99970055148aae6a118551fec4ff622ee69caff36017e37",
"com.docker.compose.container-number": "1",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "hyphe",
"com.docker.compose.service": "crawler",
"com.docker.compose.version": "1.4.2"
}
}
}
]
Yep, there is nothing for the crawler – it is configured by the backend, and sent to the crawler.
I will try to rebuild the project on Wednesday – not sure to see what is the problem right now :-(
ok I can wait Wednesday :-) ...
Seems something changed, as the crawler could not be deployed too:
backend_1 | 2015-10-06 09:44:34+0000 [ERROR - --test-corpus--] Couldn't deploy crawler
# ...
backend_1 | 2015-10-06 09:45:04+0000 [ERROR - --test-corpus--] Couldn't destroy scrapyd spider: {u'status': u'error', u'message': u"[Errno 2] No such file or directory: '/var/lib/scrapyd/eggs/hyphe.--test-corpus--'", u'node_name': u'64c7f39082d7'}
@oncletom: is this the log from your local deployment from latest master code? Looks like this is the same issue as #143. I would bet on ScrapyD's version change and the "scrapy-deploy" against "scrapy deploy" thingy as discussed there.
Maybe?
I don't really know/understand what changed since and could have possibly broken something. Will have a deeper look tomorrow.
I believe so, they released a new version that changed a few things and I guess the dedicated container downloads this latest package
Hey @oncletom, we theoretically fixed the problem with Scrapy in the dockerfiles (cf https://github.com/medialab/hyphe/commit/ae7952ab0b043928a351c249bd90dc3ff1c8d689 ), could you try to rebuild and check first if it fixes your issue, and second if the mongo issue is also there?
Hi all, for me I still here :
frontend_1 | [Mon, 26 Oct 2015 13:53:41 GMT] "GET /" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
backend_1 | 2015-10-26 13:53:41+0000 [-] "172.17.42.1" - - [26/Oct/2015:13:53:40 +0000] "POST /hyphe-api/ HTTP/1.1" 200 247 "http://localhost:8000/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
backend_1 | 2015-10-26 13:53:41+0000 [-] "172.17.42.1" - - [26/Oct/2015:13:53:40 +0000] "POST /hyphe-api/ HTTP/1.1" 200 35 "http://localhost:8000/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
backend_1 | 2015-10-26 13:53:43+0000 [-] "172.17.42.1" - - [26/Oct/2015:13:53:43 +0000] "POST /hyphe-api/ HTTP/1.1" 200 247 "http://localhost:8000/" "Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.89 Safari/537.36"
with this message : YOU ARE DISCONNECTED FROM HYPHE :-(
@Elcep Apparently your backend answers, so I guess the problem is CORS access, can you open the console in your browser and check if there's any error visible?
Yes 22 in chrome :
XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
login.js:163 Error loading corpus list
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
(index):1 XMLHttpRequest cannot load http://localhost:6978/hyphe-api/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8000' is therefore not allowed access.
service_hyphe_api.js:575 [Error: fail]
So yes it is the CORS issue, normally if you do git pull and docker-compose build, it should build you with the proper config so that you don't have the issue
hohohoh :-D it work !!! something going on ! Thank you @boogheta
Great!
Hey @ElCep, FYI we finally released a new version with a more generic Docker installation process which should allow you to easily install Hyphe now on all OS!
Dear all and dear @oncletom , I'am a Linux user, but in a flavor not well supported (yet) by hyphe (Fedora 22). So I want take the docker solution... my procedure is :
After something like 15 min it seem to be ok in my terminal ... and in docker I have :
f98b80104f4c hyphe_frontend "/app/node_modules/.b" 28 minutes ago Up 28 minutes 0.0.0.0:8000->8000/tcp hyphe_frontend_1
I have seen on this the URL of my service ... for me it's
But nothing happens in my browser ... However in my terminal :
What is going on ? Can I have a hand ?
Thank's
E.