okfn-brasil / querido-diario

📰 Diários oficiais brasileiros acessíveis a todos | 📰 Brazilian government gazettes, accessible to everyone.
https://queridodiario.ok.org.br/
MIT License
1.11k stars 409 forks source link

Error when starting service, database keys missing? #38

Closed danicuki closed 6 years ago

danicuki commented 6 years ago

When I try to run the service using docker-compose up and access http://localhost:8080/ I get this error:

Request failed with status code 400

The error that shows in the terminal is related with database:

Could not find foreign keys between these entities, 
No relation found between bidding_exemptions and gazette
juniorcarvalho commented 6 years ago

I'm also having the same problem and still can not find any solution.

Irio commented 6 years ago

Would you mind posting all the logs from the docker-compose up command?

It seems that tables don't exist because no gazette was collected using Scrapy. If that's the case, would be a good time for the project to automate data collection.

danicuki commented 6 years ago
 $ docker-compose up
Starting diario-oficial_postgres_1 ... done
Starting diario-oficial_redis_1    ... done
Starting diario-oficial_rabbitmq_1 ... done
Starting diario-oficial_api_1        ... done
Starting diario-oficial_processing_1 ... done
Starting diario-oficial_web_1        ... done
Attaching to diario-oficial_rabbitmq_1, diario-oficial_postgres_1, diario-oficial_redis_1, diario-oficial_processing_1, diario-oficial_api_1, diario-oficial_web_1
redis_1       | 1:C 17 May 22:03:50.021 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
postgres_1    | 2018-05-17 22:03:50.712 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
postgres_1    | 2018-05-17 22:03:50.713 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres_1    | 2018-05-17 22:03:50.725 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1    | 2018-05-17 22:03:50.819 UTC [19] LOG:  database system was interrupted; last known up at 2018-05-13 15:31:02 UTC
postgres_1    | 2018-05-17 22:03:51.269 UTC [19] LOG:  database system was not properly shut down; automatic recovery in progress
redis_1       |                 _._
redis_1       |            _.-``__ ''-._
redis_1       |       _.-``    `.  `_.  ''-._           Redis 3.2.11 (00000000/0) 64 bit
redis_1       |   .-`` .-```.  ```\/    _.,_ ''-._
redis_1       |  (    '      ,       .-`  | `,    )     Running in standalone mode
redis_1       |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
redis_1       |  |    `-._   `._    /     _.-'    |     PID: 1
redis_1       |   `-._    `-._  `-./  _.-'    _.-'
redis_1       |  |`-._`-._    `-.__.-'    _.-'_.-'|
redis_1       |  |    `-._`-._        _.-'_.-'    |           http://redis.io
redis_1       |   `-._    `-._`-.__.-'_.-'    _.-'
redis_1       |  |`-._`-._    `-.__.-'    _.-'_.-'|
redis_1       |  |    `-._`-._        _.-'_.-'    |
redis_1       |   `-._    `-._`-.__.-'_.-'    _.-'
redis_1       |       `-._    `-.__.-'    _.-'
redis_1       |           `-._        _.-'
redis_1       |               `-.__.-'
redis_1       |
redis_1       | 1:M 17 May 22:03:50.056 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1       | 1:M 17 May 22:03:50.056 # Server started, Redis version 3.2.11
postgres_1    | 2018-05-17 22:03:51.297 UTC [19] LOG:  redo starts at 0/15F36F0
redis_1       | 1:M 17 May 22:03:50.062 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
postgres_1    | 2018-05-17 22:03:51.318 UTC [19] LOG:  invalid record length at 0/15F37D0: wanted 24, got 0
postgres_1    | 2018-05-17 22:03:51.318 UTC [19] LOG:  redo done at 0/15F3798
redis_1       | 1:M 17 May 22:03:50.064 * DB loaded from disk: 0.002 seconds
redis_1       | 1:M 17 May 22:03:50.064 * The server is now ready to accept connections on port 6379
postgres_1    | 2018-05-17 22:03:51.383 UTC [1] LOG:  database system is ready to accept connections
api_1         | Listening on port 3000
api_1         | Attempting to connect to the database...
api_1         | Connection successful
web_1         |
web_1         | > serenata@1.0.0 dev /mnt/code
web_1         | > nuxt
web_1         |
processing_1  | /usr/local/lib/python3.6/site-packages/celery/platforms.py:795: RuntimeWarning: You're running the worker with superuser privileges: this is
processing_1  | absolutely not recommended!
processing_1  |
processing_1  | Please specify a different user using the -u option.
processing_1  |
processing_1  | User information: uid=0 euid=0 gid=0 egid=0
processing_1  |
processing_1  |   uid=uid, euid=euid, gid=gid, egid=egid,
processing_1  | [2018-05-17 22:03:54,142: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@rabbitmq:5672//: [Errno 111] Connection refused.
processing_1  | Trying again in 2.00 seconds...
processing_1  |
rabbitmq_1    | 2018-05-17 22:03:55.378 [info] <0.33.0> Application lager started on node rabbit@5f7052a2efff
processing_1  | /usr/local/lib/python3.6/site-packages/celery/platforms.py:795: RuntimeWarning: You're running the worker with superuser privileges: this is
processing_1  | absolutely not recommended!
processing_1  |
processing_1  | Please specify a different user using the -u option.
processing_1  |
processing_1  | User information: uid=0 euid=0 gid=0 egid=0
processing_1  |
processing_1  |   uid=uid, euid=euid, gid=gid, egid=egid,
processing_1  | [2018-05-17 22:03:56,105: INFO/Beat] beat: Starting...
processing_1  | [2018-05-17 22:03:56,151: ERROR/Beat] beat: Connection error: [Errno 111] Connection refused. Trying again in 2.0 seconds...
processing_1  | [2018-05-17 22:03:56,155: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@rabbitmq:5672//: [Errno 111] Connection refused.
processing_1  | Trying again in 4.00 seconds...
processing_1  |
processing_1  | [2018-05-17 22:03:58,162: ERROR/Beat] beat: Connection error: [Errno 111] Connection refused. Trying again in 4.0 seconds...
rabbitmq_1    | 2018-05-17 22:03:58.833 [info] <0.33.0> Application crypto started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.871 [info] <0.33.0> Application mnesia started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.873 [info] <0.33.0> Application inets started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.873 [info] <0.33.0> Application xmerl started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.873 [info] <0.33.0> Application os_mon started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.873 [info] <0.33.0> Application recon started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.874 [info] <0.33.0> Application jsx started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.874 [info] <0.33.0> Application asn1 started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.874 [info] <0.33.0> Application public_key started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.875 [info] <0.33.0> Application ssl started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.875 [info] <0.33.0> Application ranch started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.875 [info] <0.33.0> Application ranch_proxy_protocol started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.875 [info] <0.33.0> Application rabbit_common started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.877 [info] <0.212.0>
rabbitmq_1    |  Starting RabbitMQ 3.7.5 on Erlang 20.1.7
rabbitmq_1    |  Copyright (C) 2007-2018 Pivotal Software, Inc.
rabbitmq_1    |  Licensed under the MPL.  See http://www.rabbitmq.com/
rabbitmq_1    |
rabbitmq_1    |   ##  ##
rabbitmq_1    |   ##  ##      RabbitMQ 3.7.5. Copyright (C) 2007-2018 Pivotal Software, Inc.
rabbitmq_1    |   ##########  Licensed under the MPL.  See http://www.rabbitmq.com/
rabbitmq_1    |   ######  ##
rabbitmq_1    |   ##########  Logs: <stdout>
rabbitmq_1    |
rabbitmq_1    |               Starting broker...
rabbitmq_1    | 2018-05-17 22:03:58.879 [info] <0.212.0>
rabbitmq_1    |  node           : rabbit@5f7052a2efff
rabbitmq_1    |  home dir       : /var/lib/rabbitmq
rabbitmq_1    |  config file(s) : /etc/rabbitmq/rabbitmq.conf
rabbitmq_1    |  cookie hash    : lNYdCTgHsZs0l9zmVDbOBQ==
rabbitmq_1    |  log(s)         : <stdout>
rabbitmq_1    |  database dir   : /var/lib/rabbitmq/mnesia/rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:58.971 [info] <0.253.0> Memory high watermark set to 3189 MiB (3344231628 bytes) of 7973 MiB (8360579072 bytes) total
rabbitmq_1    | 2018-05-17 22:03:58.979 [info] <0.255.0> Enabling free disk space monitoring
rabbitmq_1    | 2018-05-17 22:03:58.979 [info] <0.255.0> Disk free limit set to 50MB
rabbitmq_1    | 2018-05-17 22:03:58.983 [info] <0.257.0> Limiting to approx 1048476 file handles (943626 sockets)
rabbitmq_1    | 2018-05-17 22:03:58.983 [info] <0.258.0> FHC read buffering:  OFF
rabbitmq_1    | 2018-05-17 22:03:58.983 [info] <0.258.0> FHC write buffering: ON
rabbitmq_1    | 2018-05-17 22:03:58.987 [info] <0.212.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbitmq_1    | 2018-05-17 22:03:59.011 [info] <0.212.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbitmq_1    | 2018-05-17 22:03:59.011 [info] <0.212.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
rabbitmq_1    | 2018-05-17 22:03:59.013 [info] <0.212.0> Priority queues enabled, real BQ is rabbit_variable_queue
rabbitmq_1    | 2018-05-17 22:03:59.031 [info] <0.282.0> Starting rabbit_node_monitor
rabbitmq_1    | 2018-05-17 22:03:59.050 [info] <0.310.0> Making sure data directory '/var/lib/rabbitmq/mnesia/rabbit@5f7052a2efff/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
rabbitmq_1    | 2018-05-17 22:03:59.055 [info] <0.310.0> Starting message stores for vhost '/'
rabbitmq_1    | 2018-05-17 22:03:59.055 [info] <0.314.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
rabbitmq_1    | 2018-05-17 22:03:59.058 [info] <0.310.0> Started message store of type transient for vhost '/'
rabbitmq_1    | 2018-05-17 22:03:59.058 [info] <0.317.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
rabbitmq_1    | 2018-05-17 22:03:59.059 [warning] <0.317.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch
rabbitmq_1    | 2018-05-17 22:03:59.063 [info] <0.310.0> Started message store of type persistent for vhost '/'
rabbitmq_1    | 2018-05-17 22:03:59.077 [info] <0.360.0> started TCP Listener on [::]:5672
rabbitmq_1    | 2018-05-17 22:03:59.077 [info] <0.212.0> Setting up a table for connection tracking on this node: tracked_connection_on_node_rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:59.078 [info] <0.212.0> Setting up a table for per-vhost connection counting on this node: tracked_connection_per_vhost_on_node_rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:59.078 [info] <0.33.0> Application rabbit started on node rabbit@5f7052a2efff
rabbitmq_1    | 2018-05-17 22:03:59.439 [info] <0.5.0> Server startup complete; 0 plugins started.
rabbitmq_1    |  completed with 0 plugins.
rabbitmq_1    | 2018-05-17 22:04:00.167 [info] <0.366.0> accepting AMQP connection <0.366.0> (172.18.0.6:49128 -> 172.18.0.4:5672)
rabbitmq_1    | 2018-05-17 22:04:00.170 [info] <0.366.0> connection <0.366.0> (172.18.0.6:49128 -> 172.18.0.4:5672): user 'guest' authenticated and granted access to vhost '/'
processing_1  | [2018-05-17 22:04:00,171: INFO/MainProcess] Connected to amqp://guest:**@rabbitmq:5672//
rabbitmq_1    | 2018-05-17 22:04:00.179 [info] <0.375.0> accepting AMQP connection <0.375.0> (172.18.0.6:49130 -> 172.18.0.4:5672)
rabbitmq_1    | 2018-05-17 22:04:00.181 [info] <0.375.0> connection <0.375.0> (172.18.0.6:49130 -> 172.18.0.4:5672): user 'guest' authenticated and granted access to vhost '/'
processing_1  | [2018-05-17 22:04:00,192: INFO/MainProcess] mingle: searching for neighbors
rabbitmq_1    | 2018-05-17 22:04:00.213 [info] <0.395.0> accepting AMQP connection <0.395.0> (172.18.0.6:49134 -> 172.18.0.4:5672)
rabbitmq_1    | 2018-05-17 22:04:00.217 [info] <0.395.0> connection <0.395.0> (172.18.0.6:49134 -> 172.18.0.4:5672): user 'guest' authenticated and granted access to vhost '/'
processing_1  | [2018-05-17 22:04:01,245: INFO/MainProcess] mingle: all alone
processing_1  | [2018-05-17 22:04:01,276: INFO/MainProcess] celery@6f5b1db87e80 ready.
rabbitmq_1    | 2018-05-17 22:04:02.172 [info] <0.427.0> accepting AMQP connection <0.427.0> (172.18.0.6:49140 -> 172.18.0.4:5672)
processing_1  | [2018-05-17 22:04:02,175: INFO/Beat] Scheduler: Sending due task tasks.parse_sections() (tasks.parse_sections)
rabbitmq_1    | 2018-05-17 22:04:02.174 [info] <0.427.0> connection <0.427.0> (172.18.0.6:49140 -> 172.18.0.4:5672): user 'guest' authenticated and granted access to vhost '/'
processing_1  | [2018-05-17 22:04:02,182: INFO/MainProcess] Received task: tasks.parse_sections[e317ba6b-6fc9-46d0-9173-774ebe0551a8]
rabbitmq_1    | 2018-05-17 22:04:02.310 [info] <0.438.0> accepting AMQP connection <0.438.0> (172.18.0.6:49144 -> 172.18.0.4:5672)
rabbitmq_1    | 2018-05-17 22:04:02.313 [info] <0.438.0> connection <0.438.0> (172.18.0.6:49144 -> 172.18.0.4:5672): user 'guest' authenticated and granted access to vhost '/'
processing_1  | /usr/local/lib/python3.6/site-packages/celery/platforms.py:795: RuntimeWarning: You're running the worker with superuser privileges: this is
processing_1  | absolutely not recommended!
processing_1  |
processing_1  | Please specify a different user using the -u option.
processing_1  |
processing_1  | User information: uid=0 euid=0 gid=0 egid=0
processing_1  |
processing_1  |   uid=uid, euid=euid, gid=gid, egid=egid,
processing_1  | [2018-05-17 22:04:02,320: INFO/ForkPoolWorker-3] Task tasks.parse_sections[e317ba6b-6fc9-46d0-9173-774ebe0551a8] succeeded in 0.13645180000457913s: None
processing_1  | [2018-05-17 22:04:02,320: INFO/MainProcess] Received task: tasks.parse_bidding_exemptions[d1fc5a1a-0e35-440c-b923-e89aa86c3dde]
processing_1  | /usr/local/lib/python3.6/site-packages/celery/platforms.py:795: RuntimeWarning: You're running the worker with superuser privileges: this is
processing_1  | absolutely not recommended!
processing_1  |
processing_1  | Please specify a different user using the -u option.
processing_1  |
processing_1  | User information: uid=0 euid=0 gid=0 egid=0
processing_1  |
processing_1  |   uid=uid, euid=euid, gid=gid, egid=egid,
processing_1  | [2018-05-17 22:04:02,364: INFO/ForkPoolWorker-2] Task tasks.parse_bidding_exemptions[d1fc5a1a-0e35-440c-b923-e89aa86c3dde] succeeded in 0.04215120000299066s: None
web_1         |  WARNING  Compiled with 14 warnings22:04:03
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5892:3: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5893:3: variable '--columnGap' is undefined and used without a fallback
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5894:3: variable '--columnGap' is undefined and used without a fallback
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5896:5: variable '--columnGap' is undefined and used without a fallback
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5897:5: variable '--columnGap' is undefined and used without a fallback
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5899:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-0 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5901:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-1 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5903:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-2 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5905:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-3 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5907:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-4 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5909:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-5 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5911:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-6 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5913:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-7 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |  warning  in ./node_modules/bulma/bulma.sass
web_1         |
web_1         | (Emitted value instead of an instance of Error) postcss-custom-properties: /mnt/code/node_modules/bulma/bulma.sass:5915:5: Custom property ignored: not scoped to the top-level :root element (.columns.is-variable.is-8 { ... --columnGap: ... })
web_1         |
web_1         |  @ ./node_modules/bulma/bulma.sass 4:14-152 13:3-17:5 14:22-160
web_1         |  @ ./.nuxt/App.js
web_1         |  @ ./.nuxt/index.js
web_1         |  @ ./.nuxt/client.js
web_1         |  @ multi webpack-hot-middleware/client?name=client&reload=true&timeout=30000&path=/__webpack_hmr ./.nuxt/client.js
web_1         |
web_1         |
web_1         |  OPEN  http://localhost:8080
web_1         |
juniorcarvalho commented 6 years ago

I think the problem is not creating the tables. When in settings.py we uncomment the line 'gazette.pipelines.PostgreSQLPipeline': 3 and the line 'gazette.pipelines.PdfParsingPipeline': 2,', and we tried to run the scrapy crawl, the following error explode: from database.models import Gazette, db_connect, create_tables ModuleNotFoundError: No module named 'database'

cuducos commented 6 years ago

I'm having a different issue, also with the web container:

web_1         |
web_1         | > serenata@1.0.0 dev /mnt/code
web_1         | > nuxt
web_1         |
web_1         |
web_1         |  ERROR
web_1         |
web_1         |   Error: Could not compile template /usr/local/lib/node_modules/nuxt/lib/app/App  .js: Cannot resolve "bulma" from "/mnt/code/bulma"
web_1         |
web_1         |   - builder.js:471 Promise.all.templatesFiles.map
web_1         |     [lib]/[nuxt]/lib/builder/builder.js:471:17
web_1         |
web_1         |
web_1         |
Irio commented 6 years ago

@danicuki I recently added a couple of commits that hopefully will fix the issues both me and you mentioned. https://github.com/okfn-brasil/diario-oficial/commit/48201be2f77d082781bf2eec9d57b34ccb3b00df and https://github.com/okfn-brasil/diario-oficial/commit/0e36994198e1e09e503928d2c4a6af8f6e6abb31

Can you try the following and tell me if it gets fixed? I was able to reproduce it and did it for me.

$ docker-compose stop
$ docker system prune
$ make setup
$ docker-compose up
danicuki commented 6 years ago
$ make setup
cp .env.example .env
docker-compose pull
Pulling postgres   ... done
Pulling api        ... done
Pulling rabbitmq   ... done
Pulling redis      ... done
Pulling processing ... done
Pulling web        ... done
docker-compose build
postgres uses an image, skipping
api uses an image, skipping
rabbitmq uses an image, skipping
redis uses an image, skipping
Building processing
Step 1/6 : FROM python:3
 ---> d69bc9d9b016
Step 2/6 : WORKDIR /mnt/code
 ---> Using cache
 ---> 83d6b34e00e8
Step 3/6 : RUN apt-get update   && apt-get -y install poppler-utils postgresql-client
 ---> Using cache
 ---> 4bc3f0d523e1
Step 4/6 : COPY requirements.txt requirements.txt
 ---> Using cache
 ---> a1280d1189af
Step 5/6 : RUN pip install --no-cache-dir -r requirements.txt
 ---> Using cache
 ---> 5ef8f31eb31b
Step 6/6 : CMD ["celery", "-A", "tasks", "worker", "-B", "--loglevel=INFO"]
 ---> Using cache
 ---> 7c78b185d45c
Successfully built 7c78b185d45c
Successfully tagged diario-oficial_processing:latest
Building web
Step 1/11 : FROM node:9.11.1-slim
 ---> fa35876db29d
Step 2/11 : ENV HOME /mnt/code
 ---> Using cache
 ---> 16a5259cf14a
Step 3/11 : ENV HOST 0.0.0.0
 ---> Using cache
 ---> 526addd2da62
Step 4/11 : ENV PORT 8080
 ---> Using cache
 ---> 6faa6ad11d1d
Step 5/11 : RUN npm install --quiet --global nuxt vue-cli
 ---> Using cache
 ---> 1a16eda08839
Step 6/11 : COPY package*.json $HOME/
 ---> Using cache
 ---> 275e3ff8fc8f
Step 7/11 : WORKDIR $HOME
 ---> Using cache
 ---> 2d6ec8ab7602
Step 8/11 : RUN npm install
 ---> Using cache
 ---> 2cdd2c7ec4c1
Step 9/11 : COPY . $HOME
 ---> 9a0c62851ae6
Step 10/11 : EXPOSE $PORT
 ---> Running in af119cbf8092
Removing intermediate container af119cbf8092
 ---> f047f12f1098
Step 11/11 : CMD npm run build && npm run start
 ---> Running in 216ee6a4f662
Removing intermediate container 216ee6a4f662
 ---> 2db952b80ef7
Successfully built 2db952b80ef7
Successfully tagged diario-oficial_web:latest
make seed
docker-compose up --detach postgres
Creating network "diario-oficial_default" with the default driver
Creating diario-oficial_postgres_1 ... done
docker-compose run --rm processing python3 -c "import database; database.initialize()"
Creating diario-oficial_rabbitmq_1 ... done
Creating diario-oficial_redis_1    ... done
Starting diario-oficial_postgres_1 ... done
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: foreign key constraint "gazettes_territory_id_fkey" cannot be implemented
DETAIL:  Key columns "territory_id" and "id" are of incompatible types: integer and character varying.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/code/database/models.py", line 22, in initialize_database
    create_tables(engine)
  File "/mnt/code/database/models.py", line 17, in create_tables
    DeclarativeBase.metadata.create_all(engine)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4004, in create_all
    tables=tables)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1940, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1549, in _run_visitor
    **kwargs).traverse_single(element)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 757, in visit_metadata
    _is_metadata_operation=True)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 121, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 791, in visit_table
    include_foreign_key_constraints=include_foreign_key_constraints
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1009, in _execute_ddl
    compiled
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) foreign key constraint "gazettes_territory_id_fkey" cannot be implemented
DETAIL:  Key columns "territory_id" and "id" are of incompatible types: integer and character varying.
 [SQL: '\nCREATE TABLE gazettes (\n\tid SERIAL NOT NULL, \n\tsource_text TEXT, \n\tdate DATE, \n\tis_extra_edition BOOLEAN, \n\tis_parsed BOOLEAN, \n\tpower VARCHAR, \n\tfile_checksum VARCHAR, \n\tfile_path VARCHAR, \n\tfile_url VARCHAR, \n\tscraped_at TIMESTAMP WITHOUT TIME ZONE, \n\tcreated_at TIMESTAMP WITHOUT TIME ZONE, \n\tterritory_id INTEGER, \n\tPRIMARY KEY (id), \n\tUNIQUE (territory_id, date, file_checksum), \n\tFOREIGN KEY(territory_id) REFERENCES territories (id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/f405)
make[1]: *** [seed] Error 1
make: *** [setup] Error 2
Irio commented 6 years ago

@danicuki Can you try again after a git pull?

danicuki commented 6 years ago

screen shot 2018-05-31 at 2 46 58 pm