BaseAdresseNationale / addok-docker

Conteneurs Addok pour Docker
157 stars 29 forks source link

Redis RDB Error #9

Closed sneycampos closed 5 years ago

sneycampos commented 5 years ago

I've the follow error when trying to docker-compose up...

Someone can help with this problem? Thanks.

addok-redis_1 | 1:C 03 Jun 21:43:02.363 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo addok-redis_1 | 1:C 03 Jun 21:43:02.363 # Redis version=4.0.2, bits=64, commit=00000000, modified=0, pid=1, just started addok-redis_1 | 1:C 03 Jun 21:43:02.363 # Configuration loaded addok-redis1 | ._
addok-redis1 | .-__ ''-._ addok-redis_1 | _.- .. ''-. Redis 4.0.2 (00000000/0) 64 bit addok-redis1 | .-`` .-.\/ ., ''-.
addok-redis_1 | ( ' , .-|, ) Running in standalone mode addok-redis1 | |`-.-...- ...-.`-._|' _.-'| Port: 6379 addok-redis1 | | `-. ._ / _.-' | PID: 1 addok-redis_1 |-. `-. -./ _.-' _.-' addok-redis_1 | |-.`-. `-..-' .-'.-'|
addok-redis1 | | `-.-._ _.-'_.-' | http://redis.io addok-redis_1 |-. `-.-.__.-'_.-' _.-' addok-redis_1 | |-.`-. -.__.-' _.-'_.-'| addok-redis_1 | |-.`-. .-'.-' |
addok-redis1 | `-. -._-..-'.-' .-'
addok-redis1 | `-. `-.
.-' _.-'
addok-redis1 | `-. _.-'
addok-redis_1 | `-.__.-'
addok-redis_1 | addok-redis_1 | 1:M 03 Jun 21:43:02.365 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. addok-redis_1 | 1:M 03 Jun 21:43:02.365 # Server initialized addok-redis_1 | 1:M 03 Jun 21:43:02.365 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. addok-redis_1 | 1:M 03 Jun 21:43:02.365 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled. addok_1 | Will log queries addok_1 | Will log Not Found addok-redis_1 | 1:M 03 Jun 21:43:02.365 # Short read or OOM loading DB. Unrecoverable error, aborting now. addok-redis_1 | 1:M 03 Jun 21:43:02.365 # Internal error in RDB reading function at rdb.c:1642 -> Unexpected EOF reading RDB file addok_1 | Will log slow queries addok_1 | ATTRIBUTION = 'BAN' addok_1 | LICENCE = 'ODbL 1.0' addok_1 | EXTRA_FIELDS = [{'key': 'citycode'},] addok_1 | FILTERS = ["type", "postcode", "citycode"] addok_1 | QUERY_PROCESSORS_PYPATHS = [ addok_1 | 'addok.helpers.text.check_query_length', addok_1 | "addok_france.extract_address", addok_1 | "addok_france.clean_query", addok_1 | "addok_france.remove_leading_zeros", addok_1 | ] addok_1 | SEARCH_RESULT_PROCESSORS_PYPATHS = [ addok_1 | "addok.helpers.results.match_housenumber", addok_1 | "addok_france.make_labels", addok_1 | "addok.helpers.results.score_by_importance", addok_1 | "addok.helpers.results.score_by_autocomplete_distance", addok_1 | "addok.helpers.results.score_by_ngram_distance", addok_1 | "addok.helpers.results.score_by_geo_distance", addok_1 | ] addok_1 | PROCESSORS_PYPATHS = [ addok_1 | "addok.helpers.text.tokenize", addok_1 | "addok.helpers.text.normalize", addok_1 | "addok_france.glue_ordinal", addok_1 | "addok_france.fold_ordinal", addok_1 | "addok_france.flag_housenumber", addok_1 | "addok.helpers.text.synonymize", addok_1 | "addok_fr.phonemicize", addok_1 | ] addok_1 | addok_1 | SQLITE_DB_PATH = '/data/addok.db' addok_1 | LOG_DIR = '/logs' addok_1 | LOG_QUERIES = True addok_1 | LOG_NOT_FOUND = True addok_1 | SLOW_QUERIES = 200 addok-redis_1 | [offset 0] Checking RDB file dump.rdb addok-redis_1 | --- RDB ERROR DETECTED --- addok-redis_1 | [offset 0] Unexpected EOF reading RDB file addok-redis_1 | [additional info] While doing: start addok-redis_1 | [additional info] Reading type 0 (string) addok-redis_1 | [info] 0 keys read addok-redis_1 | [info] 0 expires addok-redis_1 | [info] 0 already expired addok_addok-redis_1 exited with code 1 addok_1 | [2019-06-03 21:43:03 +0000] [9] [INFO] Starting gunicorn 19.9.0 addok_1 | [2019-06-03 21:43:03 +0000] [9] [INFO] Listening at: http://0.0.0.0:7878 (9) addok_1 | [2019-06-03 21:43:03 +0000] [9] [INFO] Using worker: sync addok_1 | [2019-06-03 21:43:03 +0000] [12] [INFO] Booting worker with pid: 12

sneycampos commented 5 years ago

redis log:

addok-redis_1 | 1:M 03 Jun 21:43:02.365 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. addok-redis_1 | 1:M 03 Jun 21:43:02.365 # Server initialized addok-redis_1 | 1:M 03 Jun 21:43:02.365 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. addok-redis_1 | 1:M 03 Jun 21:43:02.365 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled. addok-redis_1 | 1:M 03 Jun 21:43:02.365 # Short read or OOM loading DB. Unrecoverable error, aborting now. addok-redis_1 | 1:M 03 Jun 21:43:02.365 # Internal error in RDB reading function at rdb.c:1642 -> Unexpected EOF reading RDB file addok-redis_1 | [offset 0] Checking RDB file dump.rdb addok-redis_1 | --- RDB ERROR DETECTED --- addok-redis_1 | [offset 0] Unexpected EOF reading RDB file addok-redis_1 | [additional info] While doing: start addok-redis_1 | [additional info] Reading type 0 (string) addok-redis_1 | [info] 0 keys read addok-redis_1 | [info] 0 expires addok-redis_1 | [info] 0 already expired addok_addok-redis_1 exited with code 1

jdesboeufs commented 5 years ago

Maybe this: OOM loading DB

Out of memory ?

sneycampos commented 5 years ago

Yes, its a memory problem. I was running this on my laptop with 8GB RAM, after this comment i've run in another machine and all ok

sneycampos commented 5 years ago

@jdesboeufs have any plans to port this script to run using Aerospike as option instead Redis?

jdesboeufs commented 5 years ago

We have no plan for Aerospike. You should ask on https://github.com/addok/addok