Using docker with external DB

[arainero]

I am experiencing an issue when using an external database that data will not show up on the website. I have gotten this to fully work when using the docker database and making no modifications.

I configured an external database in the homer.env file. I set the following fields:

DB_USER=homer, DB_PASS=XXXXXXX, USE_REMOTE_MYSQL=true, and DB_HOT=x.x.x.x

After doing a docker-build and docker-compose up I see the following:

[root@dockerserver homer-docker]# docker-compose up
Starting mysql ... done
Recreating homer-cron ...
Starting homer-kamailio ...
Recreating homer-webapp ... done
Recreating bootstrap-mysql ... done
Attaching to mysql, homer-kamailio, homer-webapp, homer-cron, bootstrap-mysql
mysql        | MySQL existing, not necessary.
homer-kamailio | Kamailio container detected MySQL is running & bootstrapped
homer-webapp | Homer web app container detected MySQL is running & bootstrapped
homer-cron   | Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 325.
homer-cron   | Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 327.
homer-webapp | Module php5 already enabled
homer-kamailio |  0(16) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module
homer-webapp | Enabling module rewrite.
bootstrap-mysql | Checking if mysql is alive...
homer-cron   | Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 511.
homer-cron   | Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 511.
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 0 seconds so far
homer-kamailio |  0(16) WARNING: <core> [core/socket_info.c:1314]: fix_hostname(): could not rev. resolve 0.0.0.0
homer-kamailio | config file ok, exiting...
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | Listening on
homer-kamailio |              udp: 0.0.0.0:9060
homer-kamailio | Aliases:
homer-kamailio |
homer-kamailio |  0(17) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module
homer-kamailio |  0(17) WARNING: <core> [core/socket_info.c:1314]: fix_hostname(): could not rev. resolve 0.0.0.0
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | Listening on
homer-kamailio |              udp: 0.0.0.0:9060
homer-kamailio | Aliases:
homer-kamailio |
homer-webapp | To activate the new configuration, you need to run:
homer-webapp |   service apache2 restart
homer-kamailio |  0(17) INFO: sipcapture [sipcapture.c:455]: parse_table_names(): INFO: table name:sip_capture
homer-webapp | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.4. Set the 'ServerName' directive globally to suppress this message
homer-kamailio |  0(17) INFO: <core> [core/udp_server.c:153]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
homer-kamailio |  0(17) INFO: <core> [core/udp_server.c:203]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
mysql exited with code 0
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 2 seconds so far
mysql exited with code 0
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 4 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 6 seconds so far
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 8 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 10 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 12 seconds so far
mysql exited with code 0
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 14 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 16 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 18 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 20 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 22 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 24 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 26 seconds so far
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 28 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 30 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 32 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 34 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 36 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 38 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 40 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 42 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 44 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 46 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 48 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 50 seconds so far
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 52 seconds so far
mysql exited with code 0
bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited 54 seconds so far

This just goes on and on.

At this point I am able to log into the website, but I can not get any data to show when doing a search. I do see that data is being pushed into my external database now, however.

To help with the "bootstrap-mysql | Bootstrap container is waiting for mysql, sleeping 2 seconds, intentionally waited X seconds so far" issue I tried manually setting "mysql_started=false" to "mysql_started=true" in the "bootstrap-data/bootstrap.sh" file.

After doing a docker-build and docker-compose up I now see the following:

Starting mysql ... done
Starting homer-cron ...
Starting homer-kamailio ...
Starting homer-webapp ... done
Starting bootstrap-mysql ... done
Attaching to mysql, homer-webapp, homer-cron, homer-kamailio, bootstrap-mysql
homer-webapp | Homer web app container detected MySQL is running & bootstrapped
mysql        | MySQL existing, not necessary.
homer-webapp | mkdir: cannot create directory '/var/www/html/api/tmp': File exists
homer-webapp | Module php5 already enabled
homer-cron   | Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 325.
homer-cron   | Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 327.
homer-cron   | Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 511.
homer-cron   | Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 511.
bootstrap-mysql | Checking if mysql is alive...
bootstrap-mysql | ERROR 1045 (28000): Access denied for user 'root'@'10.10.10.220' (using password: YES)
homer-webapp | Module rewrite already enabled
homer-kamailio | Kamailio container detected MySQL is running & bootstrapped
homer-kamailio |  0(15) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module
homer-webapp | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.3. Set the 'ServerName' directive globally to suppress this message
bootstrap-mysql | Beginning initial data load....
bootstrap-mysql | Creating Databases...
homer-kamailio |  0(15) WARNING: <core> [core/socket_info.c:1314]: fix_hostname(): could not rev. resolve 0.0.0.0
homer-kamailio | config file ok, exiting...
bootstrap-mysql | ERROR 1045 (28000): Access denied for user 'root'@'10.10.10.220' (using password: YES)
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | Listening on
homer-kamailio |              udp: 0.0.0.0:9060
homer-kamailio | Aliases:
homer-kamailio |
homer-kamailio |  0(16) INFO: <core> [core/sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module
homer-kamailio |  0(16) WARNING: <core> [core/socket_info.c:1314]: fix_hostname(): could not rev. resolve 0.0.0.0
homer-kamailio |  0(16) INFO: sipcapture [sipcapture.c:455]: parse_table_names(): INFO: table name:sip_capture
homer-kamailio |  0(16) INFO: <core> [core/udp_server.c:153]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
homer-kamailio |  0(16) INFO: <core> [core/udp_server.c:203]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | loading modules under config path: //usr/lib/x86_64-linux-gnu/kamailio/modules/
homer-kamailio | Listening on
homer-kamailio |              udp: 0.0.0.0:9060
homer-kamailio | Aliases:
homer-kamailio |
bootstrap-mysql | ERROR 1045 (28000): Access denied for user 'root'@'10.10.10.220' (using password: YES)
bootstrap-mysql | ERROR 1045 (28000): Access denied for user 'root'@'10.10.10.220' (using password: YES)
bootstrap-mysql | Creating Tables...
bootstrap-mysql | ERROR 1062 (23000) at line 34: Duplicate entry '1' for key 'PRIMARY'
bootstrap-mysql | Data bootstrapped semaphore written to /homer-semaphore/.bootstrapped
bootstrap-mysql exited with code 0
mysql exited with code 0
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.
mysql        | MySQL existing, not necessary.

when I do that I no longer see the message "Bootstrap container is waiting for mysql, sleeping X seconds, intentionally waited 6 seconds so far" but I see "bootstrap-mysql | ERROR 1045 (28000): Access denied for user 'root'@'10.10.10.220' (using password: YES)" in the output.

In both situations I still can not pull up any data via the website.

Any help would be appreciated with this,

Thank you.

[arainero]

Something that I noticed is that SOME data appears to show for certain places, but not when searching. For instance, on the dashboard the SIPCapture Charts is populated:

But when I go to the search page nothing is returned.

@lmangani do you have any idea what is going on?

[lmangani]

@arainero as a starter, make sure you hit "clear" on the form to avoid ghost parameters impacting search - the statistics are generated from actual messages, so both should be on the system most definitely.

[arainero]

I tried clearing before doing a search and that didn't help.

In the database I see the data being generated. It seems like the site just isn't pulling it. I didn't see anything useful in http logs either.

On Feb 6, 2018 5:22 PM, "Lorenzo Mangani" notifications@github.com wrote:

@arainero https://github.com/arainero as a starter, make sure you hit "clear" on the form to avoid ghost parameters impacting search - the statistics are generated from actual messages, so both should be on the system most definitely.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sipcapture/homer-docker/issues/59#issuecomment-363585157, or mute the thread https://github.com/notifications/unsubscribe-auth/AZZyZxiVuXnALBXmVOd52tBcKqZqL4Mvks5tSNCSgaJpZM4R3xbt .

[lmangani]

This might sound silly but... you selected a timerange, right? This is a known deficiency in new homer installations, they default to an old date and confuse some users. If data is in the db, it should be returned.

[arainero]

I usually select "last 10 minutes" and get results.

On Feb 6, 2018 5:54 PM, "Lorenzo Mangani" notifications@github.com wrote:

This might sound silly but... you selected a timerange, right? This is a known deficiency in new homer installations, they default to an old date and confuse some users. If data is in the db, it should be returned.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sipcapture/homer-docker/issues/59#issuecomment-363593354, or mute the thread https://github.com/notifications/unsubscribe-auth/AZZyZzdbA9X3EX4rYLjEs2pfb26UzeKtks5tSNgggaJpZM4R3xbt .

[lmangani]

@arainero you'll have to provide more details to understand whats happening, such as some example data you see in DB to confirm why it wouldn't be returned as a starter.

[arainero]

In the homer_data database I see that it is making new tables for every new day.

Data is populating the tables too. I read your previous comment again about the time suggestion. I modified the dates and I was able to get one row to return in the search results, but I was unable to reproduce it. It makes me believe that this may in fact be date/time related.

@lmangani, which table would you like to see data from?

``

[arainero]

Another thing I noticed and I don't know if it is related at all, but the dashboard has stopped appearing for some reason:

This is all that I can see now.

[adubovikov]

https://github.com/sipcapture/homer7-docker