homer-cron mysql-rotate wont work

[ghost]

Hi,

i suppose that run.sh script dont work properly. After run it there is no new tables in my database, and all what i see is: root@b375e8a032dd:/# /opt/homer_mysql_rotate Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 324. Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 326. Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 510. Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 510.

[lmangani]

Hi @przeqpiciel Is this a recent installation? This should have been solved a while back.

[ghost]

You know, I am tottaly new with docker. But I just be sure i got latest git of homer-docker and yesterday (02.03.2018) i just make new docker-build. So i guess i have got recent installation.

[ghost]

Hi, When I wrote 'there is no new table' I mean script doesn't create table with template [name][actualdate]. I took quick look on tcpdump on mysql side and saw that is each INSERT query trying put data to that tables ([tableName][actualDate]). So how to automaticly create table for each day ?

[hitokiri82]

I think I'm also being affected by this bug. Using multi container solution, I cant get the system to work correctly, even from a fresh start.

I've tested starting first the bootstrap service which runs without a problem, and then the cron service and I get the following error, or some variation of it all over the output:

homer-cron   | DBD::mysql::st execute failed: Table 'homer_data.sip_capture_rest' doesn't exist at /opt/homer_mysql_rotate.pl line 444.
homer-cron   | DBD::mysql::db do failed: Table 'homer_data.sip_capture_rest' doesn't exist at /opt/homer_mysql_rotate.pl line 526.

I'm attaching the full output of the container: homer-cron output.txt

If I look into the mysql server this is the state of the database:

mysql> show tables;
+-----------------------------------+
| Tables_in_homer_data              |
+-----------------------------------+
| isup_capture_all_20150407         |
| logs_capture                      |
| report_capture                    |
| rtcp_capture                      |
| rtcp_capture_all_20170216         |
| sip_capture_call_20150407         |
| sip_capture_registration_20150407 |
| sip_capture_rest_20150407         |
| webrtc_capture                    |
| webrtc_capture_all_20170116       |
+-----------------------------------+

So it seems that the cron is right to complain that the tables dont exist with the name it is looking for.

[lmangani]

Hi @hitokiri82 could you try swapping the homer-cron container with sipcapture/homer-nocron and check if it solves your issue?

[hitokiri82]

Ok. I tried homer-nocron. It seems to have worked correctly. The output of that container is:

Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 255.
Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 256.
Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 511.
Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 511.
Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 255.
Use of uninitialized value $partstep in array element at /opt/homer_mysql_rotate.pl line 256.
Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 511.
Use of uninitialized value $partstep in numeric gt (>) at /opt/homer_mysql_rotate.pl line 511.
bash: sleep_until: command not found

So how can we test if this will work correctly from now on? Because based on the names of the database tables I assume that this has to rotate the tables everyday, right?

[lmangani]

The final error is not expected - while I try to figure that out, you can check and see tomorrow if there are new tables being created.

[hitokiri82]

Those messages I pasted repeat periodically on the log of that container

[lmangani]

@hitokiri82 I'm rebuilding the homer-nocron image with the fix, please standby

[lmangani]

@hitokiri82 could you try update the homer-nocron container and see if the errors are gone? The $partstep one was fixed quite a while ago so i'm surprised to see it here. Is this a fresh setup?

[hitokiri82]

I just did a docker pull for homer-nocron and I'm not getting a new image. The one that is on docker-hub is 4 months old.

This is a fresh setup, meaning that I recreated all the containers and all the data-volumes.

[lmangani]

Are you looking at this?

Last pushed: an hour ago

[hitokiri82]

I was looking at this one

Let me change the image source and try again.

I've added it to my docker-compose like this:

  nocron:
    container_name: homer-nocron
    image: sipcapture/homer-nocron
    depends_on:
      - mysql
    volumes:
      - homer-data-semaphore:/homer-semaphore/
    links:
      - "mysql:mysql"
    env_file:
      - ./homer.env
    restart: always

Other than the source for the image, does that look ok to you?

[hitokiri82]

With this new nocron image I get this output:

/opt/homer-docker# docker-compose up nocron
mysql is up-to-date
Creating homer-nocron
Attaching to homer-nocron
homer-nocron | Sleeping for 47183 seconds

[lmangani]

That's a good sign. It should have executed the rotation, and will sleep 13 hours until the next one.

[hitokiri82]

Is the time at which the rotation runs important, or is it just a matter of it running everyday?

The way to check if this is working correctly is that tomorrow there should be a new table with tomorrow's date, right?

[lmangani]

The time of rotation is extracted from the rotation.ini and defaults to 4AM when unset. As of the rest, correct, tomorrow you should see a new counter for 24h or 86400 and new tables for the next 2 days in db. Please confirm so we can officially swap the cron image with this one for others, too.

Thanks!

[hitokiri82]

This is the state of my database today after the nocron container ran, it seems to be working correctly:

mysql> show tables;
+-----------------------------------+
| Tables_in_homer_data              |
+-----------------------------------+
| isup_capture_all_20180308         |
| isup_capture_all_20180309         |
| isup_capture_all_20180310         |
| isup_capture_all_20180311         |
| logs_capture                      |
| report_capture                    |
| rtcp_capture                      |
| rtcp_capture_all_20180308         |
| rtcp_capture_all_20180309         |
| rtcp_capture_all_20180310         |
| rtcp_capture_all_20180311         |
| sip_capture_call_20180308         |
| sip_capture_call_20180309         |
| sip_capture_call_20180310         |
| sip_capture_call_20180311         |
| sip_capture_registration_20180308 |
| sip_capture_registration_20180309 |
| sip_capture_registration_20180310 |
| sip_capture_registration_20180311 |
| sip_capture_rest_20180308         |
| sip_capture_rest_20180309         |
| sip_capture_rest_20180310         |
| sip_capture_rest_20180311         |
| webrtc_capture                    |
| webrtc_capture_all_20180308       |
| webrtc_capture_all_20180309       |
| webrtc_capture_all_20180310       |
| webrtc_capture_all_20180311       |
+-----------------------------------+

[hitokiri82]

On the other hand, for some reason, homer hasnt been able to capture any packets since yesterday at 2018-03-08 17:04:42 UTC. Do you think it might be related to the rotation thing, or should I open a new issue?

mysql> select count(*) from sip_capture_call_20180308 order by id desc;          
+----------+
| count(*) |
+----------+
|     7757 |
+----------+
1 row in set (0.00 sec)

mysql> select count(*) from sip_capture_call_20180309 order by id desc;
+----------+
| count(*) |
+----------+
|        0 |
+----------+
1 row in set (0.00 sec)

[lmangani]

Not normal. Please check the kamailio logs - if there are insert errors they should be well visible.

[hitokiri82]

The kamailio log was empty, I think it got stuck at some point. I restarted it and now it's capturing correctly. I dont think it had anything to do with the rotation stuff anyway.

[hitokiri82]

Hello lmangani.

I see that you closed this issue. Will this solution be merged back into master? (I mean the use of a nocron container for rotating the tables)

What nocron container should we use after all? The one on sipcature, or the one on qxip?