search

Icinga / icingaweb2-module-reporting

Icinga Reporting is the central component for reporting related functionality in the monitoring web frontend and framework Icinga Web 2.
GNU General Public License v2.0
64 stars 24 forks source link

Unable to get scheduled reports #242

Open Prathameshhankare opened 3 months ago

Prathameshhankare commented 3 months ago

Describe the bug

I am unable to get scheduled report on email whereas I get the email if I send the email manually(From GUI).

Error in Service:

[root@XXXXX01 ~]# systemctl status icinga-reporting.service -l ● icinga-reporting.service - Icinga Reporting Scheduler Loaded: loaded (/etc/systemd/system/icinga-reporting.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Sun 2024-07-21 13:45:03 UTC; 10min ago Process: 27428 ExecStart=/usr/bin/icingacli reporting schedule run (code=exited, status=255) Main PID: 27428 (code=exited, status=255)

Jul 21 13:39:22 XXXXX01 systemd[1]: Started Icinga Reporting Scheduler. Jul 21 13:45:02 XXXXX01 icingacli[27428]: PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 8192 bytes) in /usr/share/icingaweb2/modules/pdfexport/vendor/textalk/websocket/lib/Base.php on line 88 Jul 21 13:45:02 XXXXX01 icingacli[27428]: Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 8192 bytes) in /usr/share/icingaweb2/modules/pdfexport/vendor/textalk/websocket/lib/Base.php on line 88 Jul 21 13:45:02 XXXXX01 systemd[1]: icinga-reporting.service: main process exited, code=exited, status=255/n/a Jul 21 13:45:03 XXXXX01 systemd[1]: Unit icinga-reporting.service entered failed state. Jul 21 13:45:03 XXXXX01 systemd[1]: icinga-reporting.service failed.

Mail Logs:

Jul 21 13:55:01 XXXXX01 postfix/pickup[25322]: 4B1E5453B3A: uid=0 from= Jul 21 13:55:01 XXXXX01 postfix/cleanup[30928]: 4B1E5453B3A: message-id=20240721135501.4B1E5453B3A@XXXXX01.wetry.com Jul 21 13:55:01 XXXXX01 postfix/qmgr[1347]: 4B1E5453B3A: from=root@XXXXX01.wetry.com, size=768, nrcpt=1 (queue active) Jul 21 13:55:01 XXXXX01 postfix/pickup[25322]: 4FD0A453B41: uid=0 from= Jul 21 13:55:01 XXXXX01 postfix/cleanup[30928]: 4FD0A453B41: message-id=20240721135501.4FD0A453B41@XXXXX01.wetry.com Jul 21 13:55:01 XXXXX01 postfix/qmgr[1347]: 4FD0A453B41: from=root@XXXXX01.wetry.com, size=771, nrcpt=1 (queue active) Jul 21 13:55:01 XXXXX01 postfix/local[30942]: 4B1E5453B3A: to=root@XXXXX01.wetry.com, orig_to=, relay=local, delay=0.13, delays=0.09/0.02/0/0.02, dsn=2.0.0, status=sent (delivered to mailbox) Jul 21 13:55:01 XXXXX01 postfix/qmgr[1347]: 4B1E5453B3A: removed Jul 21 13:55:01 XXXXX01 postfix/local[30944]: 4FD0A453B41: to=root@XXXXX01.wetry.com, orig_to=, relay=local, delay=0.17, delays=0.09/0.04/0/0.03, dsn=2.0.0, status=sent (delivered to mailbox) Jul 21 13:55:01 XXXXX01 postfix/qmgr[1347]: 4FD0A453B41: removed Jul 21 14:00:01 XXXXX01 postfix/pickup[25322]: D3DF1453B40: uid=0 from= Jul 21 14:00:01 XXXXX01 postfix/cleanup[31941]: D3DF1453B40: message-id=20240721140001.D3DF1453B40@XXXXX01.wetry.com Jul 21 14:00:01 XXXXX01 postfix/qmgr[1347]: D3DF1453B40: from=root@XXXXX01.wetry.com, size=771, nrcpt=1 (queue active) Jul 21 14:00:01 XXXXX01 postfix/pickup[25322]: D9CF7453B41: uid=0 from= Jul 21 14:00:01 XXXXX01 postfix/cleanup[31941]: D9CF7453B41: message-id=20240721140001.D9CF7453B41@XXXXX01.wetry.com Jul 21 14:00:01 XXXXX01 postfix/qmgr[1347]: D9CF7453B41: from=root@XXXXX01.wetry.com, size=768, nrcpt=1 (queue active) Jul 21 14:00:01 XXXXX01 postfix/local[31954]: D3DF1453B40: to=root@XXXXX01.wetry.com, orig_to=, relay=local, delay=0.09, delays=0.07/0.02/0/0, dsn=2.0.0, status=sent (delivered to mailbox) Jul 21 14:00:01 XXXXX01 postfix/qmgr[1347]: D3DF1453B40: removed Jul 21 14:00:01 XXXXX01 postfix/local[31954]: D9CF7453B41: to=root@XXXXX01.wetry.com, orig_to=, relay=local, delay=0.09, delays=0.06/0.02/0/0.01, dsn=2.0.0, status=sent (delivered to mailbox) Jul 21 14:00:01 XXXXX01 postfix/qmgr[1347]: D9CF7453B41: removed

Your Environment

gurhanocaktan commented 3 months ago

I have the same issue

sukhwinder33445 commented 3 months ago

Hi @Prathameshhankare @gurhanocaktan, Can you please share the debug log (icingacli reporting schedule run --debug). Do you have a cover page with image/logo? What size are these images? How many objects are included in the report you want to send?

gurhanocaktan commented 3 months ago

Hi @sukhwinder33445

icingacli reporting schedule run --debug This only returns me Scheduling job Schedule1 to run at 2024-07-26 06:00:00

There are 242 services I have. I wanted to send SLA of all these objects. I have a logo in cover page. The size of the image must be 350 x 154 .

When I click send and manually trigger , I receive the email and report .

sukhwinder33445 commented 3 months ago

Hi @gurhanocaktan,

Yes, this schedule will be executed tomorrow.

You can create a new schedule for tests with a minutely frequency. Then run the same command.

gurhanocaktan commented 3 months ago

Hi @sukhwinder33445 , When I made it minutely , I received the report succesfully 👀 .

sukhwinder33445 commented 3 months ago

Hi @gurhanocaktan,

Maybe this was just a one-time error, this can happen if the data and the cover sheet are very large. I am closing this issue for now. If it occurs again, please reopen the issue and share your debug log.

gurhanocaktan commented 3 months ago

Hi @sukhwinder33445 ,

I change the schedule again to daily . I will check if I receive or not tomorrow . I do not think it was one-time error because I created new reports and make daily . I have never received them . Maybe bug is related to daily setting . idk.

gurhanocaktan commented 3 months ago

Yes I did not get the daily one again. :/

sukhwinder33445 commented 3 months ago

Can you please provide the debug log from today. Please make sure you have the latest version 1.0.2 of this module and 0.11.1 (latest) of the pdfexport module.

gurhanocaktan commented 3 months ago

Hi @sukhwinder33445 ,

I am using the latest version of reporting module but ı realized I am using 0.11.0 module of pdfexport . I will upgrade it . Though I do not think this is the problem . When I make the occurancy minutely I get the emails and report.

How can I see the debug log from today ?

sukhwinder33445 commented 3 months ago

The logs of the reporting module should be included in your icingaweb2 logs.

sukhwinder33445 commented 3 months ago

Hi @gurhanocaktan,

Do you still have the issue even after updating pdfexport and the reporting module to the latest version?

sukhwinder33445 commented 3 months ago

Hi @gurhanocaktan,

If you cannot find the logs of the report module, please try the following:

  1. Edit the file /etc/systemd/system/icinga-reporting.service.
    • Add the --debug flag : ExecStart=/usr/bin/icingacli reporting schedule run --debug
  2. Reload the units: systemctl daemon-reload
  3. Restart the service: systemctl restart icinga-reporting.service
  4. Make sure that the service is running with the debug flag: systemctl status icinga-reporting.service
  5. Now you should see the logs:journalctl -u icinga-reporting.service --no-pager
gurhanocaktan commented 3 months ago

Hi @sukhwinder33445 ,

I still have the issue even tho I have the latest version of these modules ( I already had the latest versions ) .

I scheduled the daily report 5 minutes later and sometimes it worked. However, the day after it did not send again . I am thinking it might be about permission issues maybe .

I will try to get these logs and send you

gurhanocaktan commented 3 months ago

Btw I realized something odd ,

When I schedule the report for at 1.20pm , This is what debug returns :

icingacli reporting schedule run --debug Scheduling job Schedule1 to run at 2024-08-01 11:20:00

There is 2 hours difference.

Prathameshhankare commented 3 months ago

@sukhwinder33445, please take a look at the below debug log you requested. debuglog.txt

Prathameshhankare commented 2 months ago

@sukhwinder33445, please take a look at the below debug log you requested. debuglog.txt

Any update on the shared debug logs?

sukhwinder33445 commented 2 months ago

Hi @Prathameshhankare,

Unfortunately, the shared log file does not contain a service crash log, only normal logs, which are harmless.

But I was able to reproduce it once. I am looking for the cause.

Prathameshhankare commented 2 months ago

Hi @sukhwinder33445,

Have we identified the cause? Please let me know if we have a solution for this issue.

Prathameshhankare commented 1 month ago

Hi @sukhwinder33445,

Could you please provide an update on the progress of identifying the root cause? I’m awaiting your response.

sukhwinder33445 commented 1 month ago

Hi @Prathameshhankare,

I have successfully reproduced the crash of the daemon and acknowledged the bug. Unfortunately, I cannot give you a date when this bug will be fixed.

As a workaround, you can omit the template for reports, which leads to a lower memory usage. Please let me know if this works for you.

Prathameshhankare commented 2 weeks ago

@sukhwinder33445, I tried the workaround, omitted the template for reports, but still no luck.

sukhwinder33445 commented 2 weeks ago

Hi @Prathameshhankare,

Is it a large file (how many objects)? How many schedules do you have, and when are they triggered (frequency, time)? You can try to reduce the report size using the filter (still without template).