Feature request - to add ability for status updates to the mail

[pqvindesland]

Hi

It would be really handy to have in the emails the recent status updates and who logged the updates? maybe just the most recent status?

Regards Per

[moudsen]

Should be possible. I'm currently investigating which API call returns the requested information.

[moudsen]

Found the API for the alerting parts - building proof-of-concept in my development environment to see what Zabbix actually returns (aiming to get the "Alerts" information as depicted on the problems screen). The result will be an array that can be parsed in TWIG similar to how graphs are currently parsed as well.

[moudsen]

Concluding on issue #41 first and will start testing what information is available. Just to be sure what you are actually looking for: the associated list of actions as for the top right screen of the Problem overview, right?

[pqvindesland]

Yep many thanks that's correct it will show the the messages (if any) that was done by the engineer and the engineers name

[moudsen]

Do you only care about the acknowledgement messages or the full list of actions as per the screendump? (including the sending of messages).

[pqvindesland]

I think it would be nice to have the message that the engineer wrote so managers and other techs knows what the engineer has/is done, but only the last one a long list when the engineer has done several things might make the email a little long to read :)

[moudsen]

I am building a development v2.16 to see what comes out the API first. Probably will pick up all the registered messages as in TWIG one can decide to show none, 1, or all. In the end you can decide yourself which ones to show or not :-).

[moudsen]

MVP done. New values extracted from Zabbix for consumption in TWIG template (I've added some additional fields starting with _ for convenience in processing)

    "ACKNOWLEDGES": [
        {
            "acknowledgeid": 286,
            "userid": 1,
            "eventid": 864017,
            "clock": 1692202199,
            "message": "Test with severity change",
            "action": 28,
            "old_severity": 3,
            "new_severity": 1,
            "username": "moudsen",
            "name": "Zabbix",
            "surname": "Administrator",
            "_clock": "2023-08-16 18:09:59",
            "_actions": "Add message, Change severity, Unacknowledge event",
            "_old_severity": "Average",
            "_new_severity": "Information"
        },
        {
            "acknowledgeid": 285,
            "userid": 1,
            "eventid": 864017,
            "clock": 1692200598,
            "message": "Second text added for testing",
            "action": 4,
            "old_severity": 0,
            "new_severity": 0,
            "username": "moudsen",
            "name": "Zabbix",
            "surname": "Administrator",
            "_clock": "2023-08-16 17:43:18",
            "_actions": "Add message",
            "_old_severity": "Not classified",
            "_new_severity": "Not classified"
        },
        {
            "acknowledgeid": 284,
            "userid": 1,
            "eventid": 864017,
            "clock": 1692190741,
            "message": "Test message while acking this problem event",
            "action": 6,
            "old_severity": 0,
            "new_severity": 0,
            "username": "moudsen",
            "name": "Zabbix",
            "surname": "Administrator",
            "_clock": "2023-08-16 14:59:01",
            "_actions": "Acknowledge event, Add message",
            "_old_severity": "Not classified",
            "_new_severity": "Not classified"
        }
    ],

TWIG example

{% if ACKNOWLEDGES|length > 0 %}
                <br/>
{% for anAck in ACKNOWLEDGES %}
                <table style="border:0; text-align:left;" cellpadding="5" cellspacing="0">
                    <tr>
                        <td class="acknowledge">
                            <b>{{ anAck._clock }}</b><br/><em>({{ anAck.username }}, {{ anAck.name }} {{ anAck.surname }})</em><br/>[{{ anAck._actions }}]<br/>{{ anAck.message }}
                        </td>
                    </tr>
                </table>
{% endfor %}
{% endif %}

[moudsen]

Pending verification before publication of this release in order for you to test :-).

[moudsen]

I've concentrated on "ACKNOWLEDGE" data only, not the "ALERT" data as the Actions window in the GUI seems to combine both in the overview. This will keep the list nice/clean/short. Let me know if this works for you - will post a screenshot shortly how the result looks like with the above template.

[moudsen]

[pqvindesland]

That looks really good many thanks, I don't have time to test tonight but I will do some testing tomorrow.

[moudsen]

I expect to publish v2.16 later today. I do need some work on documentation as well. LOL.

[moudsen]

v2.16 available. Documentation updates pending but the example provisioned above should get you going into the right direction. If not, let me know and I will try to assist. Let me know if this works for you. If it does I will create the release package for v2.16.

[pqvindesland]

So that seems to have broken mailgraph I get the image path is not accessible, but that's resolved with setting correct chown permissions:

but php mailGraph.php test gives the following output: root@driftzabbix:/usr/share/zabbix# php mailGraph.php test 2023-08-16 21:59:29 : <<< mailGraph v2.16 >>> 2023-08-16 21:59:29 : # Invoked from CLI 2023-08-16 21:59:29 : # Data passed to MailGraph main routine and used for processing { "eventId": 0, "duration": 0, "recipient": "per.qvindesland@fremtind.no", "baseURL": "https:\/\/driftzabbix.intern.fremtind.no\/", "subject": "[TEST] {{ HOST_NAME|raw }}: ({{ EVENT_SEVERITY }}) {{ EVENT_NAME|raw }}", "period": "30m", "period_header": "Last 30 minutes", "debug": 1, "HTTPProxy": "", "itemId": 0 } 2023-08-16 21:59:29 : # LOGIN to Zabbix 2023-08-16 21:59:29 : % postJSON: https://driftzabbix.intern.fremtind.no/api_jsonrpc.php 2023-08-16 21:59:29 : > POST data: {"jsonrpc":"2.0","method":"user.login","params":{"username":"","password":""},"id":1,"auth":null} 2023-08-16 21:59:29 : > Received 68 bytes 2023-08-16 21:59:29 : > Token = ece863b56bf9356c811e7d5c80ad8aeb 2023-08-16 21:59:29 : # Record Zabbix API version 2023-08-16 21:59:29 : % postJSON: https://driftzabbix.intern.fremtind.no/api_jsonrpc.php 2023-08-16 21:59:29 : > POST data: {"jsonrpc":"2.0","method":"apiinfo.version","params":[],"id":2} 2023-08-16 21:59:29 : > Received 42 bytes 2023-08-16 21:59:29 : > API version 6.0.20 2023-08-16 21:59:29 : # No event ID given, picking up random event from Zabbix 2023-08-16 21:59:29 : % postJSON: https://driftzabbix.intern.fremtind.no/api_jsonrpc.php 2023-08-16 21:59:29 : > POST data: {"jsonrpc":"2.0","method":"problem.get","params":{"output":"extend","recent":"true","limit":1},"auth":"ece863b56bf9356c811e7d5c80ad8aeb","id":3} 2023-08-16 21:59:29 : > Received 36 bytes 2023-08-16 21:59:29 : > Problem data (recent) { "jsonrpc": 2, "result": [], "id": 3 } 2023-08-16 21:59:29 : - No response data received. Retrying with less recent problems ... 2023-08-16 21:59:29 : % postJSON: https://driftzabbix.intern.fremtind.no/api_jsonrpc.php 2023-08-16 21:59:29 : > POST data: {"jsonrpc":"2.0","method":"problem.get","params":{"output":"extend","recent":"false","limit":1},"auth":"ece863b56bf9356c811e7d5c80ad8aeb","id":4} 2023-08-16 21:59:30 : > Received 36 bytes 2023-08-16 21:59:30 : > Problem data (not recent) { "jsonrpc": 2, "result": [], "id": 4 } 2023-08-16 21:59:30 : ! Cannot continue: mailGraph is unable to pick a random event via the Zabbix API. It is highly likely that no active problems exist? Please retry or determine and set an event ID manually and retry.

Tailing the /var/log/zabbix/zabbix_server.log it reports: WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version. 939:20230816:235607.172 MailGraph notification failed: [MailGraph Webhook] No data received from mailGraph - please check recipient address or mailGraph log and retry. WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version. 940:20230816:235635.277 MailGraph notification failed: [MailGraph Webhook] Please define recipient for the test message! WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version. WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version. WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version. WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version. WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version. WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version.

I haven't touched the config/config.json file so it's the same as before the update with "cli_eventId": 0, and "cli_duration": 0, set.

Any clue?

[moudsen]

Looking into this by firing up the previously setup environment for testing. I'll upgrade to your Zabbix version and will install/test the new code there.

I've not changed any logic or procedures with regards to the automatic detection of events available in the system. Is this a clean environment by the way or the same one we've used before?

The only thing I did change was adding a new flag to the event.get procedure (besides picking up the ACK messages and pushing those into the TWIG output).

The deprecation messages are definitely not from me as I'm not using mysql to connect to Zabbix at all? Checking my own environment to see if the same occurs with me with this Zabbix version.

[moudsen]

Verified 6.0.19 production at my end: I do have deprecation messages but not on MYSQL.

("[Thu Aug 17 11:59:45.921753 2023] [proxy_fcgi:error] [pid 231914:tid 140590025000704] [xxx:47108] AH01071: Got error 'PHP message: PHP Notice: Parameter "/user" is deprecated. in /usr/share/zabbix/include/classes/validators/CApiInputValidator.php on line 1330'")

This is an internal Zabbix matter (irritating only).

Upgrading to 6.0.20 later today to see whether that version does show other messages and if functionality changes there that may cause mailGraph (or other) issues.

[moudsen]

Upgraded your test previous test environment to 6.0.20 and no issues found (tested mailGraph v2.14, v2.15 and v2.15).

Are you using a clean environment for testing mailGraph as it needs at least 1 event to start processing anyway? Or could be a rights issue when it actually is a new environment?

I did catch a code typo for detection of "acknowledges' from the event.get call in v2.16 - will post that change in a few minutes however that is not related to the issue at hand from your end.

Can you please locate an event in your Zabbix environment and specify event_id manually to verify if something else is happening?

Please replace mailGraph.php with the newer version I'm posting in the next minute first otherwise you will hit an error. Apologies for not catching that one earlier.

[moudsen]

Just checking as this feels MariaDB or PHP related:

PHP 8.2.9 (cli) (built: Aug 3 2023 11:39:08) (NTS gcc x86_64) Copyright (c) The PHP Group Zend Engine v4.2.9, Copyright (c) Zend Technologies

Server version: 10.6.15-MariaDB MariaDB Server

mailGraph stops when it does not have a valid Event ID hence the message "no data received". I could have a look to make that more sensible.

Awaiting your feedback.

[moudsen]

I've upgraded my main systems with v2.16 and all is working fine - checking on deprecation messages in the next days. Not understanding why your installation is broken? Happy to assist where I can!

[pqvindesland]

Yeah it's an odd one, not shure why it's failing, if I specify an eventid it picks up a lot of events and stops with 2023-08-18 09:25:25 : # Retrieve HOST information 2023-08-18 09:25:25 : % postJSON: https://driftzabbix.intern.fremtind.no/zabbix/api_jsonrpc.php 2023-08-18 09:25:25 : > POST data: {"jsonrpc":"2.0","method":"host.get","params":{"hostids":"10047","output":"extend","selectTags":"extend"},"auth":"23cf770da0d507aecb5f0d400a02ba84","id":6} 2023-08-18 09:25:25 : > Received 36 bytes 2023-08-18 09:25:25 : > Host data { "jsonrpc": 2, "result": [], "id": 6 } ! No response data received?

I did notice that it says when php mailGraph.php test is run that it's missing a couple of array key's: 2023-08-18 08:40:04 : !! (2) "Undefined array key "EVENT_VALUE"" at line #865 of "/usr/share/zabbix/mailGraph.php"

2023-08-18 08:40:04 : # Retrieve TRIGGER information

2023-08-18 08:40:04 : !! (2) "Undefined variable $p_triggerId" at line #900 of "/usr/share/zabbix/mailGraph.php"

But I don't know if they are valid errors.

do you mind sanity check the config?

root@driftzabbix:/usr/share/zabbix# cat config/config.json { "script_baseurl": "https:\/\/driftzabbix.intern.fremtind.no\/", "cli_eventId": 3993, "cli_duration": 0, "cli_recipient": "per.qvindesland@fremtind.no", "cli_subject": "[TEST] {{ HOST_NAME|raw }}: ({{ EVENT_SEVERITY }}) {{ EVENT_NAME|raw }}", "cli_baseURL": "https:\/\/driftzabbix.intern.fremtind.no\/zabbix\/", "cli_period": "30m", "cli_period_header": "Last 30 minutes", "cli_debug": 1, "cli_proxy": "", "zabbix_user": "api-user", "zabbix_user_pwd": "Password123", "zabbix_api_user": "api-user", "zabbix_api_pwd": "Password123", "subject": "{{ HOST_NAME|raw }}: ({{ EVENT_SEVERITY }}) {{ EVENT_NAME|raw }}", "smtp_server": "intsmtpgw.sparebank1.no", "smtp_port": 25, "smtp_transport": "none", "smtp_from_address": "driftzabbix@fremtind.no", "smtp_from_name": "DriftZabbix", "smtp_reply_address": "driftzabbix@fremtind.no", "smtp_from_name": "DriftZabbix monitorering", "smtp_strict": "no", "graph_match": "any", "period": "20m", "period_header": "Last 20 minutes", "retention_images": 7, "retention_logs": 14, "debug": 1 }

When I run a test from the web interface that it fails with problems accessing the cookie temporary path: 00:00:00.000 [Debug] [MailGraph Webhook] Sending request: https://driftzabbix.intern.fremtind.no/mailGraph.php?{"itemId":null,"eventId":0,"recipient":"per.qvindesland@fremtind.no","baseURL":"https://driftzabbix.intern.fremtind.no/zabbix/","duration":0,"graphWidth":"300","graphHeight":"120","subject":"{{ HOST_NAME|raw }}: ({{ EVENT_SEVERITY }}) {{ EVENT_NAME|raw }}","showLegend":"0","periods":"10m,4h,1d,7d","periods_headers":"Last 10 minutes,Last 4 hours,Last day,Last 7 days","infoTest":"Test"} 00:00:00.007 [Debug] [Mailgraph Webhook] Received response:Cookies temporary path inaccessible? 00:00:00.007 [Unknown] MailGraph notification failed: [MailGraph Webhook] An error has occurred during processing: Cookies temporary path inaccessible?

I can send you the full output of php mailGraph.php test

BUt it's really odd that it should suddenly fail.

[moudsen]

Can you send me your database once more via email? I can load it up in the existing instance of last time for you. In that database the new stuff actually works just fine so curiousity level is rising at my end.

[pqvindesland]

Yep sure I sent you the information on Linkedin

[moudsen]

I've done the following on the test box:

• Put your database in
• Changed admin password
• Verified login to Zabbix
• Changed api-user password
• Upgraded mailGraph to v2.16
• Set config.json for my environment
• Tested with eventId = 0 and with eventId = 399
• All working just fine and produced expected output as well. Validating logs of Apache, MariaDB and Zabbix-Server as a next step. Still not sure what is happening at your end ... grrrrr.

[moudsen]

Zero deprecation, errors, warning or anything else in logs of Zabbix, messages, HTTPD. I seem unable to replicate your issue ...

[moudsen]

Shared example output with you btw direct e-mail for a set eventId but when defining to zero, the code does pick up a random eventId.

[pqvindesland]

Many thanks for assiting testing this.

I had to reinstall Zabbix and attach the production DB once I felt happy that things where working, then reinstall mailGraph, then specify an eventid, I really wish that i knew what caused it (it might have saved me from working yesterday) but it works now, so I can't really complain anymore :)

Regards Per

[moudsen]

Glad to hear you got is sorted! I can feel the frustration all the way of not having the root cause ... :-). Does it now also work again without a specific eventId? Last but not least: are you happy with the new feature and can we proceed to close the request as such?

[pqvindesland]

Yep it works also without specifying any eventid but if I do php mailGraph.php test then I need to specify a eventid, but it works so I am happy.

Yep feel free to close it, the new feature works really well, I show cased it for the incident management team today which is really happy about the feature, so everyone is happy here, at least until after lunch :)