ControlSystemStudio / phoebus

A framework and set of tools to monitor and operate large scale control systems, such as the ones in the accelerator community.
http://phoebus.org/
Eclipse Public License 1.0
92 stars 89 forks source link

Alarm Server - Setting automated mail sender #2693

Closed UrbanBobek closed 1 year ago

UrbanBobek commented 1 year ago

Hi all,

How does one set the automated mail sender?

I am aware of the org.phoebus.applications.alarm/automated_email_sender setting, but I am not sure where to apply it. Is this a Phoebus setting or an alarm server setting?

I am running an alarm server instance and a (fake) SMTP server. When starting the alarm server, I provide a settings.ini file, which contains information about the SMTP server and, just to be sure, the automated_email_sender setting. I also included that setting in the Phoebus settings.ini file. I have a PV configured to send an email when an alarm occurs and I am able to receive the email on my SMTP server, but no matter where or how I set the automated_email_sender setting, it comes from the default alarm_server@example.com.

Any help would be greatly appreciated! Urban

kasemir commented 1 year ago

The alarm server is sending the emails in this case, so the settings.ini passed to the alarm server matter.

This is the code sending the mail: https://github.com/ControlSystemStudio/phoebus/blob/59c37b11cb767dd1e6788d26af00a1f06725e892/services/alarm-server/src/main/java/org/phoebus/applications/alarm/server/actions/EmailActionExecutor.java#L49

So org.phoebus.applications.alarm/automated_email_sender=fred@whatever.com should be the way to configure it. The SMTP host is set via org.phoebus.email/mailhost, and that one can also be used by the phoebus GUI to email screenshots from the context menu. The default is https://github.com/ControlSystemStudio/phoebus/blob/59c37b11cb767dd1e6788d26af00a1f06725e892/core/email/src/main/resources/email_preferences.properties#L7

I would check if you're indeed setting the expected values. In the alarm server you can't really see that at runtime, but you can run the phoebus GUI with the same -settings ... parameters and then check in Help, About, ... what you really have set for "mailhost" and "automated_email_sender".

shroffk commented 1 year ago

I think I can kind off reproduce this issue and feel it might be an issue with the order in which the preferences are read.

I have a setting.ini file with the from defined as org.phoebus.applications.alarm/automated_email_sender=tst_alarm_server@bnl.gov

When I change the sender and immediately restart the service, I still get email with the old address. I deleted the ~/.phoebus for the user running the alarm server and restarted the server. The first few emails I got reverted to the phoebus default alarm_server@example.com even though the preference had been correctly read

[epics-services-tst@epics-services-tst .phoebus]$ cat .userPrefs/org/phoebus/applications/alarm/prefs.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE map SYSTEM "http://java.sun.com/dtd/preferences.dtd">
<map MAP_XML_VERSION="1.0">
  <entry key="automated_email_sender" value="tst_alarm@bnl.gov"/>
  <entry key="config_name" value="NSLS2_OPR"/>
  <entry key="config_names" value="NSLS2_OPR"/>
  <entry key="server" value="localhost:43194"/>
</map>

I just did a second restart ( after checking the .phoebus reflected the correct settings ) and viola I started getting emails from the correct new address

image

kasemir commented 1 year ago

Your example suggests that the effective preferences are loaded in https://github.com/ControlSystemStudio/phoebus/blob/59c37b11cb767dd1e6788d26af00a1f06725e892/app/alarm/model/src/main/java/org/phoebus/applications/alarm/AlarmSystem.java#L126 before the command line -settings... are applied in https://github.com/ControlSystemStudio/phoebus/blob/59c37b11cb767dd1e6788d26af00a1f06725e892/services/alarm-server/src/main/java/org/phoebus/applications/alarm/server/AlarmServerMain.java#L637

So in the end the .phoebus/.. settings are written as expected, but they're already used ahead of that time.

But when I set breakpoints in those two places, the -settings.. are applied as designed before they are used.

This is for simply running the AlarmServerMain with a single -settings /path/to/settings.ini command line. Maybe it's different with other arguments?

shroffk commented 1 year ago

While I do use a wrapper script called via procserv, I am running the alarm server with on the -settings argument.

Are you able to recreate the issue? changing the from preference and restarting the service was enough for me to recreate it. I will do a bit more debugging once I am back at work

georgweiss commented 1 year ago

@UrbanBobek, at ESS we have crafted a solution where notifications are pushed to mobile clients (IOS and Android) registered with a generic service. Notifications are not limited to alarms even though that is a key use case.

Users at ESS requesting email-based notifications (for whatever use case) are referred to this solution (dubbed "ESS Notify").

In case you're would be interested I can point you to the (public) repos. There is of course a dependency to both Apple and Google push notification services, so this may be a "political" blocker.

UrbanBobek commented 1 year ago

Thanks for the suggestions! Here is what I found out so far. I checked the settings inside Phoebus and they were set correctly. Also, in the .phoebus directory of the machine where I am running the alarm server on, I found the setting was set correctly:

$ grep -rn automated
.phoebus/.userPrefs/org/phoebus/applications/alarm/prefs.xml:4:  <entry key="automated_email_sender" value="fred@whatever.com"/>

Previously I was running the alarm server using the following command:

./alarm-server.sh -noshell -config $CONFIG_NAME -server $ALARM_HOST:9092 -logging $CONFIG_LOCATION/alarm_server_logging.properties -settings $CONFIG_LOCATION/settings.ini 

When I tried running it without the -noshell argument however, I started receiving emails from fred@whatever address. image

Afterwards, even if I used the -noshell argument, I received emails from the correct address.

UrbanBobek commented 1 year ago

@georgweiss Is this the repo you are referring to? https://gitlab.esss.lu.se/ics-software/ess-notify-server

For now we only require mail notifications, but thank you for pointing out this option!

georgweiss commented 1 year ago

Yes.

We have separate repos for the IOS and Android apps.

kasemir commented 1 year ago

The startup is a little iffy because it allows both command line arguments like -server and -config, while also allowing to set them from preferences.

The preferences are statically initialized as soon as you access the AlarmSystem class, https://github.com/ControlSystemStudio/phoebus/blob/59c37b11cb767dd1e6788d26af00a1f06725e892/app/alarm/model/src/main/java/org/phoebus/applications/alarm/AlarmSystem.java#L126

That is meant to happen after a -settings argument has been handled in https://github.com/ControlSystemStudio/phoebus/blob/59c37b11cb767dd1e6788d26af00a1f06725e892/services/alarm-server/src/main/java/org/phoebus/applications/alarm/server/AlarmServerMain.java#L637

Commit 260f1c9067c0e6e063772313f540aae46adf3207 helped to ensure this, and when I set breakpoints, I find -settings indeed being read before the static AlarmSystem init, but that would be something to check if you can reproduce the issue.

UrbanBobek commented 1 year ago

I apologize for my late response.

Thank you for the suggestions. We upgraded to the latest version of the alarm server and we modified the startup command so that we do not use the -server and -config options. Now we get the emails from the specified sender address.