Define backend service specifications for notification system

[robyngit]

To effectively support a notification system in MetacatUI, we need a backend service to fulfill several key requirements. This issue entails defining the specifications for this backend service. The backend service will be responsible for managing user subscriptions to notifications, detecting events and triggering notifications, and either directly emailing notifications to users or integrating with an email service that handles email sending. Here is an initial list of requirements, which we can refine as this feature is discussed and developed further.

User Subscription Management: An API for managing user subscriptions to notifications.

• Subscribe/Unsubscribe: MetacatUI must be able to subscribe to or unsubscribe users from notifications related to specific datasets (seriesId or pid), portals (seriesId), or metrics events related to a dataset or a portal's collection of datasets.
• Retrieve and Set Preferences: MetacatUI will need to get or update user preferences for notification types and frequencies.
  • Options for users to select notification frequency (e.g., immediate, daily, monthly, quarterly).
  • Allow users to select types of notifications they wish to receive.
• Access Management: The service will need to manage access to private datasets and portals.
  • Users should only be able to subscribe to notifications for datasets and portals (and associated metrics) they have read access to. If a dataset or portal becomes private, users should be unsubscribed from notifications for that dataset or portal.
• Notification History: Optionally, a way to access a user's notification history, filterable by date, type, and other parameters.

Notification Triggers and Event Detection: A system for detecting events and triggering notifications.

• Detect events such as dataset and portal document changes (a dataset is obsoleted by a new version), dataset usage (downloads, views, citations), and creation of derived products.
• Trigger notifications based on these events.

Email Notification System: (this could be an external service that our backend service integrates with)

• Integration with an email service for sending notifications.
• Template management for email content, customizable for different types of notifications.
• Handle email sending in batches for efficiency, especially for summary notifications.

Other Considerations:

• The system should be scalable to handle a large number of users and notifications.
• It would be nice to handle real-time notifications efficiently.
• The service may need to be run at the CN level, rather than the MN level, since datasets can be replicated across multiple MNs, and may be viewed, downloaded, etc. from any MN. A user's account also spans across a CN, and so too should their notification preferences.
• The system needs to be implemented in such a way that any sensitive data is filtered out. If we were to use our existing logs to help with creating/tracking these events, for example, there could be some info that we likely don't want to expose like IP addresses.
• ...?

[robyngit]

@vchendrix if ESS-Dive has requirements other than those listed here, it would be great to know about those!