support callback request subscribers and job notification refactoring

[fmigneault]

Changes

• Add Job subscribers support to define OGC-compliant callback URLs where HTTP(S) requests will be sent upon reaching certain Job status milestones (fixes #230).
• Add email notification support to the new subscribers definition (extension over OGC minimal requirements).
• Deprecate Job notification_email in the OpenAPI specification in favor of subscribers, but preserve parsing of its value if provided in the JSON body during Job submission for backward compatibility support of existing servers. The Job.notification_email attribute is removed to avoid duplicate references.
• Add notification email for Job started status, only available through the subscribers property.
• Add {PROCESS_ID}/{STATUS}.mako template detection under the weaver.wps_email_notify_template_dir location to allow per-Process and per-Job status email customization.
• Refactor weaver/notify.py and weaver/processes/execution.py to avoid mixed references to the encryption/decryption logic employed for notification emails. All notifications including emails and callback requests are now completely handled and contained in the weaver/notify.py module.
• Remove partially duplicate Mako Template definition as hardcoded string and separate file for email notification.

References

• Relates to https://github.com/opengeospatial/ogcapi-processes/issues/366
• Relates to #571
• Relates to #568
• Fixes #230

[codecov[bot]]

Codecov Report

Merging #572 (fe55014) into master (1572fd0) will increase coverage by 0.16%. The diff coverage is 95.48%.

@@            Coverage Diff             @@
##           master     #572      +/-   ##
==========================================
+ Coverage   84.71%   84.87%   +0.16%     
==========================================
  Files          79       79              
  Lines       17828    17910      +82     
  Branches     2708     2716       +8     
==========================================
+ Hits        15103    15202      +99     
+ Misses       1992     1980      -12     
+ Partials      733      728       -5     

Files	Coverage Δ
weaver/notify.py	92.19% <100.00%> (+11.55%)	:arrow_up:
weaver/store/base.py	100.00% <ø> (ø)
weaver/store/mongodb.py	78.31% <ø> (ø)
weaver/wps_restapi/api.py	78.97% <ø> (ø)
weaver/wps_restapi/jobs/jobs.py	89.63% <100.00%> (ø)
weaver/wps_restapi/jobs/utils.py	77.94% <ø> (ø)
weaver/wps_restapi/swagger_definitions.py	99.82% <100.00%> (+<0.01%)	:arrow_up:
weaver/processes/execution.py	86.28% <80.00%> (+1.95%)	:arrow_up:
weaver/cli.py	86.30% <94.28%> (+0.39%)	:arrow_up:
weaver/datatype.py	79.08% <62.50%> (ø)

[fmigneault]

LGTM, if I understood this correctly, multiple subscribers are only supported for different job status.

Yes

Do you think it could be interesting to allow multiple emails notification for each status ?

Could be, but I think we can work around it with a mailing list for the moment without changing the code. If an actual use case to support multiple emails arises, it could be added later on.