improve notification system

[mikeAdamss]

What is this

Currently the notifications are quite verbose. and no longer necessary as we can provide a link back to the glue job in question.

This task is to simplify the notification system with that in mind.

What to do

1. Simplify notifications.

For each recipient webhook (DE, SE etc) there are only two possible messages, "PASS" and "FAIL" with no variable information. Change the notification logic to accomodate this.

Rewrite the following files:

• https://github.com/ONSdigital/dp-data-pipelines/blob/sandbox/dpypelines/s3_tar_received.py
• https://github.com/ONSdigital/dp-data-pipelines/blob/sandbox/dpypelines/pipeline/dataset_ingress_v1.py

To use the pass/fail notification, and use use the message function to populate an error rather than a notification, example follows:

    # Before
    try:
        decompress_s3_tar(s3_object_name, "input")
    except Exception as err:
        notify.data_engineering(
            message.unexpected_error(
                f"Failed to decompress tar file {s3_object_name}", err
            )
        )
        raise err

    # After
    try:
        decompress_s3_tar(s3_object_name, "input")
    except Exception as err:
        notify.data_engineering.failure() # Should post "FAILURE" to the DE webhook
        raise Exception(
            message.unexpected_error(
                f"Failed to decompress tar file {s3_object_name}", err
            )
        ) from err

You might need to rewrite a few tests, unsure, as needed to accomplish the task.

Acceptance Criteria

• [ ] Pipelines run without spammy notifications.

[mikeAdamss]

done