GA4 Migration: Create GET endpoint to accept open email tracking data

[MackHalliday]

User Story - Business Need

We need to create a GET endpoint to accept GET requests (pixel tracking) embedded in opened emails. This endpoint should extract the different values from the GET request. In a later user story, we'll use the extracted data and pass the data along to GA4. This user story focuses exclusively on creating the GET endpoint.

• [x] Ticket is understood, and QA has been contacted (if the ticket has a QA label).

User Story(ies)

As a VA Notify Developer I want to create a GET endpoint that extracts information from the GET request embedded in an opened email So that we can capture the necessary data for tracking email opens.

Additional Info and Resources

• Implementation and Ticket Overview
• Background on Email Tracking: We currently use UA to track email opens via pixel tracking. GA4 does not support pixel tracking directly. Instead, we will embed a dynamic URL in the email that sends a GET request to our application when the email is opened. This GET request will then trigger a POST request to GA4 with the necessary data.
• Jamie Stutt on dsva.slack.com, but please ask @MackHalliday first as she has already talked to her and does not want to ask Jamie the same questions.
• Campaign Details GA4 fields
• New JSONSchemas should use a newer revision. See here for an example.

Engineering Checklist

• [x] Create GET endpoint to accept GET request from an open email

  • URL

    url = (
       f'{public_application_root}/ga4/open-email-tracking?'
       f'campaign={notification.template.name}&campaign_id={notification.template.id}&'
       f'name=email_opens&source=vanotify&medium=email&'
       f'content={notification.service.name}/{notification.service.id}/{notification.id}'
    )
    

• [x] Blueprint created for ga4 route (no auth)

• [x] Validate all query params using schema validation on the query parameters

• [x] Unit tests for schema vaildation

• [x] Ensure type validation as well e.g. each UUID field should be a valid UUID, content should be <string <=255>/<uuid>/<uuid>, etc.

• [x] Log the query string from the GET request

• [x] Add any necessary error handling

Acceptance Criteria

• The GET endpoint is created and can accept GET requests from opened emails
• Logs the extracted values
• No database queries at all
• Returns a 204 status code

QA Considerations

• Ensure GET requests are successfully received and values are correctly extracted.

Out of Scope

• Creating a POST request to send the open email data to GA4
• Removing the old GA3/UA email tracking functionality

[mjones-oddball]

Hey team! Please add your planning poker estimate with Zenhub @cris-oddball @EvanParish @k-macmillan @kalbfled @MackHalliday @mchlwellman

[kalbfled]

The query parameters are logged many times because they are part of the URL, but I'm not sure that I'm seeing the output of the dedicated logged call I wrote in the route handler. I will wrap this up in the morning.

[kalbfled]

I found the relevant logging statement. This is the HTTPie command I used to test endpoint:

http https://dev-api.va.gov/vanotify/ga4/open-email-tracking campaign==hi campaign_id==e774d2a6-4946-41b5-841a-7ac6a42d178b name==email_open source==vanotify medium==email content=='test/e774d2a6-4946-41b5-841a-7ac6a42d178b/e774d2a6-4946-41b5-841a-7ac6a42d178b'

You could also use cUrl or a browser to do the same thing.

[cris-oddball]

PR Approved and merged, will deploy to Perf bu no farther until I discuss with the team about sending all 3 PRs up as a feature package to Prod instead of dripping them out.

[cris-oddball]

awaiting second PR and putting this under automation.

[cris-oddball]

Second PR approved and merged and sent via pipeline to Perf.

[cris-oddball]

QA PASSED

• [x] TC01: Happy Path - except 204, no response body and logging
• [x] TC02: Negative - campaign_id not a UUID, expect 400, error message, and logging
• [x] TC03: Negative - name is not 'email_open', expect 400, error message, and logging
• [x] TC04: Negative - source is not 'vanotify', expect 400, error message, and logging
• [x] TC05: Negative - medium is not 'email', expect 400, error message, and logging
• [x] TC06: Negative - content is not a string comprised of string/uuid/uuid, , expect 400, error message, and logging
• [x] TC07: campaign is None, expect 400, error message, and logging
• [x] TC:08: each query parameter is missing, expect 400, error message, and logging

since the route and query parameters are encoded as a string, not testing for strings that are not string, since it will be sent as a string anyway.

Putting the route under automation.

[cris-oddball]

error response for the above test cases:

{
    "errors": [
        {
            "error": "ValidationError",
            "message": "'hello' is not a 'uuid'"
        }
    ]
}

{
    "errors": [
        {
            "error": "ValidationError",
            "message": "'email_open' was expected"
        }
    ]
}

{
    "errors": [
        {
            "error": "ValidationError",
            "message": "'vanotify' was expected"
        }
    ]
}

{
    "errors": [
        {
            "error": "ValidationError",
            "message": "'email' was expected"
        }
    ]
}

{
    "errors": [
        {
            "error": "ValidationError",
            "message": "'foo' does not match '^[^/]{1,255}/[0-9a-zA-Z-]{36}/[0-9a-zA-Z-]{36}$'"
        }
    ]
}

{
    "errors": [
        {
            "error": "ValidationError",
            "message": "'foo/foo' does not match '^[^/]{1,255}/[0-9a-zA-Z-]{36}/[0-9a-zA-Z-]{36}$'"
        }
    ]
}

{
    "errors": [
        {
            "error": "ValidationError",
            "message": "'foo/foo/bar' does not match '^[^/]{1,255}/[0-9a-zA-Z-]{36}/[0-9a-zA-Z-]{36}$'"
        }
    ]
}

[cris-oddball]

Writing automation, but closing this ticket since Evan, Dave and I decided to just go ahead and release this today rather than wait for the other two PRs that complete this feature. This will be a Minor version bump to 1.9.0 and then the next two PRs will be patch bumps to complete the feature.

This is definitely something we want to brainstorm as a team for the next time this happens.