rudderlabs / community-user-transformations

MIT License
1 stars 1 forks source link

Re-shape incoming webhook source payloads #4

Open ericdodds opened 1 year ago

ericdodds commented 1 year ago

Contact Details

eric@rudderstack.com

Language

Javascript

Category

Custom Integrations/Other

Description

When you ingest data into a webhook source in RudderStack, the default functionality of the webhook is to standardize the payload as a `webhook_source_event`. This makes sense as users use webhook sources to ingest all kinds of data. When sending payloads from a webhook source to an event-based cloud destination, though, it is very useful to re-shape the `webhook_source_event` to reflect the RudderStack event spec so that the data shows up in the destination as expected. 

For example, if you are ingesting webhook events and you want to send them to a product analytics tool, you would want to re-shape the `webhook_source_event` into a standard `.track` or `.identify` call. 

This transformation includes a few simple updates that translate a `webhook_source_event` into a properly shaped and named `.track` call. You can adapt this to generate any type of RudderStack call (`.identify`, `.group`, etc.)

Note that some of the adjustments are required, while others are optional depending on your use case.

Code Block

export async function transformEvent(event, metadata) {

  if (event.event == 'webhook_source_event') {

    // REQUIRED SETTINGS

    // Delete messageId to prevent event duplication. Webhook sources generate a new, unique messageId by default.
    delete event["messageId"];

    // Set the correct name for the event. Webhook sources set the default event name to `webhook_source_event`. This assumes you have set a top-level key called 'event'. If you don't, then adjust the dot notation to use the desired value in the incoming payload.
    event.event = event.properties.event;

    // Reposition the properties object to be in the correct hierarchy. This assumes you have an object called 'properties' describing the event, which is usable by most RudderStack destinations.
    event.properties = event.properties.properties;

    // OPTIONAL SETTINGS

    // Set anonymousId. Webhook sources generate new unique anonymousId values by default. Depending on your incoming payload, you may have a unique value that you can use, or you may want to use some other user trait or property. The example below assumes you have a properties object in the incoming payload.
    event.anonymousId = event.properties.propertyName;

    // Update timestamps. Webhook sources default timestamps to the time the event is ingested. In many cases, webhook sources are used to ingest batches of events that occured prior to being sent to the webhook source event. Assuming your original payload contains your desired timestamp values, you can use those to update the default timestamp values. For example, if your event had a 'start_time' key in a properties object in the incoming payload, you would say event.timestamp = event.properties.start_time;
    event.timestamp = event.properties.propertyName;
    event.originalTimestamp = event.properties.propertyName;

    // Create a traits object and add traits. Webhook source events do not generate a 'traits' object, which is used to describe users with trait keys. In the example below, we are passing userId through as a property, but want to re-position that key as a user trait within the traits object. For example, if your payload has an 'email' key in the properties object, you would map "userId": event.properties.email
    let traits = {
      "trait": event.properties.propertyName;
    }

    event.traits = Object.assign(traits)
  }

  return event;
}

Input Payload for testing

{
  type: "track",
  event: "webhook_source_event",
  rudderId: "044448e2-a674-426c-ba61-8341262babcc",
  messageId: "4379907d-689a-4e3a-a2f7-477e29a02299",
  properties: {
    type: ["subscribe"],
    "data[id]": ["e2ff089583"],
    fired_at: ["2021-07-28 08:06:59"],
    "data[email]": ["[name@rudderlabs.com](mailto:name@rudderlabs.com)"],
    "data[ip_opt]": ["115.187.35.152"],
    "data[web_id]": ["161912900"],
    "data[list_id]": ["ec4689c266"],
    "data[email_type]": ["html"],
    "data[merges][EMAIL]": [
      "[name@rudderlabs.com](mailto:name@rudderlabs.com)",
    ],
    "data[merges][FNAME]": ["Name"],
    "data[merges][LNAME]": ["Surname"],
    "data[merges][PHONE]": [""],
    "data[merges][ADDRESS]": [""],
    "data[merges][BIRTHDAY]": [""],
  },
  anonymousId: "d6597ba2-54db-4bd7-8769-86ac067b4178",
}

License