DataDog / browser-sdk

Datadog Browser SDK
Apache License 2.0
291 stars 130 forks source link

Allow custom tags for Browser and RUM on the initialization #462

Open rodrigopinto opened 4 years ago

rodrigopinto commented 4 years ago

The current applications that we monitor we make extensive use of tags to be able to aggregate and filter the logs in the way the teams get easier debugging.

Running the browser log it is possible to see default tags being added based on the params of passed on the init method. As can be confirmed here https://github.com/DataDog/browser-sdk/blob/3b25ae0f92842f37be0a343dbd5d690fe973889f/packages/core/src/configuration.ts#L192-L196.

I was wondering if would be ok to have an optional parameter to the interface UserConfiguration to allow custom tags during the init of the browser logs and RUM.

bcaudan commented 4 years ago

Hi @rodrigopinto

There is ongoing work to merge the attributes and tags concepts so we are not willing to introduce this kind of API for now. However, as you are using logs, you can configure a remapper to set an env tag from the env attribute.

rodrigopinto commented 4 years ago

Hi @bcaudan thanks for the reply. And sure, remapper would work, but just out of curiosity this ongoing work you mean internally at DD or on the lib?

bcaudan commented 4 years ago

It is an internal reflexion at DD, if we don't carry it out, we could reassess this feature

michal-filip commented 3 years ago

maintainer update: behavior disabled since v4

For the record, as long as this feature isn't carried out on datadog side, you can "hack" it by including the custom tags into one of the initialization options like this:

datadogLogs.init({
  ...,
  env: 'dev,branch:master,commit:abcd12'
})

Which of course is a bug in the first place, as the commas in those options should probably get escaped.

g-dx commented 3 years ago

I too am very interested in the ability to add custom tags. Are there are any updates to report on this?

(For reference this is to enable building a custom dashboard which allows the user to select which RUM application to display using a templated variable pointing to the custom tag, rather than using the service tag which overloaded with lots of things or @application.id which uses GUIDs)

bcaudan commented 3 years ago

@g-dx, what about using a global context attribute instead of a tag?

g-dx commented 3 years ago

Hey @bcaudan, I wasn’t aware of this functionality. Sounds like it fits my usecase perfectly.

Thanks.

uluzox commented 3 years ago

Just in case you ran into that problem yourself:

We restrict the access to logs by tags. Namely we have a "tenant" tag. This cannot be added with the browser-logs. However, using the global context as proposed here an attribute of the same name can be added to the log message in the client code e.g.

datadogLogs.addLoggerGlobalContext('tenant', 'my-tenant');

On the logs pipelines section, I then added a step where this attribute is copied into a tag of the same name

Bildschirmfoto 2021-08-05 um 12 30 47
Olli73773 commented 1 year ago

Hi, we need to bring up this issue again. We use a tag based "Data Access" rule for logs. This is requiered by our Security and Compliance policy and the setup was implemented together with a Datadog TAM. The workaround with the pipeline is not realy working since the team do not see there logs at all as long the requiered tag for Data Access rule is not present.

bcaudan commented 1 year ago

Hi @Olli73773,

We still don't have plans to support passing tags from the SDK. Could you reach out directly to Datadog support or to you TAM to see how your use case could be addressed?

davidcam-nwse commented 7 months ago

Hi @bcaudan ,

I'd like to reopen this discussion. It doesn't make sense to me to support custom tags in many other places in Datadog UI (read: APM BE Traces and BE Logs, etc.) and in the official libraries (read: dd-trace for JavaScript/TypeScript/Node) and then NOT support them at the FE side in browser-sdk (RUM, FE Logs, APM FE Traces...). Sorry, but it just doesn't make sense.

This breaks the product capabilities IMHO as it doesn't allow tag-filtering in other parts of the UI.

I have an specific example that we're suffering out of it. We're not able to add custom tags in FE like we have in BE (we have tags like "application" to group services, or "cost_center" to group also by this concept. We make use of these tags for the backend services in this Datadog Feature: Cost Usage Attribution (https://nwse.datadoghq.eu/billing/usage-attribution).

With your statement the problem is that now our Usage Attribution data is basically broken, as we can't filter FE with "application" and/or "cost_center" just like we already do with backend services.

Please reconsider that decision and allow custom tags and/or tags from attributes or a similar solution to enable tag filtering in other parts.

p.s.: I'll be glad to get feedback that I'm wrong with my statements and/or there are alternative options or solutions available to overcome these, if that's the case please just let me know.

amortemousque commented 7 months ago

Hi @davidcam-nwse, Appreciate your input. At the moment, there are no plans tu support passing tags. Looking at your use case, I recommend reaching out to Datadog support to explore potential solutions.

davidcam-nwse commented 7 months ago

Thanks for your quick reply @amortemousque.

I'd also like to ask if there's a way I can process these JSON attributes in/from Datadog UI and convert those JSON attributes from my logs into tags later in Datadog.

I wonder if that would be feasible somehow (?)

(i.e.: if I add those custom tags as JSON attributes as mentioned above in the thread)

I will translate this question also to Datadog support, but please let me know if you know any known method to accomplish this (convert JSON attributes to tags in Datadog).

I will appreciate and be really grateful about any suggestions or ideas about this.

antoinerey commented 7 months ago

On our side, we do just that actually. We pass "tags" as JSON attributes, and we then use a log pipeline to convert them to tags.

Your log may roughly look like this:

{
  // ...
  "tags": {
    "application": "abc",
    "cost_center": "xyz"
  }
}

And then, you may define a log pipeline like so:

CleanShot 2024-02-12 at 10 11 59@2x

The downsides (vs native tag support) are:

  1. Your pipeline must define ALL tags it will extract. There is no way to make it generic for any tags (to the best of my knowledge).
  2. You have to update your pipeline every time you add, rename, or remove a tag from your logs.
davidcam-nwse commented 7 months ago

Thank you very much @antoinerey for the example, I think it makes a lot of sense.

julianguyen commented 7 months ago

@antoinerey How did you pass in custom attributes into the JSON? Was it through the beforeSend function in datadogRum.init() and the event argument?

antoinerey commented 7 months ago

I understand your question as "How do you avoid repeating those tags over and over again?". Am I right?

If so, we built a small wrapper around the Datadog SDK which is responsible of setting up those tags for all our logs. It roughly looks like this:

export function log(message, attributes) {
  datadogLogs.logger.info(message, {
    ...attributes,
    tags: {
      application: "ABC",
      cost_center: "XYZ",
    },
  })
}
julianguyen commented 7 months ago

@antoinerey Thanks for sharing!

Are you able to use the custom tags in the RUM view events?

antoinerey commented 6 months ago

Oh. We're not using the RUM product, only the logger. So, I can't say. 🙇