vercel (formerly zeit now) integration

[aecorredor]

I'm not sure if this should be a feature request as it may be already supported (maybe I haven't figured out the right setup).

I'd like to be able to trace my Node.js server that is deployed to Vercel (formerly Zeit Now). Vercel deploys to AWS Lambda under the hood, and it offers no direct access to specific AWS configuration (like policies or setting up the Datadog lambda forwarder).

I was wondering if there's a way of setting this up already (maybe someone has already figured it out) or if there's a chance that Vercel could be supported.

[rochdev]

The way the Lambda forwarder works is that the tracer logs the traces to CloudWatch instead of sending them to the agent with HTTP, and then the forwarder is able to read that asynchronously and send it to Datadog.

I don't know a lot about Vercel, but if it has a way to stream logs to some kind of function or service, it should be possible to do something similar. Do you know if this is the case?

[aecorredor]

@rochdev that's the thing, from their docs and discussions in their GitHub (https://github.com/zeit/now/discussions/4147) they don't expose anything similar. They basically hide all the AWS details from the user, so setting up a Lambda forwarder is not possible for a Vercel user, since there's not really an AWS account that gets assigned.

I'm thinking that unless they implement something like that, adding Serverless APM on Vercel deployments won't be possible right?

I would have thought that given their popularity and Datadog's popularity someone would have already run into this issue.

[rochdev]

Even if they don't allow access to the underlying Lambda service, I would expect at least some way to forward logs. How do they recommend for example to use any external service at all for logging?

[aecorredor]

@rochdev well, I'm actually using Datadog's Node.js agentless logging along with winston and that's working fine: https://docs.datadoghq.com/logs/log_collection/nodejs/?tab=winston30#agentless-logging. But that has nothing to do with the specific deployment platform since it's all through HTTP.

And, about the specific logs that lambda produces they expose a Datadog integration but just for that: https://vercel.com/integrations/datadog-logs. Basically you add it to your Vercel dashboard and provide the API key, and they start streaming those logs to Datadog. But again, that's only logs and there's really no access into doing anything custom with the integration.

[rochdev]

Yeah it looks like they rely on integrations for everything, and there is no integration generic enough so that it would allow you to re-forward the logs. I'm afraid it means that right now your best option is to send the traces to an instance of the agent using HTTP. Is this something that is possible for you? It will add a slight delay to the function execution, but if the latency between the service and the agent is low enough, it should not be significant, especially with flushInterval set to 0.

As a side note, our forwarder is open source so it should be fairly straightforward for them to use it to write an integration :)

[aecorredor]

@rochdev awesome, thanks for the insight.

So I had thought about the solution of sending the logs to an agent hosted somewhere but then I saw here https://github.com/DataDog/dd-trace-js/pull/682/files#diff-f3377689fae59b1bfa08b8a4cb493495 that dd-trace automatically detects the AWS lambda environment and selects the LogExporter instead of the AgentExporter. So that will probably be an issue since I won't have a way to specify that I still want to use the agent right?

[rochdev]

I don't know enough about Vercel to answer this, but only the default forwarder changes when a Lambda environment is detected. It's still possible to change the exporter using a configuration option.

For example:

const tracer = require('dd-trace')
const { AGENT } = require('dd-trace/ext/exporters')

tracer.init({
  experimental: {
    exporter: AGENT
  }
})

[aecorredor]

@rochdev nice, I'll give that a shot and see if I can get this to work for now. Thanks again for all the help 🎉

[happylinks]

@aecorredor hey! Were you able to get this working? Looking for exactly the same thing :)

[aecorredor]

@happylinks at the end I couldn't. Setting up the datadog agent running somewhere else was fine. The issue I had was related to dd-trace running in a vercel v2 node deployment. I'm running a GraphQL API with apollo-server and express, and after adding dd-trace, the server just kept crashing because dd-trace couldn't find express in the node_modules. I'm thinking since vercel bundles up the code in the background (check the output of your deployment in the vercel dashboard), node_modules is not really there, and dd-trace expects it to. So, my conclusion was that dd-trace is just not compatible with vercel right now. I haven't had more time to dig deeper into this, so if you figure something out let me know.

[happylinks]

Ah, that's too bad, hopefully Vercel adds something for that soon. I'm not using a custom express server, so dd-trace might not have been a good candidate anyway. I also saw the discussion you opened on zeit/now repo. Will follow that one! Thanks!

[hden]

Maybe related

• https://vercel.com/integrations/datadog-logs
• https://github.com/zeit/integrations/tree/master/datadog

[aecorredor]

@hden indeed that's related, but it's more specific to logging. This thread is more about enabling APM, which is related to tracing your application's performance by using something like dd-trace. Every app usually has both, logging and APM.

[hden]

Just found this: https://github.com/dbader/node-datadog-metrics

Looks like a viable workaround.

[happylinks]

@hden ohh that looks interesting! I guess you'd have to call .flush yourself, but that might be a nice solution. Will probably try this soon.

[phil-lgr]

Sorry for the noise, but I also noticed that we have some more details on each request from vercel

But I am not sure on how to consume those, any ideas?

@aecorredor have you tried bundling dd-trace with your express app?

[pbassham]

You will need to set up pipelines to process them to useful datapoints

[askkaz]

This issue is a bit stale, but I'm running into the same issue. We are using the Datadog log drain for our Next.js app in Vercel, but unable to set up APM. The APM Vercel Integration is listed as being "in development". Is there a timeline or a workaround for setting it up?

[rochdev]

@askkaz We have support for NextJS in the tracer, but serverless support for specific environments like Vercel and Lambda is usually handled in external libraries that forward tracing data to our backend without going through the Datadog agent. I unfortunately don't have any timeline, and would recommend contacting our support team with a feature request.