Open vany0114 opened 2 months ago
cc @JacksonWeber
@vany0114 unfortunately Node.js SDK does not currently support it, you can find more info about which SDKs support it here
@vany0114 unfortunately Node.js SDK does not currently support it, you can find more info about which SDKs support it here
@hectorhdzg thanks for the clarification, I guess I can implement it via telemetry processor?
Telemetry processors are no longer supported in v3 of this SDK, but you should be able to do that using v2, in latest version we are using OpenTelemetry internally, and SpanProcessors or LogProcessors are not designed for sampling, so telemetry cannot be filtered out there, a custom sampler could be used, but it only apply to request/dependency data, logs samplers are still under discussion as far as I know.
Is there any examples of how to achieve that in v3 using a customer sampler as you pointed out?
@hectorhdzg quick question, is there a way to configure the log level to avoid ingesting these runtime traces?
Executing 'Functions.HttpTrigger1' ...
Executed 'Functions.HttpTrigger1' (Succeeded ...)
I noticed that using the Application Insights SDK there's no way we can sample those, they never land in the telemetry processor, I guess it's because they are runtime traces.
I know we can configure the log level per function, like:
Function.HttpTrigger1": "None"
However, that's not ideal or maintainable and if I do this:
Function.": "None"
Then I lose all the logs.
We need to do so because they swamped our telemetry data, they are not useful for us and we're being charged for that which is super expensive.
@vany0114 can you check the SDK version of the unexpected telemetry in Kusto? all telemetry should reach the telemetry processor if using v2.x, if is actually generated by Node.js SDK is possible this is coming from console auto capturing, you should be able to turn that off if that is the case.
@hectorhdzg this is the query I'm using however I don't see the SDK version there? 🤔 can you please tell me how?
AppTraces
| where TimeGenerated > ago(1d)
| where _IsBillable == true
On the other hand, FWIW the package version we're using is 2.9.1
Yeah, I don't believe that telemetry is coming from this SDK, @ejizba do you know where is this data being generated and how to configure it?
FWIW as mentioned before, we're using Azure Functions, so I guess that telemetry comes from the Azure Functions runtime, I also see that all the traces we leave using Context.log
never reach the telemetry processors either.
That telemetry is coming from the Azure Functions host, nothing to do with Node.js.
A few ideas:
Fyi, I'm on the Azure Functions team focused on Node.js. I have some knowledge of the host log settings, but if you want the true experts you probably need an issue in this repo instead: https://github.com/Azure/azure-functions-host/issues
Do you have to set "Function" or "Function.HttpTrigger1" to "None" or is there a different level like "Error" or "Warning" that would work? I know you said you don't want "Executing" and "Executed" logs, but I'm curious which logs you do want.
I want only the logs we produce, and as I said before, the "Function.HttpTrigger1" configuration is not maintainable, we have a lot of functions + that's easy to miss when a developer adds a new one. The "Function" configuration does not work either as it disregards all the logs even ours, so I cannot set it as "Warning" for example, because then we lose all of our traces.
Sampling settings in host.json should work for these logs. The settings can be confusing (docs here), but if you mess around with them long enough you may get something that works
Trust me, we've tried everything and none of that works, that's why I opened this issue and @hectorhdzg confirmed that that configuration does not work with this SDK.
Hey guys, I'm using the SDK in an Azure Functions environment to customize the way the telemetry is done, especially to correlate requests that the SDK does not support out of the box, such as Event Grid, etc, however, I noticed that it seems it is not honoring the Adaptive sampling configuration, the configuration in my
host.json
looks like this:So I'd expect it would sample the 50% of the dependency calls, traces, and everything that's not in the
excludedTypes
, however, it's not doing so.I've also noticed that the SDK exposes a property called
samplingPercentage
that can be used for kinda the same purposes, however, I guess that's a fixed percentage, isn't it? so for example, if I set it as 50 it would sample the 50% of all kinds of requests, right?Is there a way to make it work with the Adaptive sampling configuration? or is there a way to instruct it to sample a % of a given type of request, say, that samples 50% of dependencies and 60% of traces?