Transactions not capture when doing stress test

[sigmaadam]

Hi,

I have system with micro services, when I'm doing regular flow, I can see in application insight the transaction, including the call to mongoDB.

When I'm running stress test via Jmeter, the mongo calls not appearing in application insight, and all the operation is not grouped.

[hectorhdzg]

@sigmaadam to be able to capture mongoDB telemetry, Application Insights SDK must be loaded before MondoDB, there is some patching in place and import order is important, please double check this is the case in your stress tests

[sigmaadam]

I do succeed to catch mongoDB telemetry,

Only when I'm doing stress test, it doesn't catch.

[hectorhdzg]

@sigmaadam I never used Jmeter before, but please double check Application Insights SDK is first to be loaded in that environment, if you can create a repro project would be great so we can debug and see what is happening

[sigmaadam]

Hi,

I'm loading the application insight SDK first line in my project:

const serviceName = 'admin-center-api'
import { MonitoringTracerService } from "cloud_common_utilities/monitoring/monitoringTracerService"
MonitoringTracerService.init(serviceName)

This is my library that wrap application insight:

//import tracer from 'dd-trace';
import { ServiceBusReceivedMessage } from '@azure/service-bus';
import * as appInsights from 'applicationinsights'
import { TelemetryClient } from 'applicationinsights';
import { CorrelationContextManager } from 'applicationinsights/out/AutoCollection/CorrelationContextManager';
import { DependencyTelemetry, EventTelemetry, MetricTelemetry } from 'applicationinsights/out/Declarations/Contracts';
import { ParamsEnum } from '../env/enumParams';
import envUtils from '../env/envUtils';
import { TrackDependencyModel } from 'cloud_common_models/models/monitoring/trackDependencyModel';
import { TrackEventModel } from 'cloud_common_models/models/monitoring/trackEventModel';
import { TrackMetricModel } from 'cloud_common_models/models/monitoring/trackMetricModel';
import {UUIDService} from '../uuid/uuidUtils'

export type ServiceBusConsumeCB = (message: ServiceBusReceivedMessage) => any
export class MonitoringTracerService {
    private static client: TelemetryClient | undefined

    public static init(serviceName: string = '', samplingRate: number  | null = null) {
        const method = 'init'

        const applicationInsightKey = envUtils.getString(ParamsEnum.APPLICATION_INSIGHT_INSTRUMENTATION_KEY)

        if (applicationInsightKey) {
            try {
                appInsights.setup(applicationInsightKey)
                    .setSendLiveMetrics(true)

                if (serviceName !== '') {
                    appInsights.defaultClient.context.tags[appInsights.defaultClient.context.keys.cloudRole] = serviceName;    
                }

                if (samplingRate !== null) {
                    appInsights.defaultClient.config.samplingPercentage = samplingRate // samplingRate% of all telemetry will be sent to Application Insights    
                }

                appInsights.start();
                this.client = appInsights.defaultClient;
                console.log(`Monitoring start successfully for service: ${serviceName}`)
            }
            catch (err) {
                console.error(`Monitoring start with error for service: ${serviceName}`, JSON.stringify(err))
            }
        } else {
            console.warn(`Monitoring missing applicationInsightKey`)
        }
    }

    /*
client.trackException({exception: new Error("handled exceptions can be logged with this method")});
client.trackTrace({message: "trace message"});
client.trackDependency({target:"http://dbname", name:"select customers proc", data:"SELECT * FROM Customers", duration:231, resultCode:0, success: true, dependencyTypeName: "ZSQL"});
client.trackRequest({name:"GET /customers", url:"http://myserver/customers", duration:309, resultCode:200, success:true});
    */

    public static trackDependency(event: TrackDependencyModel) {
        if (this.client) {
            const trackedEvent: DependencyTelemetry = {
                dependencyTypeName: event.dependencyTypeName,
                name: event.name,
                data: event.data,
                duration: event.duration,
                resultCode: event.resultCode,
                success: event.success
            }
            if (event.target) {
                trackedEvent.target = event.target
            }

            this.client!.trackDependency(trackedEvent)
        }
    }
    public static trackTrace(message: string) {
        if (this.client) {
            const traceEvent = {
                message
            }

            this.client!.trackTrace(traceEvent)
        }
    }
    public static trackException(exception: Error) {
        if (this.client) {
            const exceptionEvent = {
                exception
            }

            this.client!.trackException(exceptionEvent)
        }
    }

    public static async startServiceBusConsumeOperation(cb: ServiceBusConsumeCB, message: ServiceBusReceivedMessage, operationName: string, operationId: string | number | Buffer | undefined, parentId: string | number | Buffer | undefined) {
        return new Promise((resolve, reject) => {
            try {
                if (operationId === undefined || operationId === '') {
                    operationId = UUIDService.generateUUID()
                }
                if (parentId === undefined || operationId === '') {
                    parentId = UUIDService.generateUUID()
                }
                const correlationContext = CorrelationContextManager.generateContextObject(
                    operationId.toString(), // Use a guid here
                    parentId.toString(), // Supply the item id of the parent telemetry item this context should be a "child" of, or null
                    operationName // Use a string here
                );

                CorrelationContextManager.runWithContext(correlationContext, async () => {
                    this.trackTrace("Received message")
                    const res = await cb(message)
                    resolve(res)
                });
            }
            catch (err) {
                this.trackException(err)
                console.error(err.message)
                resolve([500, err.message])
            }
        })
    }

    public static trackEvent(event: TrackEventModel) {
        if (this.client) {
            const trackedEvent: EventTelemetry = {
                name: event.name
            }
            if (event.measurements) {
                trackedEvent.measurements = event.measurements
            }

            this.client!.trackEvent(trackedEvent)
        }
    }

    public static trackMetric(event: TrackMetricModel) {
        if (this.client) {
            const trackedMetric: MetricTelemetry = {
                name: event.name,
                value: event.value,
            }
            if (event.kind) {
                trackedMetric.kind = event.kind
            }
            if (event.count) {
                trackedMetric.count = event.count
            }
            if (event.min) {
                trackedMetric.min = event.min
            }
            if (event.max) {
                trackedMetric.max = event.max
            }
            if (event.stdDev) {
                trackedMetric.stdDev = event.stdDev
            }

            this.client!.trackMetric(trackedMetric)
        }
    }

    /* In case we want to log messages

    getSpan(methodName: string): Span {
        return tracer.startSpan(methodName)
    }

    log(level: string, message: any, span: Span) {
        const time = new Date().toISOString();
        const record = { time, level, message };

        tracer.inject(span.context(), formats.LOG, record);

        console.log(JSON.stringify(record));
    }*/
}

When I'm doing single operation I can see the mongoDB call:

Then when I'm calling with stress test, mongoDB call not appearing:

[JacksonWeber]

@sigmaadam Can you please try this with the new version of Application Insights (version 3.X) as we rely on OpenTelemetry instrumentations for MongoDB. Thank you.