Correlation between Frontend and Backend.

obiwanjacobi commented 1 month ago

I have a React app running with @microsoft/applicationinsights-web that makes some HTTP calls to my NodeJS backend (B4F). The BE uses this package. Both are configured pointing to the same Application Insights instance on Azure.

I would like to correlate the FE calling the backend. I have added some headers to the request containing the traceId, sessionId and traceParent (from the FE). If this can be picked up automatically on the BE what do I call these headers in that case. Alternatively what code do I need to write to pass the data onto the BE AppInsights?

The BE also makes other HTTP calls to other systems I would like to correlate as well. I am using trackDependency for that but I am not seeing the 'call sequence' in the AppInsights detail view on the Azure Portal...? I suspect it is missing a session or parent id in order to tie them together. Any hint for that would be great.

obiwanjacobi commented 1 month ago

Hey @JacksonWeber, how is the investigation going?

JacksonWeber commented 1 month ago

Hi @obiwanjacobi, apologies for the delay, finally got some time to break this down.

Looks like there's two questions here:

There shouldn't be any additional code change necessary (no need to manually handle context propagation!) assuming you've instrumented both the frontend and backend applications with application insights SDKs and are trying to track an HTTP/HTTPS call between the two. If you're having trouble setting this up I can provide a sample app that demonstrates the setup.
If you're looking to track HTTP/HTTPS calls to other services, you shouldn't need to utilize the trackDependency method as these calls should be auto-collected and auto-correlated. Since this isn't working for you, could you provide an example of a call to one of these calls to other systems and a copy of your package.json would be helpful. Thanks!

obiwanjacobi commented 1 month ago

1) We use Koa on the BE perhaps that is the issue? An example is very much appreciated. 2) We use fetch. This is an example of our post function. Similar for get, put etc.

private async post(resource: string, requestBody: any, headers: Record<string, string>): Promise<Response> {
        const allHeaders = headers
            ? { ...headers, Authorization: this.authorization }
            : { "Content-Type": "application/json", Authorization: this.authorization };

        const url = `${this.baseUrl}/rest/api/2/${resource}`;
        const start = Date.now();
        const response = await fetch(url, {
            method: "POST",
            headers: allHeaders,
            body: requestBody,
        });

        logRequest("POST", url, response.status, start);    // <= trackDependency

        return response;
    }

We use a mono-repo so the FE and BE is mixed.

package.json:

{
  "name": "<project name>",
  "version": "1.0.0",
  "private": true,
  "scripts": {
    "== our scripts ==" : "<removed>"
  },
  "dependencies": {
    "@azure/identity": "^3.4.1",
    "@azure/keyvault-secrets": "^4.7.0",
    "@fluentui/react": "^8.112.0",
    "@fluentui/react-calendar-compat": "^0.1.5",
    "@fluentui/react-components": "^9.46.4",
    "@fluentui/react-datepicker-compat": "^0.4.35",
    "@fluentui/react-icons": "^2.0.218",
    "@koa/cors": "^4.0.0",
    "@koa/router": "^12.0.1",
    "@lexical/react": "0.12.4",
    "@microsoft/applicationinsights-web": "^3.2.1",
    "@nx/webpack": "^16.10.0",
    "@rjsf/core": "^5.14.3",
    "@rjsf/fluentui-rc": "^5.14.3",
    "@rjsf/utils": "^5.14.3",
    "@rjsf/validator-ajv8": "^5.14.3",
    "@swc/helpers": "~0.5.2",
    "@tanstack/react-query": "^5.0.5",
    "@tanstack/react-query-devtools": "^5.4.3",
    "@tanstack/react-table": "^8.11.7",
    "@types/he": "^1.2.3",
    "ajv-errors": "^3.0.0",
    "applicationinsights": "^3.1.0",
    "async-wait-until": "^2.0.12",
    "axios": "^1.0.0",
    "azure-devops-extension-api": "4.229.0",
    "azure-devops-extension-sdk": "4.0.2",
    "azure-devops-ui": "^2.236.0",
    "chalk": "^4.1.2",
    "commander": "^10.0.1",
    "console-table-printer": "^2.11.1",
    "cors": "^2.8.5",
    "export-from-json": "^1.7.4",
    "express": "~4.18.1",
    "figlet": "^1.6.0",
    "he": "^1.2.0",
    "immer": "^10.0.3",
    "koa": "~2.14.1",
    "koa-404-handler": "^0.1.0",
    "koa-better-error-handler": "^11.0.4",
    "koa-body": "^6.0.1",
    "koa-morgan": "^1.0.1",
    "koa-static": "^5.0.0",
    "lexical": "0.12.4",
    "lodash.startcase": "^4.4.0",
    "loglevel": "^1.9.1",
    "markdown-to-jsx": "^7.3.2",
    "morgan": "^1.10.0",
    "node-cache": "^5.1.2",
    "node-fetch": "^2.7.0",
    "parse-json": "^7.0.0",
    "patch-package": "^8.0.0",
    "re-resizable": "^6.9.11",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "react-drawio": "^0.1.2",
    "react-grid-layout": "^1.4.4",
    "react-markdown": "^8.0.7",
    "react-modal": "^3.16.1",
    "react-router-dom": "^6.18.0",
    "shelljs": "^0.8.5",
    "source-map-support": "^0.5.13",
    "tslib": "^2.3.0",
    "typescript-logging": "^2.1.0",
    "typescript-logging-category-style": "^2.1.0",
    "undici": "^5.27.0",
    "uuid": "^9.0.1",
    "vss-web-extension-sdk": "^5.141.0",
    "yaml": "^2.2.2"
  },
  "devDependencies": {
    "@babel/core": "^7.14.5",
    "@babel/preset-react": "^7.14.5",
    "@nrwl/nx": "^7.8.7",
    "@nx/esbuild": "16.10.0",
    "@nx/eslint-plugin": "16.10.0",
    "@nx/jest": "16.10.0",
    "@nx/js": "16.10.0",
    "@nx/linter": "16.10.0",
    "@nx/node": "16.10.0",
    "@nx/playwright": "^16.10.0",
    "@nx/react": "16.10.0",
    "@nx/storybook": "^16.10.0",
    "@nx/vite": "16.10.0",
    "@nx/web": "16.10.0",
    "@nx/workspace": "16.10.0",
    "@playwright/test": "^1.36.0",
    "@storybook/addon-essentials": "7.4.6",
    "@storybook/addon-interactions": "^7.2.1",
    "@storybook/core-server": "7.4.6",
    "@storybook/jest": "~0.1.0",
    "@storybook/manager-api": "^7.4.6",
    "@storybook/react-vite": "7.4.6",
    "@storybook/test-runner": "^0.13.0",
    "@storybook/testing-library": "~0.2.0",
    "@storybook/theming": "^7.4.6",
    "@swc/cli": "^0.1.63",
    "@swc/core": "~1.3.85",
    "@swc/jest": "0.2.20",
    "@testing-library/jest-dom": "^6.1.4",
    "@testing-library/react": "14.0.0",
    "@types/cors": "^2.8.15",
    "@types/decompress": "^4.2.5",
    "@types/express": "~4.17.13",
    "@types/jest": "^29.4.0",
    "@types/koa": "~2.13.5",
    "@types/koa__router": "^12.0.3",
    "@types/lodash.startcase": "^4.4.9",
    "@types/morgan": "^1.9.7",
    "@types/node": "^18.14.2",
    "@types/node-fetch": "^2.6.6",
    "@types/react": "18.2.24",
    "@types/react-dom": "18.2.9",
    "@types/react-grid-layout": "^1.3.5",
    "@types/react-modal": "^3.16.2",
    "@types/shelljs": "^0.8.11",
    "@types/unzip-stream": "^0.3.2",
    "@types/uuid": "^9.0.7",
    "@typescript-eslint/eslint-plugin": "^5.60.1",
    "@typescript-eslint/parser": "^5.60.1",
    "@vitejs/plugin-basic-ssl": "^1.0.1",
    "@vitejs/plugin-react": "~4.0.0",
    "@vitejs/plugin-react-swc": "~3.3.2",
    "@vitest/coverage-c8": "~0.32.0",
    "@vitest/ui": "~0.32.0",
    "ajv": "^8.12.0",
    "babel-jest": "^29.4.1",
    "decompress": "^4.2.1",
    "esbuild": "~0.19.2",
    "eslint": "~8.46.0",
    "eslint-config-prettier": "8.1.0",
    "eslint-plugin-import": "2.27.5",
    "eslint-plugin-jsx-a11y": "6.7.1",
    "eslint-plugin-playwright": "^0.15.3",
    "eslint-plugin-react": "7.32.2",
    "eslint-plugin-react-hooks": "4.6.0",
    "jest": "^29.4.1",
    "jest-environment-jsdom": "^29.4.1",
    "jest-environment-node": "^29.4.1",
    "jsdom": "~22.1.0",
    "kill-port": "^2.0.1",
    "nodemon": "^3.1.0",
    "nx": "16.10.0",
    "otpauth": "^9.1.5",
    "postcss-url": "^10.1.3",
    "prettier": "^2.6.2",
    "react-docgen": "^7.0.3",
    "rollup-plugin-copy": "^3.5.0",
    "sass": "^1.69.0",
    "scheduler": "^0.23.0",
    "tfx-cli": "^0.14.0",
    "tfx-umbrella": "file:./packages/tfx-umbrella",
    "the-new-css-reset": "^1.11.0",
    "ts-jest": "^29.1.0",
    "ts-json-schema-generator": "^1.5.0",
    "ts-node": "10.9.1",
    "typedoc": "^0.25.10",
    "typedoc-plugin-missing-exports": "^2.2.0",
    "typescript": "~5.1.3",
    "unzip-stream": "^0.3.1",
    "verdaccio": "^5.0.4",
    "vite": "~4.5.3",
    "vite-plugin-dts": "~2.3.0",
    "vitest": "~0.32.0"
  },
  "optionalDependencies": {
    "@nx/nx-darwin-arm64": "16.10.0",
    "@nx/nx-darwin-x64": "16.10.0",
    "@nx/nx-linux-x64-gnu": "16.10.0",
    "@nx/nx-linux-x64-musl": "16.10.0",
    "@nx/nx-win32-x64-msvc": "16.10.0"
  },
  "engines": {
    "npm": ">=9.6.7",
    "node": ">=18.17.0"
  }
}

JacksonWeber commented 4 weeks ago

As for the simple example app, I've attached a zipped project. web-node-correlation-test.zip

Your issue appears to be that you're using the fetch, which we don't support auto-instrumentation for yet. Support for fetch will be coming upon adoption of the fetch instrumentation from upstream OpenTelemetry in the Azure Monitor OpenTelemetry project (which 3.X of this project is a wrapper around).

If you need support for this instrumentation today to allow the use of fetch, you can pair Azure Monitor OpenTelemetry with the community instrumentation for the fetch API here.

JacksonWeber commented 3 weeks ago

Closing with this comment. Please reopen if you have further questions.

obiwanjacobi commented 3 weeks ago

I have tried your example but I don't see the frontend and backend calls correlated in AppInsights..? All I see is the backend with it's 'dependency' to bing that is being called. Am I doing something wrong or are my expectations unrealistic?

JacksonWeber commented 3 weeks ago

@obiwanjacobi Updated the example above to be simpler and more straightforward. Let me take a look regarding the correlation.

JacksonWeber commented 3 weeks ago

@obiwanjacobi Fixed the issue with the original sample app. You should now see correlated telemetry like this example in the Azure Portal:

obiwanjacobi commented 2 weeks ago

I'm sorry but it does not work at my end. I am not sure if I am doing something wrong...

I run npm install on both frontend and node-server folders
Copy in my AppInsights connection string (both frontend & node-server)
Open frontend url in Firefox (no errors in console or network (not counting favicon fail)) and click button

I only see the bing dependency being logged...

JacksonWeber commented 2 weeks ago

@obiwanjacobi Apologies, looks like there were issues with CORS in the example. I've updated the example to set the correct CORS policy for you in the node-server and updated the frontend HTML to enableCorsCorrelation: true.

Steps are:

run npm install in both the frontend and backend packages
Update both the index.html in frontend and index.js in the node-server folder with your connection string
Navigate into the node-server folder and run node index.js
Navigate into the frontend folder and run node index.js
Navigate to http://localhost:8080 (frontend server address) once both are running
Hit the "Start" button element to send an XMLHTTP request to the node server (which makes the call to the bing.com dependency)

Final result should look like:

microsoft / ApplicationInsights-node.js

Correlation between Frontend and Backend. #1333