Closed jmichel17 closed 1 month ago
hi @jmichel17, thanks for reporting this issue.
Since you're using github actions, would it be possible for you to use the agentless mode? https://docs.datadoghq.com/continuous_integration/setup_tests/javascript/?tab=cloudciprovideragentless
This should make the setup easier and hopefully fix the flush issue. As for the leak, they seem to be from requests to the agent, so this change should improve that as well.
@juan-fernandez Thank you for the response.
Unfortunately, I have been unable to resolve this for my team as of yet. I tried utilizing agentless mode to ensure full flushing. In addition I tried all of the following but with still incomplete reporting up to the DD CI dashboard:
I should also add... I also tried upgrading to Node v16 in case that mattered, but unfortunately got the same result.
hi @jmichel17 , thanks for the information. There seems to be an error before the flush. Did you try to run your tests with DD_TRACE_DEBUG=1 DD_TRACE_LOG_LEVEL=debug
? It might give us an indication of what's happening. Additionally, could you use NODE_OPTIONS=-r dd-trace/ci/init
for the initialization rather than using the test environment? It shouldn't make a difference but it's slightly simpler (more info in https://docs.datadoghq.com/continuous_integration/setup_tests/javascript/?tabs=onpremisesciproviderdatadogagent#instrument-your-tests)
Hi,
I am experiencing the very same on my side. In our case, we have a test suite of 218 test cases and what we see it's being reported is actually a random number of tests that can go from 90 to 160, just by rerunning the same job.
Just tried with the options you commented @juan-fernandez and I did see something "interesting"
For many payloads I see that the parent_id
is set to 0000000000000000
while in many others I do see a more true-ish value such as 7c7ce0be7d14822d
Encoding payload: [{"trace_id":"43ac2edc28887db4","span_id":"3a185e3b9a5258eb","parent_id":"0000000000000000",
I also saw other traces like the following
PASS src/components/Landing/.../....spec.tsx
Extract from carrier: {"x-datadog-trace-id":"6613246714027553703","x-datadog-parent-id":"8970291666562089517"}.
PASS src/contexts/Localization/....spec.ts
Extract from carrier: {"x-datadog-trace-id":"1938286422715588783","x-datadog-parent-id":"0000000000000000"}.
or like this
• GET http://www.apitesturl.com/api/payee/trend/payouts
If you still wish to intercept this unhandled request, please create a request handler for it.
Read more: https://mswjs.io/docs/getting-started/mocks
at Object.warn (node_modules/msw/src/utils/internal/devUtils.ts:17:11)
at applyStrategy (node_modules/msw/src/utils/request/onUnhandledRequest.ts:208:18)
at onUnhandledRequest (node_modules/msw/src/utils/request/onUnhandledRequest.ts:233:3)
at handleRequest (node_modules/msw/src/utils/handleRequest.ts:80:5)
at setupServerListener (node_modules/msw/src/node/createSetupServer.ts:75:24)
console.warn
[MSW] Warning: captured a request without a matching request handler:```
Do you have any suggestion?
Thanks
a 0
parent id is expected for test spans, so I don't think that's an issue. Could you provide the following info:
DD_TRACE_DEBUG=1
and DD_TRACE_LOG_LEVEL=error
? Oh ok good to know.
The versions I am using are dd-trace: 4.13.1
jest: 27.5.1
, using agentless (DD_CIVISIBILITY_AGENTLESS_ENABLED: true
).
I run it with this command where I use react-scripts test for this repo
DD_CIVISIBILITY_AGENTLESS_ENABLED=true DD_TRACE_DEBUG=1 DD_TRACE_LOG_LEVEL=debug DD_SITE=datadoghq.com NODE_OPTIONS="-r dd-trace/ci/init" DD_ENV=ci DD_SERVICE=my-service yarn test:dd src --coverage
where test:dd is the following:
"NODE_OPTIONS=\"-r dd-trace/ci/init\" DD_ENV=ci DD_SERVICE=my-service react-scripts test"
Worth mentioning that in other repo, I run it with jest (without any react-scripts on top of it) and I do not have any issues.
I've just run it using the options DD_TRACE_DEBUG=1 DD_TRACE_LOG_LEVEL=error
as you recommended and I see the following.
Error: Intelligent Test Runner configuration could not be fetched. Request to settings endpoint was not done because Datadog application key is not defined.
at error (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/log/writer.js:104:11)
at onError (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/log/writer.js:81:16)
at Channel.publish (node:diagnostics_channel:56:[9]
at Object.error (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/log/index.js:67:20)
at /home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/plugins/ci_plugin.js:36:15
at /home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/ci-visibility/exporters/ci-visibility-exporter.js:146:9
at getItrConfiguration (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/ci-visibility/intelligent-test-runner/get-itr-configuration.js:42:14)
at /home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/ci-visibility/exporters/ci-visibility-exporter.js:140:7
Error: No original method shard.
at assertMethod (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/datadog-shimmer/src/shimmer.js:133:[11]
at wrapMethod (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/datadog-shimmer/src/shimmer.js:39:3)
at Object.wrap (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/datadog-shimmer/src/shimmer.js:82:7)
at /home/runner/work/my-project/my-project/node_modules/dd-trace/packages/datadog-instrumentations/src/jest.js:198:11
at requireOrImportModule (/home/runner/work/my-project/my-project/node_modules/jest-util/build/requireOrImportModule.js:53:28)
at runJest (/home/runner/work/my-project/my-project/node_modules/@jest/core/build/runJest.js:[26]
Will check the application key thing just in case
Oh ok good to know.
The versions I am using are
dd-trace: 4.13.1
jest: 27.5.1
, using agentless (DD_CIVISIBILITY_AGENTLESS_ENABLED: true
).I run it with this command where I use react-scripts test for this repo
DD_CIVISIBILITY_AGENTLESS_ENABLED=true DD_TRACE_DEBUG=1 DD_TRACE_LOG_LEVEL=debug DD_SITE=datadoghq.com NODE_OPTIONS="-r dd-trace/ci/init" DD_ENV=ci DD_SERVICE=my-service yarn test:dd src --coverage
where test:dd is the following:
"NODE_OPTIONS=\"-r dd-trace/ci/init\" DD_ENV=ci DD_SERVICE=my-service react-scripts test"
Worth mentioning that in other repo, I run it with jest (without any react-scripts on top of it) and I do not have any issues.
I've just run it using the options
DD_TRACE_DEBUG=1 DD_TRACE_LOG_LEVEL=error
as you recommended and I see the following.Error: Intelligent Test Runner configuration could not be fetched. Request to settings endpoint was not done because Datadog application key is not defined. at error (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/log/writer.js:104:11) at onError (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/log/writer.js:81:16) at Channel.publish (node:diagnostics_channel:56:[9] at Object.error (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/log/index.js:67:20) at /home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/plugins/ci_plugin.js:36:15 at /home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/ci-visibility/exporters/ci-visibility-exporter.js:146:9 at getItrConfiguration (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/ci-visibility/intelligent-test-runner/get-itr-configuration.js:42:14) at /home/runner/work/my-project/my-project/node_modules/dd-trace/packages/dd-trace/src/ci-visibility/exporters/ci-visibility-exporter.js:140:7 Error: No original method shard. at assertMethod (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/datadog-shimmer/src/shimmer.js:133:[11] at wrapMethod (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/datadog-shimmer/src/shimmer.js:39:3) at Object.wrap (/home/runner/work/my-project/my-project/node_modules/dd-trace/packages/datadog-shimmer/src/shimmer.js:82:7) at /home/runner/work/my-project/my-project/node_modules/dd-trace/packages/datadog-instrumentations/src/jest.js:198:11 at requireOrImportModule (/home/runner/work/my-project/my-project/node_modules/jest-util/build/requireOrImportModule.js:53:28) at runJest (/home/runner/work/my-project/my-project/node_modules/@jest/core/build/runJest.js:[26]
Will check the application key thing just in case
thanks, that's helpful. Don't worry about the application key, it's not mandatory.
This error message Error: No original method shard.
is worrisome though.
We don't test react-scripts
so that might be related.
What jest testRunner are you using? jest@27 has jest-jasmine2
as default, so if there's none specified, that'd be the one.
It's jest-circus
as I can see traces of it in the logs
We encountered the same issue. All tests were executed in a Github action job, but none of them were reported in the datadog CI test dashboard. The log displayed the error message "Error: No original method shard.”
We resolved this issue by upgrading Jest from version v27 to v29. Although we have yet to determine the root cause, the test report is now displaying normally in the datadog CI test dashboard.
closing due to lack of activity. Feel free to reopen if you encounter this issue again.
Expected behaviour
Instrumenting
jest
tests withdd-trace
should not leak any handles.Actual behaviour
My
jest
test suite contains 76 tests in it. When executing all of these tests in a Github actions job, only ~32 were actually getting reported to the Datadog CI test dashboard. Adding an arbitrarily selected 10 second wait after the jest tests completed resulted in 75 of the 76 getting reported up to the dashboard. Presumably, the difference in tracing coverage was due to the fact that thedd-trace
library was not flushing its traces.I've yet to see all 76 tests reported. On the runs where 75 were reported with the help of a wait, the following error was logged:
In combination with jest's
--detectOpenHandles
and--runInBand
options plus theleaked-handles
npm package (https://github.com/Raynos/leaked-handles), I discovered there is evidencedd-trace
is the culprit leaking handles and potentially causing the forced exit. And in turn probably also the cause of the 76th test not getting reported.The pertinent logs produced by
leaked-handles
are as follows:and...
Steps to reproduce
The trouble in reproducing this is we aren't doing anything exotic in our
jest
test suite. We run our tests with thejest-environment-jsdom
test environment and include thedd-trace
library per your documentation:Environment
Operation system: https://hub.docker.com/layers/node/library/node/14.19-buster-slim/images/sha256-a3b69090a0643dc06e80783ed3414c7e5decf8f4768c79124c379a7717fffe4c?context=explore
Node.js version: v14.19
Tracer version: v2.9.1
Agent version:
Relevant library versions: v2.9.1