search

elastic / apm-agent-php

Apache License 2.0
252 stars 69 forks source link

TPM is showing weird values, but only for PHP applications #951

Closed firecow closed 8 months ago

firecow commented 1 year ago

Describe the bug Kibana APM TPM visualization is way off

The service simply has the APM extension installed and various APM environment variables set, it has no custom instrumentation implemented in the php code, it relies solely on auto instrumentation by the extension.

It runs in 3 containers across 3 nodes in production.

Right, now it has a sample rate of 1%

It gets on avg 100req/sec, and not much spiky behavior, and this is what I'm seeing.

image

I can assure you, that if our service does in fact receives 272737 TPM (4545 req/sec), our production environment would be in a heap of trouble :smile:

We have a nodejs application running in production, and for that application TPM is shown correctly in Kibana APM visualizations.

To Reproduce Unfortunately I cannot reproduce.

I've tried "hammering" a local setup with 100req/sec, and tried different sample rates, but to no avail (I had a feeling it had something todo with sample rate value)

I've tried running 3 containers locally, but still I cannot reproduce.

I've scaled containers to 6 in production, the TPM value stayed the same :sob:

I've modified sample rate from 0.01 to 0.02 directly in production to no avail as well.

Expected behavior I would expect our service to have 6000'ish TPM

intuibase commented 8 months ago

Hey @firecow

Could you please try to reproduce it with latest veersion of APM agent? We added support for attaching sample_rate to transaction in the latest release - it should be handled correctly.

Please let us know if it solved issue.

SergeyKleyman commented 8 months ago

Closing it for now - please let us know if it's still relevant.