getsentry / sentry-laravel

The official Laravel SDK for Sentry (sentry.io)
https://sentry.io
MIT License
1.25k stars 189 forks source link

`client_sample_rate` for auto instrumented Queue spans is always 1 in UI even if sampled at a lower rate. #946

Open Fwang36 opened 3 weeks ago

Fwang36 commented 3 weeks ago

How do you use Sentry?

Sentry SaaS (sentry.io)

SDK version

4.8.0

Laravel version

11.9

Steps to reproduce

  1. Set up tracesSampler to conditionally sample an auto instrumented Queue span.

        $transactionName = \Illuminate\Support\Str::of($context->getTransactionContext()?->getName())->value();
    
        if (str_contains($transactionName, LogTestMessage::class)) {
    
            return 0.1;
        }
    
        return 0.99;
  2. Enable the logger and check the logs to see the sampling occuring.
  1. Search for the event Id in the logs and see that the samping rate is set to 1.

Link to the transaction

Expected result

client_sample_rate should show as .1 like I have set in the tracesSampler

Actual result

client_sample_rate shows as 1 even though I do not have 1 set anywhere in my tracesSampler

cleptric commented 3 weeks ago

Hmm, this works fine for me: https://sentry-sdks.sentry.io/performance/trace/4169ef053f87462c813a181ede76c051

'traces_sampler' => function() {
    return 0.1;
},

Image

stayallive commented 3 weeks ago

Can you show a bit more about how exactly the tracesSampler is configured?

I also cannot reproduce and the sample rate is shown correctly as 0.99.

Fwang36 commented 3 weeks ago

Hey sorry, I should of clarified more. For me It only happens on the queue spans. The normal network request spans work fine when I sample them.

I edited the tracesSampler to include a network request span, and the client sample rate is shown properly. Here is the entire tracesSampler -

    public static function tracesSampler(SamplingContext $context): float
    {

        $transactionName = \Illuminate\Support\Str::of($context->getTransactionContext()?->getName())->value();

        if (str_contains($transactionName, LogTestMessage::class)) {

            return 0.1;
        }

        if (str_contains($transactionName, "store-cache")) {
            return 0.98;
        }

        return 0.99;
    }

This is the new Queue span that shows a client_sample_rate of 1 instead of 0.1

And this is the new http.server /store-cache network span that correctly shows 0.98

stayallive commented 3 weeks ago

Okay, I've setup a database queue like you did and fired a job but still:

Image

I did simplify the traces_sampler to just the following in my config/sentry.php:

    'traces_sampler' => static function() {
        return 0.99;
    },

Wondering what variable we are missing here that causes this.

I am on Laravel 11.16 and using the 4.9.0 of the Laravel SDK, but that has no relevant changes as far as I can see.

Fwang36 commented 3 weeks ago

I changed the traces_sampler to yours and also updated laravel and Sentry. I still see the same thing. Maybe it's because of how I am triggering the transaction? I tried to replicate a users set up who also ran into this issue. I'm running php artisan queue:work to start the queue worker and then php artisan app:dispatch-log-test-message to trigger the transaction.

I made a screen recording of exactly how I'm doing it if this helps at all

https://github.com/user-attachments/assets/de25fc79-bc93-41d1-8c62-c4b0a304a503

cleptric commented 2 weeks ago

There might be something off when we transform the data into an envelope. I have no other ideas tbh. cc @stayallive

stayallive commented 2 weeks ago

I am so puzzled right now. I really appreciate all the info you are giving is. And seeing this I think I'm replicating it correctly on my end but with a different result so trying to figure out where the missing link is.

Are you able to show us how you dispatch the job and maybe even a little of what the job is doing (is it doing anything "special" like a HTTP call or calling a route in the application etc.).

Again, appreciate the help with figuring this out because I think there is a bug indeed. It's just very specific 🥲

cleptric commented 2 weeks ago

One thought I just had is, that it looks like we apply the parent sampling decision we propagate to the queue job with higher priority than the traces sampler, hence it's always 1.

stayallive commented 6 days ago

This is a good one to test indeed, will need to just try some stuff. Def something not right here.