getsentry / sentry

Developer-first error tracking and performance monitoring
https://sentry.io
Other
39.06k stars 4.19k forks source link

N+1-queries - threshold number of spans #76835

Open peterHakio opened 2 months ago

peterHakio commented 2 months ago

Problem Statement

I have a multi tenant setup, where I need to query across all tenants. This is generally done with one query per tenant. So I would love to be able to adjust the setting threshold number of queries in the admin panel

Image

Image

Solution Brainstorm

No response

Product Area

Settings

getsantry[bot] commented 2 months ago

Assigning to @getsentry/support for routing ⏲️

getsantry[bot] commented 1 month ago

Routing to @getsentry/product-owners-settings for triage ⏲️

getsantry[bot] commented 1 month ago

Routing to @getsentry/product-owners-performance for triage ⏲️

Dhrumil-Sentry commented 1 month ago

@peterHakio Are you basically asking to be able to query the number of N+1 span per tenant (which I assume is a tag on your spans?)

peterHakio commented 1 month ago

@peterHakio Are you basically asking to be able to query the number of N+1 span per tenant (which I assume is a tag on your spans?)

We send one query peer tenant, which is the bascialy the same query. Just performed with a different Postgres search_path. When we have 10 tenants this will show up as 10 identical queries each with there own span. Here is our setup for JS.Kysely


export async function invokeForAllCompanies<T extends Record<string, any>>(
  transactionLogic: (trx: Transaction<DB>) => Promise<T[]>,
  schemaNames?: string[],
) {
  const schemaNamesToQuery = schemaNames ?? (await getAllSchemaNames())
  return (
    await Promise.all(
      schemaNamesToQuery.map(async (schemaName) => {
        const kyselyDb = await getGenericDb()
          .transaction()
          .setIsolationLevel("serializable")
          .execute(async (trx) => {
            await sql`
              SET
                search_path = ${sql.raw(schemaName)};
            `.execute(trx)

            return trx
          })
        return transactionLogic(kyselyDb)
      }),
    )
  )
    .map((a, index) => a.map((b) => ({ ...b, orgId: schemaNamesToQuery[index] ?? "" })))
    .flat()
}

I know it is not something everyone is facing, so I though it would make sense to be able to adjust the threshold maching the number of companies we are having. (later we might need another solution). And it seemed meaningful to be able to adjust this threshold since 5 could seem alittle arbitrary and somebody could set it even lover in order to detect N + 1 in testing. Which is in many cases done on smaller problems.

getsantry[bot] commented 1 month ago

Routing to @getsentry/product-owners-insights for triage ⏲️

Dhrumil-Sentry commented 1 month ago

Thanks a lot for this feedback, We'll add it to our backlog but we don't have concrete timelines as of now cc @bcoe