It should be possible to install `logfire` without automatically enabling `logfire-api` use

[samuelcolvin]

Description

The ideal would be I guess

logfire.enable_logfire_api('module.path')

But that would be a breaking change. Perhaps we do this via an env variable, so

https://github.com/pydantic/logfire/blob/cc79cf448a71f8c7a0f57deacaf3676e603168c4/logfire-api/logfire_api/__init__.py#L9-L14

becomes some thing like

_real_logfire_enabled = False
if not os.getenv('LOGFIRE_API_DISABLED'):
    try:
        logfire_module = importlib.import_module('logfire')
        sys.modules[__name__] = logfire_module
        _real_logfire_enabled = True
    except ImportError:
        pass

if not _real_logfire_enabled:
    if not TYPE_CHECKING:  # pragma: no branch
        ...

Either way, I need a way to have logfire installed without activating logfire-api calls.

[samuelcolvin]

I still think this would come in useful, but it's no longer urgent.

[Kludex]

I don't understand the reason for it. The idea was exactly to use logfire when installed.

[alexmojaki]

I am also confused.

A library using logfire-api typically won't call configure because that's up to the application. So the application isn't forced to actually use logfire just because it's installed, they just need to set LOGFIRE_IGNORE_NO_CONFIG.

[samuelcolvin]

For the same reason you can opt to enable logging for just some of the modules that have logging integrated - you might want use logfire but not collect data from all code which has logfire-api integrated.

I also think it's unfortunate if installing logfire causes code which was previously be behaving in one way to suddenly "break" - e.g. raise an error because you haven't called logfire.configure().

Datapoints that support the idea that you should be able to switch logfire-api off for certain libraries/modules:

• rust's tracing doesn't suddenly start sending data if you install tracing, you need to enable it for each package
• (as above) you can control which modules you collect logs from
• otel library instrumentation doesn't automatically start collecting data just because you have the relevant package installed

I think it's pretty uncontraverial that you should be able to decide which modules you want to receive traces/spans from.

[alexmojaki]

I also think it's unfortunate if installing logfire causes code which was previously be behaving in one way to suddenly "break" - e.g. raise an error because you haven't called logfire.configure().

It emits a warning if you don't configure, not an error. And it doesn't send/collect data. We can maybe prevent the warning specifically for logfire_api.configure() since that's not the case the warning was intended for.

In general you can control where logs come from to some extent by configuring head sampling based on code.filepath. It's very clunky and has caveats but I don't think Python OTEL offers anything better. We definitely want to improve that, including sampling based on instrumentation scope: https://pydanticlogfire.slack.com/archives/C06EDRBSAH3/p1727892858026169

[samuelcolvin]

What if I have package foo that uses logfire-api, but I don't want to send that data to Logfire?

Currently there's no way to set that AFAIK, as per my examples above, all other simpler cases allow this control, some with opt-in and some with opt-out.

We should allow opt-out of sending data from a particular module by not enabling logfire-api for that module.

[alexmojaki]

There's definitely no reasonably simple way to opt out of specific modules. But LOGFIRE_API_DISABLED wouldn't help with that.

Ideally packages would provide a method like instrument_my_package which accepts a logfire instance so that users can not only opt-in but use different configurations (e.g. different sampling) for each package.

[samuelcolvin]

maybe logfire-api (and perhaps logfire too) should provide something akin to std library loggings "logger" which you could use to enable and disable specific instrumentations.

To avoid adding extra concepts, perhaps we could allow you to mute/disable specific scopes, so if a package instruments itself with

import logfire_api
...
logfire = logfire_api.Logfire(otel_scope='potato')

We provide a convenient way for logfire calls to do nothing when the otel scope is potato, something like

logfire.mute_otel_scopes('potato')

obviously this isn't just dropping spans during the batch exporter since that would lead to missing parents - we need to make it so calls to the logfire returned by logfire_api.Logfire(otel_scope='potato') never creates a span.

[alexmojaki]

The generalization of this has already been requested, which is to sample x% of traces from a scope: https://pydanticlogfire.slack.com/archives/C06EDRBSAH3/p1727892858026169