conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.14k stars 970 forks source link

[feature] Include build/run environment variables in --format=json output #13258

Open sfackler opened 1 year ago

sfackler commented 1 year ago

What is your suggestion?

The json generator in Conan 1 included the environment variables set by the installed packages in the deps_env_info section. This generator was replaced by the --format=json flag in Conan 2, but its output does not include environment variables. They are present in the conanbuildenv/conanrunenv scripts, but those are more painful to properly parse out programmatically than JSON.

Have you read the CONTRIBUTING guide?

memsharded commented 1 year ago

Hi @sfackler

Thanks for your suggestion. I think adding self.dependencies["xxx"].buildenv_info and self.dependencies["xxx"].runenv_info to the serialized node representation of the graph that goes in the --format=json (I guess you meant conan install/create/graph --format=json output, did you?)

There are a couple of challenges to serialize that, because they represent multiple operations, like append, prepend, unset, etc, so it is not as easy as serializing a list of values, if we want to do it correctly, but lets give it a try.

sfackler commented 1 year ago

Yep, I'm interested in conan install's output.

While it'd definitely be good to be able to split this out granularly per-dependency with each operation specified, my personal use-case would be satisfied with a top-level "resolved" form of the environment variables like what ends up in the coanbuildenv/conanrunenv scripts. I think that was how the old deps_env_info output behaved as well.

It seems like something like this could work to represent the fully precise output:

"runenv_info": {
    "FOOBAR": [
        { "type": "append", "value": "fizzbuzz" }
    ]
}
memsharded commented 1 year ago

I think the aggregated representation would be difficult to represent in the graph json. The cpp_info is already serialized, but not aggregated, because the aggregation actually only happens now at VirtualBuildEnv and VirtualRunEnv generators, and this is even sometimes customized via generate() calls.

Aggregating things at the graph level with deps_cpp_info and deps_env_info was quite limiting and problematic, and it was deprecated in 1.X in favor of self.dependencies, and now finally removed in 2.0. So there is no longer any generic aggregation of upstream values in the graph. Furthermore, this would be quite costly to evaluate for every node in the graph, so this is not something that can be done easily. Computing only for the root node would be quite asymmetric.

I think dumping environment consumer side could be more a responsibility of a deployer, but it depends what you want to do. What usage are you trying to do with that information?

sfackler commented 1 year ago

I'm using Conan to manage native dependencies in a broader build system compiling Rust code, so I want to be able to run conan install, and then run e.g. cargo build or cargo test with the environment configured based off of the Conan buildenv. I essentially want to be able to do source conan_install_dir/conanbuild.sh && cargo test but with some more programmatic control and not having to add an extra layer of bash in the command invoation.

A deployer may make more sense for this than the --forma=json output.