[question] How to use source_buildenv

[Drllap]

What is your question?

Hi! I'm trying to create a conan package that generates a library with open API. I have added

tool_requires = "openapi-generator/7.3.0"

to my recipe.

If I add

    def build(self):
        self.run("where openapi-generator")

this works.

But then I thought maybe it makes more sense to generate the code in the source method, so I tried this:

    def source(self):
        self.run("where openapi-generator")

but this doesn't work, even if I add source_buildenv = True

The first question is, where would generating the code make the most sense? In build, source, or generate? What am I missing in regards with source_buildenv, why isn't the environment the same in build and source?

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @Drllap

The first question is, where would generating the code make the most sense? In build, source, or generate?

I'd say that generated code is somehow different from the user source, and might belong better to generate() method, than to source() method.

The main reason is that source() must be invariant, and there are lots of chances that generated code is not invariant, and it can change (otherwise why not adding it to the original source and forget about the code generation?). For example changing the version of the tool_requires might generate different code. This would be handled a bit better in the generate() that generates code by default inside temporary "build" folders, than operating in the source_folder.

Also, the temporary build folders like "build" are typically already .gitignored, so no need to track those and make sure that the generated code doesn't slip as changes when committing changes to source.

What am I missing in regards with source_buildenv, why isn't the environment the same in build and source?

It seems there is something missing in the docs. The source_buildenv only works when building in the local cache. The conan source command is not expanding dependencies or injecting tool_requires, the local flow depends on:

• conan install
• Activate the generated conanbuild script (.bat/.sh)
• conan source (will find things, because activated by the previous step)

[Drllap]

@memsharded This doesn't work either:

    def generate(self):
        print("GENERATE")
        self.run("where openapi-generator")

It appears that the conanbuild.bat/conanrun.bat aren't generated until after the generated method returns

[memsharded]

The generate() should do a explicit instantiation of the VirtualBuildEnv and generate the files before the self.run(), can you please try that?

[Drllap]

Not sure what you want me to test but, I tried this:

    def generate(self):
        print("GENERATE")
        ms = VirtualBuildEnv(self)
        ms.generate()
        #  self.run("where openapi-generator")
        print("END GENERATE")

Then I run conan install . and get this:

======== Installing packages ========
openjdk/21.0.1: Already installed! (1 of 2)
openjdk/21.0.1: Creating JAVA_HOME environment variable with : C:\Users\palli\.conan2\p\b\openjdc66756723c25\p
openapi-generator/7.3.0: Already installed! (2 of 2)
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'env_info' used in: openjdk/21.0.1, openapi-generator/7.3.0

======== Finalizing install (deploy, generators) ========
conanfile.py (open-api-gen/1.0.0): Calling generate()
conanfile.py (open-api-gen/1.0.0): Generators folder: D:\dev\Norbit\conan\generat-fromp-open-api-test\build\generators
GENERATE
END GENERATE
conanfile.py (open-api-gen/1.0.0): Generating aggregated env files
conanfile.py (open-api-gen/1.0.0): Generated aggregated env files: ['conanbuild.bat', 'conanrun.bat']
Install finished successfully

If I uncomment the self.run line I get this:

======== Installing packages ========
openjdk/21.0.1: Already installed! (1 of 2)
openjdk/21.0.1: Creating JAVA_HOME environment variable with : C:\Users\palli\.conan2\p\b\openjdc66756723c25\p
openapi-generator/7.3.0: Already installed! (2 of 2)
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'env_info' used in: openapi-generator/7.3.0, openjdk/21.0.1

======== Finalizing install (deploy, generators) ========
conanfile.py (open-api-gen/1.0.0): Calling generate()
conanfile.py (open-api-gen/1.0.0): Generators folder: D:\dev\Norbit\conan\generat-fromp-open-api-test\build\generators
GENERATE
conanfile.py (open-api-gen/1.0.0): RUN: where openapi-generator
INFO: Could not find files for the given pattern(s).

ERROR: conanfile.py (open-api-gen/1.0.0): Error in generate() method, line 52
        self.run("where openapi-generator")
        ConanException: Error 1 while executing

so the *.bat files aren't generated. Maybe this has something to do with the deprecation warning?

[memsharded]

You are right, there is a small detail that when doing the job in generate(), the conanbuild default environment aggregator is not there. I have been able to write a successful test as:

    def test_generate_buildenv(self, client):
        c = client
        pkg = textwrap.dedent("""
            from conan import ConanFile
            from conan.tools.env import VirtualBuildEnv
            import platform

            class Pkg(ConanFile):
                name = "pkg"
                version = "0.1"
                tool_requires = "tool/0.1"

                def generate(self):
                    VirtualBuildEnv(self).generate()
                    cmd = "mytool.bat" if platform.system() == "Windows" else "mytool.sh"
                    self.run(cmd, env="conanbuildenv")
            """)
        c.save({"conanfile.py": pkg})
        c.run("create . -vv")
        assert "MY-TOOL! tool/0.1" in c.out

        c.run("install .")  # to generate conanbuild script first, so it is available
        assert "MY-TOOL! tool/0.1" in c.out

The solution is to do the env="conanbuildenv" to specify the generated environment file.