[feature] Dynamic versioning in workspaces

[aander80]

What is your suggestion?

Hello!

If using dynamic versioning (for instance using dunamai then I imagine it could be a bit confusing and perhaps tedious to work with workspace where you are "locking" the version of an editable in conanws.yml. Every commit will produce a new version, and the version in conanws.yml will very quickly be out of sync with the actual version. The built binaries, if including metadata for its dependencies, will have incorrect information.

As an example of this, please have a look at my workspace test repo. There are instructions in README.md for setting it up. By setting up the repo and running ./example_scripts/version_mismatch.sh you will get different versions of liba, libb and app although they are the same in conanws.yml.

As a potential solution to this, I am thinking that you can specify name/* instead of name/version in conanws.yml, and conan workspace commands will dynamically and temporarily check the version of those packages, add them as editables during the current command, run the command, and at the end remove them as editables -- alternatively just updating existing editables at the start of the conan workspace command. Perhaps this is usable/possible with a conanws.py, although I have not been able to get it to work or understand how it is supposed to work.

This is also elaborated upon a bit in my test repo under the Version diff section.

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Thanks very much for your feedback @aander80

I have done a quick proof of concept in a new test in https://github.com/conan-io/conan/pull/17272/commits/a377ab72d88d61f166d20f8c296af64f5b503241

See the workspace:

import os
name = "myws"

workspace_folder = os.path.dirname(os.path.abspath(__file__))

def editables():
    result = {}
    for f in os.listdir(workspace_folder):
        if os.path.isdir(f):
            name = open(os.path.join(f, "name.txt")).read().strip()
            version = open(os.path.join(f, "version.txt")).read().strip()
            result[f"{name}/{version}"] = {"path": f}
    return result

This assumes that name.txt and version.txt encode the dynamic information of the package. I am happy to see that the current proposal for workspaces definition seems to be flexible.

Now, it is true that we might want to be able to provide some helpers or API to give access to some higher level builtin capabilities, like leveraging the recipe set_version()/set_name() to extract the data. This might require a bit more of thinking that this looks like a good start at least.

[memsharded]

I managed to add in https://github.com/conan-io/conan/pull/17272/commits/6c023e362c3aaa23e22dbb1a86f67e91d82ad9a2 a small API that allows loading Conanfiles for automatic invocation of set_name()/set_version(). Just a draft to discuss with the team ideas.

[aander80]

That looks very promising! I tried using it but got error NameError: name 'workspace_api' is not defined when running conan workspace info. I don't know if it is expected to work quite yet outside of tests (or even the tests themselves) but I pushed a change in my test repo where I added conanws.py if you want to have a closer look. Otherwise I look forward to testing something once you have had a discussion with the team and have something ready to test! :)

[memsharded]

I am not getting that error, but using your project I got another error:

Error in Conan initialization: conanfile.py: Error in set_version() method, line 23
        self.run("dunamai from git --style semver --bump", cwd=self.recipe_folder, stdout=stdout)
        AttributeError: 'NoneType' object has no attribute 'cmd_wrapper'

This is expected, the current implementation do not provide the cmd_wrapper, doesn't allow self.run() inside the set_version(). We would need a more complex construction of the conanfile to be able to support this.

[memsharded]

And for real usage the code would be something like:

for f in os.listdir(workspace_api.folder):
        if os.path.isdir(f) and os.path.exists(os.path.join(f, "conanfile.py")):

Checking that the conanfile.py actually exists, to avoid error for other folders like .git that would fail otherwise.