Closed maitrey closed 3 months ago
Hi @maitrey
First a quick question. When you say "the model", is this the sources used to build the package binary from source? Does the model live in a git repository?
The model is a simulink model and yes it lives in a git repository.
Thanks for the info.
Then the access to the model best approach could be via the Git
scm model. Not adding the sources as exports_sources
, but just capturing the url+ commit information: https://docs.conan.io/2/examples/tools/scm/git/capture_scm/git_capture_scm.html#examples-tools-scm-git-capture
This way, the access to the source code is done via the standard access to the git repo. Those with access to the git repo can build from source, and see the source, those without access to the git repo will only be able to use the pre-compiled binaries exclusively, and if they try to build from source, it will fail telling them they don't have permissions to the model source repo.
Then, to provide different level of access to different customers to different packages, definitely the recommended way is setting up a different repository for each customer.
We donot want to give customers access to our repository. We wanted to manage it via artifactory as a distribution platform. :(
We donot want to give customers access to our repository. We wanted to manage it via artifactory as a distribution platform. :(
But this is what I am saying, setting up different Artifactory repositories, one for each customer, in Artifactory, with different permissions, and distribute via Artifactory. So you don't have to give access to your main Artifactory repositories to the customers, just to their own dedicated ones. I think there is something that I might not be understanding.
Then we are on the same page but question is in order to repackage the content belonging to each customer artifactory repo , I need to take care of mapping PREV with RREV incase if there are multiple versions of the same package. Also for traceability reasons, I need to additional information to the re-packaged content , is my understanding right?
is in order to repackage the content belonging to each customer artifactory repo
But no need to re-package, you can just copy packages from your own repository to the customer repository. If you have already the package and it is working fine, why creating another? There are still things that I don't understand.
I need to take care of mapping PREV with RREV incase if there are multiple versions of the same package. Also for traceability reasons, I need to additional information to the re-packaged content , is my understanding right?
Sorry, I don't know what you mean. Yes, you need to copy to your customers repositories the packages you want, the versions you want with the revisions you want, like every release process, you don't release every commit, you pick one and make a release out of it, same but with package binaries, isn't it?
Also for traceability reasons, I need to additional information to the re-packaged content , is my understanding right?
You mean adding properties in Artifactory? what "additional information"?
James you meant copy packages and I meant makinga recipe and downloading it and repackage like : https://docs.conan.io/1/creating_packages/existing_binaries.html And when I follow the above method it could be that I need to take care of adding the PREV and RREV. For example properties like git tags, which variant etc.. customer could subscribe to one variant but not the other ...
Do you mean that you use conan export-pkg
to create packages? Why is that re-packaging? the docs you linked are for creating a package for an existing binary, not for another Conan package. How is it related to distributing different content to different customers in server repositories? The way packages are created is not really related to how different server repositories will give access to different customers to the contents of the repo.
And when I follow the above method it could be that I need to take care of adding the PREV and RREV.
Adding the prev
and rrev
where? Those are built-in in Conan, no need to add them anywhere?
For example properties like git tags, which variant etc.. customer could subscribe to one variant but not the other ...
The git tag of the original source to build the package is intrinsic part of the recipe, typically the conandata.yml
, otherwise the package cannot be built at all. The variant
if it is the package binary identified by a package_id
, that is also built-in and core, no need to add it anywhere?
I am afraid that I don't understand neither the big picture nor the implementation, the naming/nomenclature used, the flows...
''' class HelloConan(ConanFile): name = "hello" version = "0.1" settings = "os", "compiler", "build_type", "arch"
def build(self):
if self.settings.os == "Windows" and self.settings.compiler == "Visual Studio":
url = ("https://<someurl>/downloads/hello_binary%s_%s.zip"
% (str(self.settings.compiler.version), str(self.settings.build_type)))
elif ...:
url = ...
else:
raise Exception("Binary does not exist for these settings")
tools.get(url)
def package(self):
self.copy("*") # assume package as-is, but you can also copy specific files or rearrange
def package_info(self): # still very useful for package consumers
self.cpp_info.libs = ["hello"]
''' Instead of the zip use the conan_package.tgz and copy required variant binary/header for each artifactory customer repo. So for example I have a package A in X repository , lets say in 3 variants. I would use the above recipe to download package A from X repository and copy the needed variant as package B in Y repository. In order to trace it, I need to add properties like which variant I added , version name would be same.
Instead of the zip use the conan_package.tgz and copy required variant binary/header for each artifactory customer repo.
So are you hardcoding the full Artifactory URL down to the conan_package.tgz
? I am afraid that this is not a recommended practice, you are abusing a lot of internal implementation details that can break at any time, and you also have to hardcode the package_id
, recipe_revision
and all of those things into the URL, instead of having the Conan recommended resolution, based on profiles, etc. Not to say that dependency resolution is none, so anything that will have any dependency like shared libraries or whatever will be extra complicated to manage.
So it seems we are not talking really about different repositories in the server side, but about creating new packages from existing ones while breaking the dependency chain, for final distribution. It seems the ticket you want to track is this one: https://github.com/conan-io/conan/issues/13171
To extract binaries from Conan there are other recommended and documented mechanisms, like deployers
, doing copy()
in generate()
, etc. I would recommend at this point a 2 step process:
conan install ... --deployer=mydeployer
to colllect binaries from a dependency graphconan export-pkg ...
to create the new binary from the collected binariesThose, provided the right and common profiles, will do the task without needing to worry about the details of the URLs or hardcoding anything. The deployer
can automate a lot of the data collection and generation of files for the final package.
Okay james. Sorry, it took quiet some iteration to exactly explain what I had in mind(for that atleast you will remmeber me :D , I hope). Thanks for your inputs. I will try this and get back to you.
To begin with , I tried to use full_deploy and direct_deploy. Both need a conanfile.txt. Is my assumption correct ?
The CCI imgui
recipe packages its different backends as source files, which sounds somewhat similar to your use case: https://github.com/conan-io/conan-center-index/blob/8305ea0ec1be297f700d10531682332339e8d14e/recipes/imgui/all/conanfile.py#L73-L79
An example of a recipe consuming these: https://github.com/valgur/cupoch/blob/9bd39ef5bb77f20583d085bfaeaff6e5830245d3/conanfile.py#L171-L186
To begin with , I tried to use full_deploy and direct_deploy. Both need a conanfile.txt. Is my assumption correct ?
No, deployers can also work with any conanfile
, also .py
, and even without conanfile at all, but using --requires=
argument.
The "vendor" feature has been merged in https://github.com/conan-io/conan/pull/16073, will be released in next Conan 2.4.
I think that complements well the alternatives cited above, and it could be a better fit than the deployers for actual later redistribution as a Conan package too, which seemed the original intention.
So this issue can be closed, please create new tickets against the new Conan 2.4 "vendor" feature if necessary (or about the other alternatives like deployers), many thanks!
What is your question?
Dear Conan Folks,
We have a use case for example a component has a simulink model , we generate code at build time and compile it , generate the executable package it in 3 different variants(3 different compilers). Now we have a requirement:
Have you read the CONTRIBUTING guide?