dennisklein
commented
1 year ago Sorry if this is a stupid question.
not at all!
why do we not use conan or vcpkg for FairRoot?
Short answer: lack of manpower
Long answer: Too long to write everything here now in one post in all glory details, just a few comments:
- We have been looking at/evaluating several user-level package managers in the past (conan included, vcpkg not at the time). Back then (~2 years ago) spack looked most promising although still having its issues till today, which is why we did not yet migrate FairSoft to it in production. Note, that other package management solutions have not been idle in that time and our conclusion may be very well outdated, but that will always be a challenge.
- However, spack has been adopted in FAIR/GSI's computing farm environment and used with success, so I am currently working on updating spack FairSoft (we first will use it in a smaller context within one experiment context and upon success maybe switch it for all FairSoft users this year still). So, from our side, it is very unlikely that we will invest time in another package manager any time soon. But see my conclusion further down.
- "The devil is in the details": I don't know your experience with the dependency tree we try to "package" in FairSoft, but once you actually do it, you will encounter tons of obstacles - some projects are not particularly "packagable" or "packaging-friendly" which means you have to carry around package-level patchsets and pay the maintenance price for those. Over the last years, we have made several contributions to various of the projects to improve that situation, but there are still open points.
- It is very hard to combine all use cases with any one of the package management solutions:
- source distribution is usually the easiest to achieve
- binary distribution models of many of those "popular" package managers are mediocre at best
- user-level pkg managers have usually no integration with the system package manager: on gnu/linux platforms you already have strongly managed software, I have yet to find a user-level pkg manager that does integrate well with it.
- then you have mac or windows folks, who don't have system software managers and are not even aware of this issue (and also do not care)
- none of the user-level package managers have good models for pinning software dependency trees (this is one point where spack is good at but which comes also with quite some complexity)
- at the end of the day we are doing science and one important pillar of science is reproducibility, having a pkg mgmt solution which is only focussing on the latest versions of everything is only part of the solution we are looking for.
- some users just care about a fast and easy way to get the FairRoot up and running - but then - over time - the developer use cases come flying in, they then want to be able to modify that software stack in all kinds of ways, but easy and fast.. and all with the same pkg manager software :wink:
In the end this is all solvable with enough manpower, which we do not have a lot of. But I see it this way: There is no technical reason, why FairRoot and its dependencies cannot be packaged in many packaging solutions/communities in parallel. If you would want to contribute a (maintained :wink:) FairRoot package in vcpkg or conan - I would be happy to link to your packages in our docs for all our users to benefit if you wish so! (and regarding packaging-friendliness of FairRoot and co: If you encounter any obstacles, you are welcome to report them, we want to fix those!)
Also FairSoft itself is open: You are welcome to submit pull requests to help with our current packaging efforts in this repo (cmake legacy script or the soon-to-be spack distro). We do not yet have a lot of outside (our group) contributions, so FairSoft is lacking polished contribution docs, but if you are interested, we can easily connect and figure out the details/questions.
They are the 2 most popular package managers among C++ projects in industries and they also integrate seamlessly with the cmake build system. I know spack is designed for the HPC system. However, we still have a lot of users who also need to run FairRoot in their own machines.
Hmm, what exactly makes spack HPC system specific? What prevents you from using it locally on your laptop?
fuhlig1
Hi,
Sorry if this is a stupid question. But why do we not use conan or vcpkg for FairRoot? They are the 2 most popular package managers among C++ projects in industries and they also integrate seamlessly with the cmake build system. I know spack is designed for the HPC system. However, we still have a lot of users who also need to run FairRoot in their own machines.