search

pfultz2 / cget

C++ package retrieval
http://cget.readthedocs.io
Other
453 stars 27 forks source link

handle projects with submodules #58

Open Glavnokoman opened 6 years ago

Glavnokoman commented 6 years ago

should be easy with default clone command updated to 'git clone --resurse-submodules ...'. If there are no submodules it generates no error, if there are - does the right thing.

pfultz2 commented 6 years ago

Currently, cget doesn't support cloning with git at all. We could provide a filter to clone with git, like what pip does, with something like git+<url>. Although, in general, its not really as useful in cget since there is no -e edit mode.

Glavnokoman commented 6 years ago

It could be added at least for github which is currently supported.

pfultz2 commented 6 years ago

It uses the source tarballs from github, there is no need to use a VCS since the files won't ever be modified. Also, using git from github can be slower, and the source tarballs are preferred by the github team(since its easier for their servers to cache).

Glavnokoman commented 6 years ago

I see. What would be the least ugly and most functional workaroung then?

pfultz2 commented 6 years ago

I see. What would be the least ugly and most functional workaroung then?

I dont know details about the project so maybe I am missing something, but the best way is to have cget manage the dependencies instead of git. So if a superproject consist of project A, B, and C, instead of cloning the superproject, just install project A, B, and C(in the correct order) instead.

If project A, B, and C, all have circular dependencies, then its best to either provide source tarballs manually(this is what boost does) or use git subtree.

Glavnokoman commented 6 years ago

the best way is to have cget manage the dependencies instead of git

I realize that. But there are multitude of projects doing just the opposite. Like KhronosGroup/Vulkan-Loader for one. If I depend on those projects I can not simply put them in requirements.txt or cget one-liner into install_dependencies.sh. Currently I just skip it but that is not so nice, and .travis part still requires some workarounds...

pfultz2 commented 6 years ago

Projects need to provide downloadable sources, this is very common practice for projects with open-source distributions(all projects in Debian can be downloaded as source tarballs and built).

As a workaround for projects that dont provide downloadable sources, you will need to get the sources using whatever tool the project requires to get the sources and then point cget to that directory.

Another option would be to create a recipe that would go through those steps as well. This will let you keep the one-liner in your install_dependencies.sh file. Unfortunately, you won't be able to take advantage of cget's download cache.

The bigger issue is that an open source project should always provide sources that can be downloaded through a web browser(or tools such as wget or curl). Protocols such as http and https are readily available with any tool. You should contact authors of projects that do not provide downloadable sources before doing any workarounds.