Hunter is slow

[v3c70r]

Hunter is a cool. But it fetches and compiles dependencies that exist in system libraries, which is unnecessary. I think it'll be faster to install dependencies using system package manager when possible.

[pretyman]

The reason it fetches the dependencies it is to produce a reproducible build, it is hard to debug a problem that could be originated by in a system library running version x.x.Y when the developer is has a different library version x.x.Z of that library in his system and is not able to reproduce it . If you ever been through this situation, you would find it is much faster to compile the libraries - happened something similar in Poppa's course under Linux, students had a version of a library installed in their system but Graphite was compiling with headers provided in its build directory, but linking dynamically to the system libraries, students spent days [more than 1 week?] to get their Assignment running due to a crash calling a function in some LU library - I don't remember the library to be honest.

However, @ruslo provides already pre-compiled libraries for certain toolchains [compilers] in a cache on the web. Personally I don't use online cache, so I don't know how to use it.

Ruslan, would you be so kind to point to instructions on how to use the cache to avoid the local compilation?

Would it be necessary to use the polly https://github.com/ruslo/polly toolchain library? Or can we do without?

I'm trying to push hunter for students to use in their practical assignments due to ease of the dependency management, but having them use polly could make the process non-trivial for some students.

Thanks

[ruslo]

Personally I don't use online cache, so I don't know how to use it

There is no any special moves about it. Hunter do check it automatically (except when you explicitly forbid it by HUNTER_USE_CACHE_SERVERS=NO). So I guess your environment (compiler) is simply doesn't match Travis CI environment.

Ruslan, would you be so kind to point to instructions on how to use the cache to avoid the local compilation?

It's easy for Visual Studio and Xcode - you just have to install right version (see https://github.com/ruslo/hunter/wiki/Uploading-binaries-to-server#environment-notes and https://www.appveyor.com/docs/installed-software/#visual-studio). It will be trickier for GCC. I do usually run docker if want to repeat something. Also you can start custom repository for saving binaries and upload your environment specific binaries. I got a few ideas for future improvements.

Would it be necessary to use the polly https://github.com/ruslo/polly toolchain library? Or can we do without?

No. If you're not using toolchains (e.g. just -G Visual Studio or -G Xcode) you don't have to set them.

but having them use polly could make the process non-trivial for some students

Polly is a set of CMake toolchains. It's easier than writing your own. If you don't want to use build.py you don't have to. When you run build.py you can see what CMake commands executed under the hood. Usually it's just -DCMAKE_TOOLCHAIN_FILE=/path/to/polly/toolchain.cmake.

[pretyman]

Thanks Ruslan!

Tsing, could you tell us what compiler you are using? I will see if I can upload the binaries for your compiler and I can check if local compilation was avoided.

[pretyman]

Usually it's just -DCMAKE_TOOLCHAIN_FILE=/path/to/polly/toolchain.cmake.

Oh, you have no idea :-) I once asked a student to open the Terminal and he answered: "termi-what? I never downloaded that..." It was an Apple-boy, so that kind of explains it

[v3c70r]

Sorry for the late reply. I'm using g++-5.4.0 on Ubuntu. I have successfully compiled code. Just feel a little bit unnecessary to download and compile the dependencies while I've already have all of the libraries in my system : ).

[ruslo]

The reason it fetches the dependencies it is to produce a reproducible build, it is hard to debug a problem that could be originated by in a system library running version x.x.Y when the developer is has a different library version x.x.Z of that library in his system and is not able to reproduce it

Kind of but not the only one. I've added new page to documentation: https://docs.hunter.sh/en/latest/faq/interaction-with-other-package-managers.html

I will see if I can upload the binaries for your compiler and I can check if local compilation was avoided

If you work in local network you can just share folder with binaries, for GitHub uploading see this description: https://docs.hunter.sh/en/latest/faq/why-binaries-from-server-not-used.html#uploading-to-server

[pretyman]

Hi Tsing,

I have not yet gotten around to upload the binaries for your compiler, but I think it would make sense to have them in the official hunter repository since it is the default compiler of a major Linux distribution.

Just feel a little bit unnecessary to download and compile the dependencies while I've already have all of the libraries in my system : ).

If you are referring to the Concordia network disk space, I completely agree with you, since it is quite small. But if you are referring to your local machine, I would disagree because I had old apps which stopped compiling with a dist-upgrade in ubuntu, and when compilation errors were fixed it started behaving differently, for me it was just not worth it.

You are free to disagree, but personally I prefer to have total control of my library dependencies and leave the system package manager libraries to the system apps.

What you could do is recommend this project to the clueless students, as there is relatively little effort setting up the project with hunter. Even when I put the binaries for your compiler up, very likely students will have different compilers and will have to end up waiting for the compilation time.

What would be interesting is to know the compiler version of the lab computers so these could be uploaded (somewhere else, if not in official repository) as well, reducing the space needed for the sources.

Best regards!

[ruslo]

in the official hunter repository since it is the default compiler of a major Linux distribution

If you find a painless way to install newest version of GCC on sudo:false machines I can set this as default. There are a lot of packages that need more c++11 features than current 4.8 version do provide.

I would disagree because I had old apps which stopped compiling with a dist-upgrade in ubuntu, and when compilation errors were fixed it started behaving differently, for me it was just not worth it

Good argument, I forgot about it. Will add to the docs, thanks :)

very likely students will have different compilers

What would be interesting is to know the compiler version of the lab computers

Just for your info by "I got a few ideas for future improvements" I mean this feature: https://github.com/ruslo/hunter/issues/495 (i.e. Hunter can control version of compiler so we don't have to worry about user's environment).

[v3c70r]

Thank you for your reply. I agree with you that using Hunter solves inconsistency of dependencies and I will use it in my new projects. Unfortunately I'm not a student in the class. But I believe this tool makes most of the students' lives easier :-).

[pretyman]

I thought you were being Teaching Assistant for COMP371 hence you were looking at this repository :-P

I'll leave this issue open until I get the binaries up for GCC 5.4.0, or else I'll forget

[v3c70r]

I was TA for COMP477 actually. BTW I believe Concordia lab is using GCC 4.7.2 on their Linux.