[question] MsBuild tools.microsoft.msbuild:max_cpu_count

[fellinga]

Hello,

we are using conan in CI/CD and we were wondering if we can utilize all of the CPUs in a MsBuild build when calling "conan build .". We already know that we can mimic /maxcpucount:2 with tools.microsoft.msbuild:max_cpu_count=2 in our default.conf file, but how can we set /maxcpucount without any arguments? Because setting /maxcpucount would use all available CPUs by default... "If you include the -maxcpucount switch without specifying a value, MSBuild will use up to the number of processors on the computer." (source).

Another question: is there a better way to set tools.microsoft.msbuild:max_cpu_count=2 in CI/CD besides manually appending it to the default.conf?

Thanks in advance!

Have you read the CONTRIBUTING guide?

• [ ] I've read the CONTRIBUTING guide

[memsharded]

Hi @fellinga

Thanks for your question.

We already know that we can mimic /maxcpucount:2 with tools.microsoft.msbuild:max_cpu_count=2 in our default.conf file, but how can we set /maxcpucount without any arguments?

It is not clear if you are using MSBuild (VS native project) or via CMake. In any case, note that you can also use the tools.build:jobs conf, and it will define the <ProcessorNumber>{}</ProcessorNumber> in MSBuildToolchain or to /MP flags in CMakeToolchain. This is equally or even more important for multi-processor compilation parallelism than max_cpu_count.

Because setting /maxcpucount would use all available CPUs by default.

Not really, if the solution doesn't have multiple projects to build in parallel. For intra-project compilation parallelism, you need the above conf.

 njobs = conanfile.conf.get("tools.build:jobs",
                               default=_cpu_count(),
                               check_type=int)

This is how the default number of jobs is computed, by default it is already the cpu_count, and that is used for the compilation parallelism

I see it seems there is a small bug in the max_cpu_count management for CMake, but need clarification if you are using it or not.

[memsharded]

One thing more: the max_cpu_count is not set by default to cpu_count, because our experience is that having both /MP and /m:x flags at the same time to the number of cpus, is counter productive, it can easily become slower because having too many concurrent process. I wouldn't recommend setting max_cpu_count to the number of cpu-counts, at least without deactivating or reducing first the tools.build:jobs

[fellinga]

Hi @memsharded,

thanks for answering so quickly! We use MSBuild with VisualStudio and do not use CMake at all.

I'll try to explain my issue a little better, sorry if it was not clear enough: Currently we run "MSBuild.exe mySolution.sln /maxcpucount /p:Configuration=Release in gitlab-ci and are quite happy with it but want to change this command to "conan install . " so that we can use a common pipeline for all projects. Therefore we added:

def build(self): msbuild = MSBuild(self) msbuild.build("mySolution.sln")

to our conanfiles. Now when i call "conan install . " it generates "MSBuild.exe mySolution.sln /p:Configuration=Release" which is almost exactly what we want besides the missing maxcpucount argument. I do not want to specify a hardcoded number of cores since my CI runners do not all have the same amount of CPUs. How can we solve this issue?

Thanks!

[memsharded]

Understood. Yes, I think this specific use case is not covered by the current conf value, but I think it might be doable to take it into account, i will reopen (it was closed by by #17292 and will give it a try.

Thanks for the feedback!

[memsharded]

Proposing https://github.com/conan-io/conan/pull/17301, and the maxcpucount=0 value to mean "all cores", I think it is better than inventing some other arbitrary string.

[memsharded]

https://github.com/conan-io/conan/pull/17301 was merged, so using maxcpucount=0 Conan conf will mean => /m which means all cores. It will be in next Conan 2.10 release. Thanks for the feedback!