search

Trepan-Debuggers / remake

Enhanced GNU Make - tracing, error reporting, debugging, profiling and more
http://bashdb.sf.net/remake
GNU General Public License v3.0
796 stars 75 forks source link

Possible Enhacement Idea #126

Open amishscientist opened 3 years ago

amishscientist commented 3 years ago

When I first look at a build to profile I normally find the box with the most cores on it and run /usr/bin/time -v make -j NumOfCPU

I then look at Percent of CPU this job got: in the output of time from the build command. If it is close to the number of cores I gave the make command then I know I can probably look into things like dist_cc or icecreamcc to get faster builds, but if I gave that thing 24 cores and it only got a 800% then things like dist_cc or icecreamcc aren't going to do any good because I couldn't even uses all the cores on my current box all the way.

It might be nice if remake could give us some data or stats on how busy the make job queue is during the build. Average Job Queue length, How much time it spent with a full job queue and maybe if it never hit the max number of jobs in the queue.

rocky commented 3 years ago

This is open-source software, so feel free to experiment and implement what you suggest.

amishscientist commented 3 years ago

Any idea where a good place to start would be?

rocky commented 3 years ago

Here is a guess, first try to do this totally externally. If you can't, then describe what thing internally would make this work.

In my opinion being able to describe in pseudo code how things would work would help. Pretend you are the computer you see x so you do y, and so on.

But if you are a low-level kind of guy then src/job.c is where all "jobs" or external commands that get run is kicked off from.

Arrowbox commented 3 years ago

If you check out the json_output branch, I also capture the -j value. I've been working on dumping the same information about parallelness. My current approach is to sum all the time taken running recipes and divide that by total wall time. If it perfectly scaled, then recipe/total = number of cores.

martencassel commented 2 years ago

Here is a guess, first try to do this totally externally. If you can't, then describe what thing internally would make this work.

In my opinion being able to describe in pseudo code how things would work would help. Pretend you are the computer you see x so you do y, and so on.

But if you are a low-level kind of guy then src/job.c is where all "jobs" or external commands that get run is kicked off from.

rocky tip is an epic one.. on the mindset to have when hacking up code.... very helpful to me.