frerich / clcache

A compiler cache for MSVC, much like ccache for gcc
Other
325 stars 83 forks source link

ThreadPoolExecutor / concurrent.futures.as_completed prevent cProfile from providing actual info #364

Open Artalus opened 5 years ago

Artalus commented 5 years ago

Basically, what I get after combining pstats:

This doesn't help me if I want to profile processSingleSource() which is called inside scheduleJobs. What I actually want to see:

Notice how instead of scheduleJobs->as_completed->wait I get actual detalization of functions called inside processSingleSource.

I managed it by replacing ThreadPoolExecutor construct with simple processSingleSource() call, asserting that len(sourceFiles) == len(objectFiles) == 1. In my use case (CMake + Ninja) there is always 1 source file per 1 compiler call, so using thread pool becomes a waste of syscalls, since it will always create 1 thread to do the job and return immediately after this thread is done.