Profiling is missing designation of job.

[alecazam]

So we are converting the json to Catapult/Perfetto format to view it in Perfetto. But this system really needs a tid, and if not, then all timings must be a subrange of the parent. It tries to stort the start/end times into a plausible hierarchy, but often fails, and really fails on the profiling output from remake. Moved from 'X' type events, to 'B' and 'E' and this improved things a little. But really each job should be on it's own id.

So my proposal is that each json element should designate a job id. This would correspond with whatever job listens for the output of the process (and threads). This can just be the index of the array of those jobs. So it's 0 ...11 for a -j 12 run. This is guaranteed to be non-overlapping. And if premake honors the -j count setting of make, then there should be a limited number of these per profile. Note if there are more jobs than cores, and each job can run multiple threads, there will be overlap in the timings, but that's to be expected. We typically run with core or core-1 count on make.

The profiles should not just be a jumble of start/end times. That doesn't make for a flamegraph, or anything interesting to have tons of overlapping or truncated times (the Perfetto sub-range limitation).

[alecazam]

Also the json outputing a time for the "deps" and "recipe" fields seems odd. I'll have to look through the source to understand what this is trying to convey, but seems to be set on top level scopes I think. The remaining are set to null like this one.

  {
            "name":"../foo.lua",
            "file":"foo.d",
            "line":812,
            "start":1713480824.076899000,
            "deps":null,    <- these are times with no correspondence some times
            "recipe":null,  <- these are times with no correspondence some times
            "end":1713480824.076899000,
            "depends":[
            ]
        },

[rocky]

Thanks for the information. If you are up for it, please put in a Pull Request (PR). I am out of town for a few weeks and will not be able to look at this for possibly a month since there are already many other backlogged I have when I get back.

[alecazam]

Never every looked at the code. Just a first time user of remake trying to look into the profiling output. I do have a profiling tool for macOS called kram-profile, and it can display Perfetto dumps.

So far we just use Python to unoverlap the tracks, and assigns a tid to the json converted to Perfetto json. I can hopefully share that python file, but it's imprecise to try to figure out what tracks are what from a job system. So we run with -j 12, but end up with like 17 tracks.

[alecazam]

We ended up adopting ninja. It has built-in profiling per build task, and it schedules the content tasks more closely across the cores.