derekbruening
commented
4 months ago For analyzing core-sharded-on-disk files: my approach is to add a new filetype indicating core-sharded, and have the scheduler read every input to find its filetype record so the analyzer can check the filetype at init time.
That all works fine for typical runs: but it breaks many specific modes, from the readahead:
Breaks online: scheduler init just blocks! => new option read_inputs_in_init, turned off for ipc readers.
Breaks inv checks b/c input record ord is too far => using output ord until 1st instr.
Breaks unit tests which have no filetype records => stopping readahead at pagesize marker.
Breaks replay-as-traced b/c input record ord is too far => disabling read-ahead for as-traced.
Breaks record filter legacy null check b/c legacy trace has no filetype and it reads the timestamp which means the stream's last-timestamp is now > stop-timestamp and so the resulting output trace has the filter-end marker 1st thing!
$ zcat ../src/clients/drcachesim/tests/drmemtrace.legacy-for-record-filter.x64.tracedir/drmemtrace.threadsig.10506.7343.trace.gz | od -A x -t x2 -w12 | awk '{printf "%s | %s %s %s%s%s%s\n", $1, $2, $3, $7, $6, $5, $4}' | head
000000 | 0019 0000 0000000000000001
00000c | 0016 0004 000000000000290a
000018 | 0018 0004 000000000000290a
000024 | 001c 0002 002ede0c00c77676
000030 | 001c 0003 0000000000000001=>
$ zcat record_filter_tests_tmp_output/null_filter/drmemtrace.threadsig.10511.8961.trace.gz | od -A x -t x2 -w12 | awk '{printf "%s | %s %s %s%s%s%s\n", $1, $2, $3, $7, $6, $5, $4}' | head
000000 | 001c 0017 0000000000000000
00000c | 0019 0000 0000000000000001
000018 | 0016 0004 000000000000290f
We'd like to be able to write out new core-sharded drmemtrace files for the result of software thread interleaving on virtual cores from the trace scheduler. The plan is to run the record_filter tool on the analysis framework (built for record operation) on top of the scheduler with the desired schedule config, and add support to generate new zipfile chunks to the record_filter tool (which we want anyway to be able to remove instructions from thread-sharded files: part of #6593).