search

djkoloski / rust_serialization_benchmark

Benchmarks for rust serialization frameworks
514 stars 48 forks source link

bench action: no codegen, but run lscpu #74

Closed mumbleskates closed 3 weeks ago

mumbleskates commented 3 weeks ago

it is no longer necessary to fetch codegen tools, as they are no longer needed by default since 1a92464a3d39be45cf5b96af55f211991f70359e

however, at the level these libraries often operate, the runtime CPU matters. github actions' hosting infrastructure is pretty mysterious, so maybe we can pull the veil back a little bit

(i'm seeing regressions in the bilrost "prepend" path for a couple of these benchmarks; this is nothing new, it's actually amazing how much of the modern software & hardware stack seemingly revolves around the assumption that nobody does this)

djkoloski commented 3 weeks ago

I think this is a good change to make. We should include this in our benchmark data as a metadata field and use it when generating the README.

mumbleskates commented 3 weeks ago

are there more supporting changes to make here to format that into the readme? i have not delved into the tools folder much at all

djkoloski commented 3 weeks ago

You're welcome to make those changes if you want, but I can also just make them into an issue for future work.

Generally, the bencher crate orchestrates benching and gathering all the data. It calls out to the parser crate to turn the raw bench output into JSON, then the formatter crate to generate the README using that JSON. The JSON format is specified by the schema crate, which is shared between the parser and formatter.

mumbleskates commented 3 weeks ago

okay! that's a great guideline, thanks; i'll see if i can just add a field we pass through and try to pipe some lscpu output into it.

Do you feel this change can be merged independently of that? I'm ambivalent. I'm mildly curious what these machines even are but only mildly; i think i can infer that they are Xeons purely based on the comparative performance of my code (yeah, really lol; something about the µcode generation is reliably significantly different between the two; AMD cpus are almost always faster in prepend, but i have not been able to get quite the same wins out of Intel yet). It'd be cool to have that in the actions logs but maybe it's not worth making this change until it actually shows up in the output.

djkoloski commented 3 weeks ago

Yeah we can merge this right now. Just mark it as no longer a draft and I'd be happy to merge it.

mumbleskates commented 3 weeks ago

marked