Go perf

[CrimsonVoid]

k6 run -u 50 --duration 30s loadTest.js	lang (branch)	http_reqs/sec	http_req_duration avg
go (main)	2,156 /s	22.67 ms
go (perf)	2,924 /s	16.78 ms
rust (main)	2,310 /s	21.13 ms

k6 run -u 50 --duration 30s loadTestSmallResponses.js	lang (branch)	http_reqs/sec	http_req_duration avg
go (main)	7,728 /s	6.35 ms
go (perf)	16,244 /s	2.98 ms
rust (main)	13,387 /s	3.64 ms

csv load times	filename	go (main)	go (perf)	rust (main)
stop_times.txt	1.15 s	835 ms	677 ms
trips.txt	31 ms	30 ms	24 ms

[CrimsonVoid]

A little summary for some of the more notable commits:

• readCsv perf ff6f04d17955cf1e210a1ce06022e287a297ec40 - Setting ReuseRecord was the biggest win here followed up by streaming csv lines. Also refactored csv file parsing out into a helper func (parseCsvFile) used by getStopTimes and getTrips
• avoid copies acfe03246a69e150da3c2b8e26c415dd96d4cdbd - Reuse Trip and StopTime structs in TripResponse instead of copying. This was another fairly decent win
• store struct ptrs instead of indices efa6054e80b3528418dc40bf313a8220e3f11bc2 - This was a fun one. The main change here is using a list of references instead of indices for stopTimesIxByTrip and tripsIxByRoute (see getStopTimes and getTrips). Pointers reference the same structs in stopTimes and trips (if you have a gc might as well use it)
• write resp json to RespW e2b866f0ca174b22f6b6cdb86a0995e26719d36b - A huge bump for loadTestSmallResponse.js.

[sebnyberg]

Did a similar refactor for fun that ended up almost identical to yours.

The only difference is that I went all the way and read the CSV contents into the exact form expected by the JSON encoder and compressed keys with fnv.

https://github.com/sebnyberg/transit-lang-cmp/blob/main/trogsit/app.go

[benhoyt]

@CrimsonVoid This is nice work and analysis! One thing to note is that you don't need a bufio.Reader when using csv.NewReader, because the first thing csv.NewReader does is wrap the input in a bufio.NewReader. See https://cs.opensource.google/go/go/+/refs/tags/go1.21.3:src/encoding/csv/reader.go;l=183