Closed JohnHBrock closed 1 year ago
Looks good, makes total sense. Am away at kubecon at the moment but assuming the tests pass, I'll look at getting this merged when I'm back
Hi, just a reminder to review this when you get a chance. Thanks!
Sorry for the delay. Yep looks good
Changing the file reader buffer from
bytes
tobytearray
significantly reduces CPU usage. Usingbytes
is inefficient because it's immutable: you get the classic string building problem, where repeatedly appending to an immutable sequence requires O(n^2) operations.From the python docs:
I tried
io.BytesIO
too, butbytearray
has slightly better performance in my testing.