Rewrite frame_buffer to use bytearray()

[Lukasa]

Good artists copy, great artists steal. One of @njsmith's great decisions with h11 was to write his buffer in terms of bytearray. This has some major efficiency gains on Python 3.4.

An easy win for us would be to rewrite the buffering internals of our frame buffer using h11's, while removing the things that h11's buffer implements that we don't need (in particular, any line-oriented constructs). Before we do it, we should write a benchmark of the buffering performance under reasonable HTTP/2 conditions to see what kind of improvements we see.

While I'm here: @fijal, the comments in h11's framebuffer say:

# Note that starting in Python 3.4, deleting the initial n bytes from a
# bytearray is amortized O(n), thanks to some excellent work by Antoine
# Martin:
#
#     https://bugs.python.org/issue19087

...

# make sure PyPy also has the optimization

Does it?

[njsmith]

I have a branch somewhere to extract h11's buffering logic and put it up on pypi as a standalone project, since it can usefully be dropped into lots of sans-io s situations... That's another option.

On Mar 7, 2017 3:51 AM, "Cory Benfield" notifications@github.com wrote:

Good artists copy, great artists steal. One of @njsmith https://github.com/njsmith's great decisions with h11 https://github.com/njsmith/h11 was to write his buffer https://github.com/njsmith/h11/blob/master/h11/_receivebuffer.py in terms of bytearray. This has some major efficiency gains on Python 3.4.

An easy win for us would be to rewrite the buffering internals of our frame buffer using h11's, while removing the things that h11's buffer implements that we don't need (in particular, any line-oriented constructs). Before we do it, we should write a benchmark of the buffering performance under reasonable HTTP/2 conditions to see what kind of improvements we see.

While I'm here: @fijal https://github.com/fijal, the comments in h11's framebuffer say:

Note that starting in Python 3.4, deleting the initial n bytes from a# bytearray is amortized O(n), thanks to some excellent work by Antoine# Martin:## https://bugs.python.org/issue19087

...

make sure PyPy also has the optimization

Does it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/python-hyper/hyper-h2/issues/474, or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOaO5Ldfx9wkNW_mxYDgqXbnXaTVADks5rjUTWgaJpZM4MVYl8 .

[Lukasa]

Yup, I'd be happy to depend on such a module.