pyodide / micropip

A lightweight Python package installer for Pyodide
https://micropip.pyodide.org
Mozilla Public License 2.0
68 stars 16 forks source link

MAINT Store data in bytes not io.BytesIO #91

Closed ryanking13 closed 6 months ago

ryanking13 commented 8 months ago

This is a split off of #90.

This PR changes how wheel data is stored in WheelInfo class during the package installation: instead of converting the downloaded wheel data to io.BytesIO and storing it, store it as bytes and convert it to io.BytesIO when we need to pass it to other methods that accepts io.BytesIO object.

No functional change is intended, just to avoid the mistake of having to do a seek(0) after reading the data.

There seems a slight overhead of converting bytes to io.BytesIO, but I think it is not critical.

Benchmark ```python import io import timeit num_iterations = 10000 bytes_size = 100_000 bytes_obj = b'0' * bytes_size def bench_bytes(): for i in range(num_iterations): io_obj = io.BytesIO(bytes_obj) io_obj.read() def bench_bytesio(): io_obj = io.BytesIO(bytes_obj) for i in range(num_iterations): io_obj.seek(0) io_obj.read() print('Benchmarking using a single BytesIO object and seek()...') print(timeit.timeit(bench_bytesio, number=1)) print('Benchmarking converting bytes to io.BytesIO multiple times...') print(timeit.timeit(bench_bytes, number=1)) ``` ``` Benchmarking using a single BytesIO object and seek()... 0.0006137999998827581 Benchmarking converting bytes to io.BytesIO multiple times... 0.001084399999854213 ```
rth commented 6 months ago

@ryanking13 Thanks! Let us know if you plan to address Hood's suggestions, otherwise +1 to merge even as is.

ryanking13 commented 6 months ago

Thanks for the review! I had forgotten about this PR.