mjishnu / pypdl

A concurrent pure python downloader with resume capablities
https://pypi.org/project/pypdl/
MIT License
44 stars 8 forks source link

`create_segment_table` calculation "errors" #13

Closed KennyChenBasis closed 6 months ago

KennyChenBasis commented 6 months ago

Hi,

I was reviewing this project's code and came across what looks to be some distasteful code in create_segment_table. In

        if segment != (segments - 1):
            end -= 1  # [0-100, 100-200] -> [0-99, 100-200]
        # No segment_size+=1 for last setgment since final byte is end byte

end -= 1 should also be applied to the last segment; if the size is 200, the last byte is at index 199 (0-index, inclusive).

There is also careless use of floating points -- int(size / segments * segments) doesn't always equal size, e.g. int(109 / 11 * 11) returns 108. This problem is accidentally mitigated by the above problem. However, cases where the calculation is 2 (or greater) less can also occur, though you'll need very large files: int(9999999999999966 / 17 * 17) returns 9999999999999964.

Setting the last segment's end as size - 1 is acceptable, though the sane thing to do is to also have partition_size be an int.

mjishnu commented 6 months ago

thanks for reviewing, your analysis is correct. when the size is large it would result in incomplete segments. I will fix it or if you are willing feel free to create a PR.

you can use divmod to easily get the partition_size and the remaining bytes

partition_size, add_bytes = divmod(size, segments)
mjishnu commented 6 months ago

fixed you can check f09ae1b. do tell me if you find any other bugs.

KennyChenBasis commented 6 months ago

LGTM