search

zach-morris / plugin.program.iagl

The Internet Archive Game Launcher (IAGL) will launch Games from the Internet using Kodi
GNU General Public License v3.0
344 stars 49 forks source link

Multi Threaded Downloading #225

Closed zach-morris closed 2 years ago

zach-morris commented 2 years ago

I'm attempting to incorporate multi threaded downloading into the addon. Functionality would work generally as follows: Download of a particular file is initialized Request is made to determine content-length of the file. Some files from archive.org provide this information and some dont If no content-length is provided or the file is sufficiently small (say less than a few MB), then just download the file as normal If the content-length is provided and the file is large enough, then:

  1. Create a sparse file (empty) of the correct size
  2. Generate n threads to download chunks of the file in the appropriate byte ranges using a range header
  3. Write the chunks to the appropriate place in the sparse file

I've got some of this down, but the thread safeness of steps 2 and 3 seem be not thread safe, so I'm struggling how to best tackle this issue without libraries that aren't available outside of the standard python 3 libaries in Kodi. All the research I've been doing bring up libraries like aiohttp or request-threads which aren't available...

Anyone have experience with this in Kodi?

zach-morris commented 2 years ago

Started implementation of this, and it seems to work. Need to do some more testing before it's ready to publish https://github.com/zach-morris/plugin.program.iagl/commit/1cfff8e6c3818508e90f1bcfa982765d87950b49

zach-morris commented 2 years ago

Implemented in 3.0.4