Resume package downloading

[GoogleCodeExporter]

Currently opkg doesn't resume downloading of packages.
In that case problem appears while downloading of big packages using slow 
internet connection (2G mobile internet).
During timeouts and frequent disconnections big packages won't be downloaded at 
all.
Therefore it would be great if opkg could resume download.

Original issue reported on code.google.com by KirG...@gmail.com on 30 Dec 2013 at 4:12

[GoogleCodeExporter]

This would be a useful enhancement.

This is possible with both the curl backend: "curl_easy_setopt(..., 
CURLOPT_RESUME_FROM, size);" and the wget backend: "wget -c". I'd like to have 
it controlled by a command line option (maybe "--resume-download").

Original comment by paul.betafive on 1 Jan 2014 at 10:05

• Changed state: Accepted
• Added labels: Milestone-1.0, Priority-Medium, Type-Enhancement

[GoogleCodeExporter]

I have a patch that implements this feature and also fixes issue #117 in a 
following way:

I use cache directory (/var/lib/opkg/cache by default) to save all downloaded 
data (package list and packages itself).
All downloads are resumable.

*) To implemented package download md5 hash sum of package is saved in 
<package_name>.hash before downloading.
After that downloading of package starts.
<package_name>.hash is removed while completing download.
<package_name>.hash is used to check if file that is been resumed is the same 
as on the server (not changed).
So the following case will work:
download_packages -> interrupt -> update_package_list -> resume_packages.

*) To implement package list download server modification timestamp is saved in 
<package_list>.gz.stamp before downloading.
The rest of downloading logic is similar to package downloading.

*) To implement downloading of packages into current or specified directory 
packages are downloaded into cache directory first and then copied into 
specified directory.

Original comment by KirG...@gmail.com on 1 Jan 2014 at 12:33

[GoogleCodeExporter]

How does this interact with the existing caching implemented by 
opkg_download_cache() in libopkg/opkg_download.c? Could you send the patches to 
the mailing list (opkg-devel@googlegroups.com) please?

I also know that the package list is downloaded in three separate places 
(opkg_update_cmd in libopkg/opkg_cmd.c, opkg_update_package_lists in 
libopkg/opkg.c, release_download in libopkg/opkg_release.c). These really need 
merging into a single way of handling the package feed. That may affect your 
second point.

Original comment by paul.betafive on 1 Jan 2014 at 12:53

[GoogleCodeExporter]

This patch replaces existing caching.
As for downloading package list I changed all instances of calls to download 
method where I found it.

I'd like to submit a patch but I think I broke indents.
I do not understand indent style that is used opkg. There is a mix of tabs and 
spaces and I think all files should be reindented using tabs-only indents 
before submitting any patches.

Original comment by KirG...@gmail.com on 1 Jan 2014 at 9:39

[GoogleCodeExporter]

The indentation is already broke. Fixing it will break any existing patches and 
complicate branch merges within git. So the plan is to fix the indentation as 
the final change before the 0.3 release because after that point we won't need 
to forward-merge from the opkg-0.2.x branch any longer. I hope that makes sense.

For now, don't worry too much about the indentation. Just try to stay close to 
whatever is already there for consistency and we'll clean it up within the next 
6 months.

Original comment by paul.betafive on 1 Jan 2014 at 10:16

[GoogleCodeExporter]

Just came back from vacation and submitted patch that fixes this issue and also 
issue #117.
I called patch "Implemented resuming of interrupted downloads".
The logic of how it works was changed a little.
The main idea is to use caching for all downloads (local and remote) and 
timestamping to detect source file changes.

Original comment by KirG...@gmail.com on 8 Jan 2014 at 9:38

[GoogleCodeExporter]

I think this is fixed by your recent patches when curl is enabled and I don't 
think we can fix it when wget is enabled due to the differences between busybox 
wget and GNU wget. If that's right I'll mark this as closed.

Original comment by paul.betafive on 1 Feb 2014 at 2:17

[GoogleCodeExporter]

Right, you can close this issue.

Original comment by KirG...@gmail.com on 2 Feb 2014 at 1:46

[GoogleCodeExporter]

Original comment by paul.betafive on 3 Feb 2014 at 1:10

• Changed state: Fixed