Suggestion for version detection in HTLM sources

[hurried6158]

Currently, version detection for HMTL sources where there is no version string in the HTLM or the apk link is not supported. A case where such support would be nice is for protonapps.com as discussed in #970 and #606. I want to suggest an approach that might or might not be feasible in an Android app (my knowledge in android and app development for that matter is lacking).

I would suggest taking the checksum of something like the first 30 bytes (header of the zip/apk file) of the APK and store this. During a check for updates, only download 30 bytes of the APK, checksum and compare to the currently stored checksum. If they differ, one could conclude that there is a new version available.

Hopefully this approach is sensible.

/H

[ImranR98]

Really good idea 👍 If I can implement it, it should become the default instead of hashing the version string.

[hurried6158]

I'm glad you enjoy the idea. I think the ideal way of doing this is with a HTTP range request. Not all servers hosting files support this, but many (or most?) do (this is the case with protonapps). If they do, they should return HTTP response code 206 wheb the HTTP range request is processed. Since not all do, it might be worth having the version string support left as a fallback. And for scenarios with no support for HTTP range request, and no presense of a version string, it would be unsupported by Obtainium I suppose?

[hurried6158]

(Sorry for accidentally pressing "Close with comment")

[hurried6158]

So I did some experimenting by curling the Proton Calendar APK with a HTTP range requiest using curl:

> curl -r 0-29 -O https://protonmail.com/download/CalendarAndroid/ProtonCalendar-Android.apk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    30  100    30    0     0    110      0 --:--:-- --:--:-- --:--:--   109
> xxd -e ProtonCalendar-Android.apk
00000000: 04034b50 00000000 08210000 00000221  PK........!.!...
00000010: 00000000 00000000 00000000     0066  ............f.

Looking at the (APK) ZIP file header, unfortunately the relevant fields in the header such as "File last modification date" and "CRC32 of uncompressed data" are all set to 0. This would probably mean that checksumming/hashing these 30 bytes won't really provide enough information that differs between two releases.

Perhaps one approach would be to checksum/hash something like the first 1KB of the APK. They should reasonably differ I suppose, but I can't say for sure.

EDIT:

Interestingly, this does not seem to be the case with Proton Pass, which has the CRC32 field in the ZIP header set accordingly:

> curl -r 0-29 -O https://proton.me/download/PassAndroid/ProtonPass-Android.apk
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    30  100    30    0     0    131      0 --:--:-- --:--:-- --:--:--   130
> xxd -e ProtonPass-Android.apk
00000000: 04034b50 00000000 08210008 18d90221  PK........!.!...
00000010: 00340e63 00380000 00390000     0000  c.4...8...9...
> xxd -e -s 14 -l 4 ProtonPass-Android.apk  # Retrieve CRC-32 of uncompressed data
0000000e: 0e6318d9

[TotallyAvailable]

How about starting in the opposite end. If you have a company proudly carrying the Open Source tag, providing you several ways of obtaining their Apps, why not try reaching out and ask.

• Are they willing to change those things to contain actual useful information.
• Are they willing to "adopt" certain ways the Open Source community has come up with to get their updates in consistent and trustable ways.
• Are they willing to give you tips on how to make your preferred way of updating work.

Even if you end up going to implement a feature to allow coverage for a few more use cases, just like with #946, you'll eventually run into issues of keeping all those things working. Not saying don't do or try it, more resilience is always better of course. More of a "don't try to brute force it" before kindly asking.

Also related to #1097, if they start seeing behavior they might not like, that would be 1 more available option removed from the "could work" bucket for making it work.

[hurried6158]

@TotallyAvailable while I agree with a lot of what you're saying, in the case of Proton, there are numerous people who have done such inquiries already with no positive outcome. They only support protonapps.com and google play store as channels for downloading their apps. They do however post Proton VPN and Proton Mail (most often a few weeks behind) on Github releases.

[ImranR98]

Added: https://github.com/ImranR98/Obtainium/releases/tag/v0.14.35-beta

It's optional but is on by default (for any newly added HTML apps, not for existing ones) since it seemed to work well. Hopefully I don't regret that.

[hurried6158]

Thanks! I will let you know if it works in the case of protonapps as soon as they release an update for any of the applicable apps.

[hurried6158]

It seemingly works. Got the Proton Calendar update that was released today. Thanks again for implementing this!