illogical-robot / apkmirror-public

APKMirror.com bugs
http://www.apkmirror.com
111 stars 32 forks source link

Request: ditch APKM format and use a zip format instead #119

Closed AndroidDeveloperLB closed 3 years ago

AndroidDeveloperLB commented 4 years ago

According to my tests on various APKM files, it takes a long time (4 seconds for a simple 23 MB file, and that's before installing) to handle them compared to a zipped file like on SAI app, which practically takes an instant.

This is terrible in terms of performance. Please stop using this format. Use better alternatives, ones that are more commonly used and are more efficient.

Attached sample APKM file.

music.zip

patrickdrd commented 4 years ago

+1

MrDodojo commented 4 years ago

+1, also if anyone wants to transfer a .apkm file to a zip use https://github.com/souramoo/unapkm

the4anoni commented 4 years ago

+1

AndroidDeveloperLB commented 4 years ago

@archon810 It's not a duplicate. I requested to make it open sourced because no other app, including even on PC can handle it.

Now I think you should ditch it, seeing how inefficient it is compared to a normal ZIP file. What advantages do you see in APKM anyway? It's slow, not a common standard, not open sourced, can't be used by other apps, can't be opened via the PC. Because of this, someone had to make a repository to be able to handle it and extract the files from it into a normal ZIP file. Try to compare SAI to APK-mirror app in terms of speed - how long does it take to install an app.

MrDodojo commented 4 years ago

But SAI migh include it soon.

patrickdrd commented 4 years ago

no, sai developer said that he won't

AndroidDeveloperLB commented 4 years ago

Good for him. May this format be removed entirely. So useless. Adding support to it only makes it more legitimate. From now on, I will prefer to check on other websites for APK files instead of APK-mirror. Till now, I always used only APK-mirror, because I loved it and I loved Android-police. Making an app and a file format that is against the spirit of Android made me think the opposite about APK-mirror.

TPS commented 4 years ago

APKPure has a very good app to handle their APKs/bundles, with the only downside being some ads/tracking, so not much different than APKMirror's site.

AndroidDeveloperLB commented 4 years ago

@TPS Can't reach the link. In any case, at least it's a format that anyone can use (including edit and extract easily, without an additional app), whether it's on Android or PC. It's their app so it's their choice how to make money from it, if at all. According to the website, they even explain exactly what it contains, and they offer one for PC too: https://apkpure.com/xapk.html

Imagine what would have happened in the story of MS Office, if all of its formats were open from the beginning. Don't you think it would have had a much better competition? For years I couldn't open its documents well via OpenOffice and LibreOffice. I'm sure there are still cases that it won't open them well.

Or with Chromium, or with Firefox, if those weren't open sourced. There are many forks of those. Some are quite good. It doesn't mean the original isn't good, and it doesn't mean they will be gone just because there is a competition.

MrDodojo commented 4 years ago

Making an app and a file format that is against the spirit of Android made me think the opposite about APK-mirror.

also me too, makes me want to make an online .apkm to .zip converter that takes apkmirror links directly....

the4anoni commented 4 years ago

APKPure has a very good app to handle their APKs/bundles, with the only downside being some ads/tracking, so not much different than APKMirror's site.

https://github.com/Aefyr/SAI/issues/104 I've just tried something, and it worked :)

AndroidDeveloperLB commented 4 years ago

Thank you for re-considering it. APKM seems much slower than a simple zip file. it adds up to 10 more seconds on some devices. And even if it was open, according to what I see, it will still suffer from the same inefficiency. I don't see any way that it's superior.

archon810 commented 4 years ago

We're discussing internally a move to the zip format, with some metadata additions that improve the experience for our Installer, along with simultaneous support for .apks and .xapk.

Stay tuned, as we'll also need to support both encrypted and unencrypted formats for a while.

AndroidDeveloperLB commented 4 years ago

This is the way :)

MrDodojo commented 4 years ago

the files wont be still encrypted right?

archon810 commented 4 years ago

Correct.

Rikk commented 4 years ago

+1 The chosen format should have been zip (.apks). @archon810, thanks for listening.

TPS commented 4 years ago

Also thanks to @souramoo for cracking .APKM, which might have influenced the decision to abandon it.

dotjaz commented 4 years ago

Duplicate of #113. As I specified there, we are not going to move to the zip format at this time.

Well, that makes ditching Apkmirror and Android Police soooo much easier.

TPS commented 4 years ago

Duplicate of #113. As I specified there, we are not going to move to the zip format at this time.

Well, that makes ditching Apkmirror and Android Police soooo much easier.

@dotjaz You stopped reading too soon:

We're discussing internally a move to the zip format, with some metadata additions that improve the experience for our Installer, along with simultaneous support for .apks and .xapk.

TPS commented 4 years ago

@archon810 I'm not pushing for progress per-se, but am wondering what progress there might be.… Could we get an update?

AndroidDeveloperLB commented 4 years ago

@TPS They probably wait for Google to release Android R, to see if it will have a new, official standard. My guess is that sadly Google won't offer a new official standard.

archon810 commented 4 years ago

I pushed the Installer update to the alpha channel that supports installing unencrypted apkms, as well as xapk, apks, and zip. If there are no issues, I'll push it out to beta soon, and once enough people updated, we'll start shifting apkms to the unencrypted format.

AndroidDeveloperLB commented 4 years ago

@archon810 "unencrypted apkm" ? What are those? Where can I see an example of this? Are they ZIP files? Or is it now the new one for APK-mirror website?

archon810 commented 4 years ago

Zip. As I mentioned, once enough people updated (we also have a forced update built into the app that I'll trigger on), APKMirror will switch to this new format. The app will support both for a while.

AndroidDeveloperLB commented 4 years ago

@archon810 I don't see you mentioned it. You mentioned "unencrypted apkms, as well as xapk, apks, and zip" . You didn't mention what are "unencrypted apkms". You could have used 7z for example.

Anyway, so this means APKS will be about the same as APKM ? For both, we will be able to open as ZIP file if we wish?

I suggest to consider using a different format than ZIP though, to allow to send via GMAIL too. Now it's a good time, since I know APKM was possible to send via GMAIL, and if you switch to ZIP, it won't be allowed, again, because Gmail will block it as it's the same as ZIP.

I've noticed, for example, that the "zstd" and "xz" compression formats (even uncompressed, when possible) are not blocked on Gmail.

Of course, this might not work in the future, if Gmail will block those too. But for now, it will work the same as APKM.

https://commons.apache.org/proper/commons-compress/examples.html

shah-sudeep commented 4 years ago

@archon810 I don't see you mentioned it. You mentioned "unencrypted apkms, as well as xapk, apks, and zip" . You didn't mention what are "unencrypted apkms". You could have used 7z for example.

Anyway, so this means APKS will be about the same as APKM ? For both, we will be able to open as ZIP file if we wish?

I suggest to consider using a different format than ZIP though, to allow to send via GMAIL too. Now it's a good time, since I know APKM was possible to send via GMAIL, and if you switch to ZIP, it won't be allowed, again, because Gmail will block it as it's the same as ZIP.

I've noticed, for example, that the "zstd" and "xz" compression formats (even uncompressed, when possible) are not blocked on Gmail.

Of course, this might not work in the future, if Gmail will block those too. But for now, it will work the same as APKM.

https://commons.apache.org/proper/commons-compress/examples.html

Now I think, you are not being open, zip format is much better format in compared to zstd or xz( not in terms of compression) bcoz all file manager do support it and without any additional app, we can open that format from either mobile or pc. You could use link instrad of attaching the package file. Going to those formats instead of zip will create another format, so let us stick with our previous decision on making the format open and open means zip format bcoz although other formats too are open but they needs additional library to open these format while zip format doesn't.

shah-sudeep commented 4 years ago

I pushed the Installer update to the alpha channel that supports installing unencrypted apkms, as well as xapk, apks, and zip. If there are no issues, I'll push it out to beta soon, and once enough people updated, we'll start shifting apkms to the unencrypted format.

what will happen to the currently updated files that are already on the server in encrypted apkm format will those be converted too or only upcoming files? If those already on server are not converted then we will need the app to support both format always.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep Both formats are open sourced, and are used in various compression apps. Look at the link I've provided, of Apache. You can't attach a link instead of sending the file, when you want to send the file. It's not creation of a new format. Those formats exist. I think most compression apps out there support those formats already. I know that for PC, the popular 7-zip app does, and for Android, ZArchiver does. Most users who open APKM files to check what's inside will have a compression app, whether on PC or Android. And, you can mention "what is APKM?" in the website and on the app, just like APKPure website does.

Those formats also seem more modern than the old ZIP format, at least according to the benchmarks I've found. Think about it as moving to WEBP instead of JPG/PNG, for example. Sure many don't know it, but it's supported in various places already, and usually better.

the4anoni commented 4 years ago

@shah-sudeep Both formats are open sourced, and are used in various compression apps. Look at the link I've provided, of Apache. You can't attach a link instead of sending the file, when you want to send the file. It's not creation of a new format. Those formats exist. I think most compression apps out there support those formats already. I know that for PC, the popular 7-zip app does, and for Android, ZArchiver does.

Those formats also seem more modern than the old ZIP format, at least according to the benchmarks I've found. Think about it as moving to WEBP instead of JPG/PNG, for example. Sure many don't know it, but it's supported in various places already, and usually better.

For SAI most used format is .zip/.apks. Now you're making your own format.

AndroidDeveloperLB commented 4 years ago

@the4anoni It's not my format. It's already an existing format, and it's already used in various compression apps. As I wrote, opening such files is done by compression apps, something that users that do it already have. You can have "what is APKM file" like APKPure website, to make it open source by explaining what's inside, so that others could handle it.

The APKM format can be open. You State what it's based on, and what's inside. Telling that it's just ZIP format doesn't mean it's fully open. That's only the first step, and to make it available everywhere, I suggest to think of a more modern format.

I just noticed that Gmail blocks ZIP files, so I thought maybe it could be nice to use something else, to allow bypassing it. Of course, ZIP files should be much better than what we have today, but I also want to see APKM fully open sourced.

AndroidDeveloperLB commented 4 years ago

Anyway, it's just a suggestion. I will be happy with ZIP too. But please write exactly what's in the file, how it's structured.

shah-sudeep commented 4 years ago

@shah-sudeep Both formats are open sourced, and are used in various compression apps. Look at the link I've provided, of Apache. You can't attach a link instead of sending the file, when you want to send the file. It's not creation of a new format. Those formats exist. I think most compression apps out there support those formats already. I know that for PC, the popular 7-zip app does, and for Android, ZArchiver does. Most users who open APKM files to check what's inside will have a compression app, whether on PC or Android. And, you can mention "what is APKM?" in the website and on the app, just like APKPure website does.

Those formats also seem more modern than the old ZIP format, at least according to the benchmarks I've found. Think about it as moving to WEBP instead of JPG/PNG, for example. Sure many don't know it, but it's supported in various places already, and usually better.

I do know this that those format compression is better and are modern but what I wanted to tell was that zip format is supported without using any archiver app mainly on android, also most file manager support zip by default buts needs addon or may nor support other format, so it will be better to stick with the zip format, that's it. I think you need to manage to send those files as before, when there was no apkm format. As everything has some pros and cons .apkm format too had that pros but now converting into xz or other format will bring more cons than pros.

shah-sudeep commented 4 years ago

@shah-sudeep Both formats are open sourced, and are used in various compression apps. Look at the link I've provided, of Apache. You can't attach a link instead of sending the file, when you want to send the file. It's not creation of a new format. Those formats exist. I think most compression apps out there support those formats already. I know that for PC, the popular 7-zip app does, and for Android, ZArchiver does.

Those formats also seem more modern than the old ZIP format, at least according to the benchmarks I've found. Think about it as moving to WEBP instead of JPG/PNG, for example. Sure many don't know it, but it's supported in various places already, and usually better.

For SAI most used format is .zip/.apks. Now you're making your own format.

naming format won't matter, if it uses open compression method. the only ques remains is that whether that format can be supported by other apps or will it be proprietary format naming.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep I know ZIP is more popular. And I know many don't need extra to open it. But, so far APKM was possible to send via Gmail, and now if it's ZIP based, it will be blocked.

Try it yourself. Create a ZIP file that has APK files, change file extension to APKM, and send to yourself. It won't allow it.

Try to send APKM file today via Gmail. You will succeed. Try to do it with APKS and XAPK, and you will fail. That's because Google blocks ZIP files that have APK files inside. Same goes for many other compression methods.

APK-mirror now has a chance to change it, and also keep this behavior that doesn't exist for APKS and XAPK files. This was possible so far. Of course I don't like the format that was chosen, and of course I prefer ZIP, but it's also nice to be able to send via Gmail. It can have a small advantage on the way, compared to others.

That being said, it doesn't mean Gmail will always be this way, and some day it might also be blocked. So this should also be considered.

shah-sudeep commented 4 years ago

@shah-sudeep I know ZIP is more popular. And I know many don't need extra to open it. But, so far APKM was possible to send via Gmail, and now if it's ZIP based, it will be blocked.

Try it yourself. Create a ZIP file that has APK files, change file extension to APKM, and send to yourself. It won't allow it.

Try to send APKM file today via Gmail. You will succeed. Try to do it with APKS and XAPK, and you will fail. That's because Google blocks ZIP files that have APK files inside. Same goes for many other compression methods.

APK-mirror now has a chance to change it, and also keep this behavior that doesn't exist for APKS and XAPK files. This was possible so far. Of course I don't like the format that was chosen, and of course I prefer ZIP, but it's also nice to be able to send via Gmail. It can have a small advantage on the way, compared to others.

That being said, it doesn't mean Gmail will always be this way, and some day it might also be blocked. So this should also be considered.

I do agree with your facts, but still I don't agree with you points.

  1. Why do we need to send an email attachment instead of the link of that apk?

  2. can we not send a encrypted zip files with apk inside it?( Though this will require manual archiving)

I think gmail do block the zip with apk inside to protect users from downloading malware apps through attachment.

That being said having more compatible rather than having some other compression format for this reasoning alone can't be justified. And about the archive size, I don't think it differs much. as the split apk present inside the split apps package is already in using zip method and won't let recompressing it to make any notable difference in size.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep

  1. You don't always have the link to the APK. If the format is going to be open, it could be generated too.
  2. I don't understand the question. You mean password protection? If so, ZIP still allows you to see what's inside, sadly. I don't think there is a way to overcome this. Weird thing is that I can't find of any compression file format that allows to have password-protection and also block seeing what's inside unless you have a password.

I actually liked that I can send APKM via email too.

"I think gmail do block the zip with apk inside to protect users from downloading malware apps through attachment." Maybe, but this includes even sending files to people you know and work with, and even sending to yourself.

The compression itself is optional for users, as it can be for ZIP. It's just yet another advantage of using a more modern format. It's not just the size, BTW. It can also be about speed of decompressing. This can also be taken into account.

shah-sudeep commented 4 years ago

@AndroidDeveloperLB

  1. You don't always have the link to the APK. If the format is going to be open, it could be generated too.

That's a good point but used by a very few people, still we can use sites like wetransfer for that.

  1. I don't understand the question. You mean password protection? If so, ZIP still allows you to see what's inside, sadly. I don't think there is a way to overcome this. Weird thing is that I can't find of any compression file format that allows to have password-protection and also block seeing what's inside unless you have a password.

So even though zip is password protected, Gmail can detect it and block it, it's sadπŸ˜‘.

I actually liked that I can send APKM via email too.

But this is not used by many people in general. Also apkm below 25 mb can only be send, what about the larger files? as split apk available on apkmirror or other sites do contain all split packages, most of the time these packages are larger in size so, gmail attachment won't be much useful.

Maybe, but this includes even sending files to people you know and work with, and even sending to yourself.

Sad lifeπŸ˜‘, it's google way, no-one can do anything about that.

The compression itself is optional for users, as it can be for ZIP. It's just yet another advantage of using a more modern format.

I don't think we can come to agree on the same point, as our perspective is different.

It's not just the size, BTW. It can also be about speed of decompressing. This can also be taken into account.

Definitely, this can be a matter. But not for files below 25 mb which u can use as an attachment, the difference would be probably of few milli seconds.

For larger files it may make a difference but while decompression may take lesser time for newer formats, it may not be as efficient for the cpu and resources used during the compression. Though it can reduce the bandwidth.

The best way to share any apk file is to create a sharable folder in drive and share it's link, which can be of any size within gdrive storage limit.

There are pros and cons on every factor. IYO modern compression format will be better, but IMO much widely used format would be better as there isn't much vast difference between those format size, duration of compression and decompression, nd used resources as much as I know. There is but not much for most of the small sized files.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep Why use an additional step to upload, if you can do it in a single step? If you know it's a small file, you can avoid it.

And I know that Gmail doesn't allow large files, but many apps do fit into 25MB. The APK-mirror installer (though not APKM of course) itself is less than that, for example.

Thing is, if you allow APKM to be sent via Gmail, this can include not just split APK files. It can include single ones.

I don't think we can come to agree on the same point, as our perspective is different. About what exactly? I wrote that you could compress with any level you wish. That's correct for all compression methods I know. And many offer "uncompressed" too.

As for large files, this can actually have even more advantage, because then you might save on bandwidth, and on storage (in case you save on the device or on Drive, for example). Currently, APKS and XAPK are not compressed, and the advantage of this is faster opening of them. The downside is that they take more space, and I don't think you have a choice for either of them to have compression level (even though technically should be possible).

What's wrong with having compression level an optional feature?

shah-sudeep commented 4 years ago

@AndroidDeveloperLB

Why use an additional step to upload, if you can do it in a single step?

okay, but not all the time.

And I know that Gmail doesn't allow large files, but many apps do fit into 25MB. The APK-mirror installer (though not APKM of course) itself is less than that, for example.

yeah,bcoz apkmirror app is just .apk not split apks. a lot of apps can come within 25mb but much more can't.

Thing is, if you allow APKM to be sent via Gmail, this can include not just split APK files. It can include single ones.

creating a whole new format which will be then converted into zip again bcoz single ones means .apk file which is again a zip file. You lost the point here 🀣, bcoz android system will definitely need the apk file within zip format, not in any other. what will single ones do with other format? will android installer accept that? won't it take time to convert those single ones to apk again?

About what exactly? I wrote that you could compress with any level you wish. That's correct for all compression methods I know. And many offer "uncompressed" too.

I tried with different compression level for the zip format but 99% of apps didn't changed their file size by noticable amount. The split apk are already a zip package and re-compressing those doesn't make a difference.

As for large files, this can actually have even more advantage, because then you might save on bandwidth, and on storage (in case you save on the device or on Drive, for example).

same as above mentioned, no file size difference, no saving of storage, no saving of bandwidth.

Currently, APKS and XAPK are not compressed, and the advantage of this is faster opening of them. The downside is that they take more space, and I don't think you have a choice for either of them to have compression level (even though technically should be possible).

again same as above, try compression of split apk packages with different level and format, I don't think you will get any noticable difference most of the time.

What's wrong with having compression level an optional feature?

Nothing, if it really matters. ✌️✌️

shah-sudeep commented 4 years ago

@AndroidDeveloperLB Ahh, and main thing, can we use zstd or xz for multiple files or folder? I think those format can only be used for compression of single file only, which mean we will need archiving first into some other format before compressing into those.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep

creating a whole new format which will be then converted into zip again bcoz single ones means .apk file which is again a zip file. You lost the point here 🀣, bcoz android system will definitely need the apk file within zip format, not in any other. what will single ones do with other format? will android installer accept that? won't it take time to convert those single ones to apk again?

I don't understand what you write here. APKM isn't a standard of Android to install split-APK files (and single ones), and so does ZIP file. There is no official standard on Android for a file that includes split APK files inside (and single APK files too). All apps that you find out there to install split APK files use a new API that just needs InputStream. It doesn't care where it is. It could be on the cloud too.

I tried with different compression level for the zip format but 99% of apps didn't changed their file size by noticable amount. The split apk are already a zip package and re-compressing those doesn't make a difference.

Which file formats did you compare? APKS and XAPK are uncompressed. Not sure about APKM though. It might be compressed, as it's very slow to open, but maybe it's just its encryption. Anyway, this always depends on the file you try, and as I wrote, it's an optional feature for users. If you don't want to use it, or if you notice that for the given case you try, it doesn't help, don't use it. I think you are right about the compression. Could be useless, at least for most cases, as usually developers try to make APK files small, and as Google tries to do it too. However, I'm sure there are cases that can be compressed nicely. Just like on benchmarks, if you see a few results showing something, it doesn't mean it's this way for all cases.

Ahh, and main thing, can we use zstd or xz for multiple files or folder? I think those format can only be used for compression of single file only, which mean we will need archiving first into some other format before compressing into those.

Sadly true. Why would they make it this way? Doing what you wrote could work, and choosing a different format could also work. Again, this is just a suggestion to think about, because currently APKM does support being sent via Gmail.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep BTW, a trick to send via email: split compressing into files of 25MB. Of course, this will be annoying to download.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep I've noticed 7z allows encryption of the files within, so if you password-protect them you can't see what's inside. However, Gmail blocks 7z files that were made this way, even if they include images alone...

Really annoying :(

shah-sudeep commented 4 years ago

@AndroidDeveloperLB I don't want to explain again, but for the last time I will.

I don't understand what you write here. APKM isn't a standard of Android to install split-APK files (and single ones), and so does ZIP file. There is no official standard on Android for a file that includes split APK files inside (and single APK files too). All apps that you find out there to install split APK files use a new API that just needs InputStream. It doesn't care where it is. It could be on the cloud too.

All the .apk files whether that be single ones or those inside the split packages are itself a zip file, it's just that the extension is renamed as .apk. There is no standard for the split-package file but what it contains inside do have.

Let's talk about the single .apk files, You mean it should be in .apkm format with some modern compression method which mean you have two way to do it either make a package of all the files that's inside the apk package, it will broke the apk file signature, so it won't work. Next way is to re-archive the single .apk file, it will be success. But while installing you will need to extract first the base.apk file first in order to install which will need more time and resources, so at the end failure bcoz without re-archive it could have just installed inside the app directory whatever be the method.

About the split-apps, the archive can play a role but this this still needs to make single stream of the files first bcoz those mentioned format doesn't support multiple files or folder, which too will consume time due to double archive. Though some bandwidth may be saved but installation time would increase. And about the file size you won't get any noticable difference by re-compressing the .apk split files which are already a package.

I tried with different compression level for the zip format but 99% of apps didn't changed their file size by noticable amount. The split apk are already a zip package and re-compressing those doesn't make a difference.

Which file formats did you compare? APKS and XAPK are uncompressed. Not sure about APKM though. It might be compressed, as it's very slow to open, but maybe it's just its encryption.

Please try yourself once.

Anyway, this always depends on the file you try, and as I wrote, it's an optional feature for users. If you don't want to use it, or if you notice that for the given case you try, it doesn't help, don't use it. I don't think it will.

I think you are right about the compression. Could be useless, at least for most cases, as usually developers try to make APK files small, and as Google tries to do it too. However, I'm sure there are cases that can be compressed nicely. Just like on benchmarks, if you see a few results showing something, it doesn't mean it's this way for all cases.

but for 99%, bcoz I tried with most of the apps installed on my phone, and maybe I got difference in Youtube vanced and chrome only.

Ahh, and main thing, can we use zstd or xz for multiple files or folder? I think those format can only be used for compression of single file only, which mean we will need archiving first into some other format before compressing into those.

Sadly true. Why would they make it this way? Doing what you wrote could work, and choosing a different format could also work. Again, this is just a suggestion to think about, because currently APKM does support being sent via Gmail.

Overall we can't compress in those formats you mentioned, if we do we need to use double package.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep

Let's talk about the single .apk files, You mean it should be in .apkm format with some modern compression method which mean you have two way to do it either make a package of all the files that's inside the apk package, it will broke the apk file signature, so it won't work. Next way is to re-archive the single .apk file, it will be success. But while installing you will need to extract first the base.apk file first in order to install which will need more time and resources, so at the end failure bcoz without re-archive it could have just installed inside the app directory whatever be the method.

Whether the APK file/s are in a ZIP or inside a different file format, you will always have to extract it somehow. You need the InputStream of of whatever file you have.

As for the optional way to compress (with real compression or not), again, this is not the main thing I suggested. It's only optional. And it should work better in some cases. I've now tested this one: https://www.apkmirror.com/apk/google-inc/android-auto/android-auto-5-5-6029-release/android-auto-google-maps-media-messaging-5-5-602933-release-android-apk-download/download/ This APK file takes 20.31MB Using zstd, "no compression" - it somehow got to 12.92MB Using xz, again "no compression" - it somehow got to 12.67MB . Using ZIP, with "no compression", it got to 20.31MB, which makes sense as it's the same as the APK.

So again, it really depends on the case.

Double archive is a problem. I know. If it's not really compressed though (meaning just storing), the difference in extraction shouldn't be noticeable.

Do you know of other known compression/archiving formats that do allow multiple files inside, while also not being blocked by Gmail? I wanted in the beginning to suggest 7zip, but it's also blocked.

shah-sudeep commented 4 years ago

@AndroidDeveloperLB Thanks for the test, but IMO it's useless bcoz these formats won't work with the android system. I too do know that those files can be compressed is much lesser size with some newer methods but we can't do so. All the apk (single) are a zip file, they are kept inside the system partition system/data/app/name.package/base.apk and you can't keep this apk file any any other format other than the zip format bcoz android system can't read it. It's not windows where all the program files are extracted while installing, only /lib files are, others remains inside the .apk file( which must be zipped not any other method bcoz android core system only understands zipped files as .apk. any other method used to create .apk file won't be read by android.

So, from your point, we will need to package app(.apk) using in other method and then distribute it, but during the installation it will need to extract those files and re-package into .apk format but by using zip method.

For split apps to, the distribution method can be different, but all those split apks files inside the package will need to be the original .apk only( which is already a zip package. Let us suppose an example, .apk file is a bag and .apkm is a container. So for single package, we don't need to do any suffling within the bag, bcoz at the last we will need the bag in the original(zipped) condition to be accepted by the system. If you like you can but that will be like doing same work just to show that we are soing work, bcoz the suffled bag won't be accepted by the system and u will need to re-suffle to the original order at the time of installation.

But it is different in case of split apps, bcoz although we can't re-order anything inside the bag(.apk), but we can keep it however we like in the container(.apkm) bcoz at the end we will need bags only, not the container. But having concept of suffling the books inside the bags(.apk) to accommodate in less place inside the container won't work. If you do so u will need to re suffle all those books in those bags in the order that they were previously in(zipped).

Summary, we can use whatever method in the container but not in the base package. So while compressing the split packages, it doesn't reduce in size by much bcoz it is already compressed once. If we hadn't, we could have brought down the size by much.

Try on .apkm files by installing all the split files inside the container and then backing up by using SAI. nad then trying to create a new container in whichever format or method you like. ( but don't extract those split apk inside bcoz they do need to be in the same format as they are now, only container can be altered.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep The extraction is the same as from APKM works now. It's just an InputStream that you get from within the file. You don't create new files on the way. It doesn't matter if it's a single APK file or multiple. Check the code of the app. It doesn't need real files. In general, it could be from Google Drive directly (though a bit slower).

And as I've shown, even if it's a single APK, for some reason you gain in storage space in some cases (without compression at all, to my surprise). It reduced from 20.31MB to ~13MB ... this alone is nice to have.

Why do you think it's not much? ~7/20 is about 35%.

Why do you need more proof , and of APKM files that are closed source and we don't know exactly if they are compressed or not? I've shown it about APK files. Those are what you get as input. Not APKM files.

shah-sudeep commented 4 years ago

do try installing then,πŸ™„πŸ™„πŸ™„, you can't bcoz android won't support it be it inputstream or whatever method you use to install ( as much as I know). If you can, do try placing the .apk file inside the data directory if you have rooted dvice. And do try on the split apk package not on the single apk. I too do know the 35% gain but what will be the use of gain if you can't run the app?

For split package the gain will be useful but can't gain much due to the re-compressing issue.

Either I don't know anything about the .apk format and android system or I am failing to explain to you correctly.

I am saying again that container can be used in any format but the base .apk files can't be touched. Do look at the .xapk, .apks, .apkm files all uses different container but inside them contains the same .apk files untouched.

AndroidDeveloperLB commented 4 years ago

@shah-sudeep Of course I can. I just need to change the code to support it (Check the code of SAI to see how it's done). InputStream is what this app uses, because the new Android API allows it (and actually forces you to do it as such). What you need is to get the InputStream of each APK file and pass it to the API. That's it. It doesn't care where you got this InputStream from. It can be from the weirdest file format you can think of. As long as it's APK content that you send it, it doesn't matter. I already told you. Check the snippets here about to extract the InputStream from the file you choose (ZSTD/XZ or whatever) : https://commons.apache.org/proper/commons-compress/examples.html

So as you've noticed, you get only a single file being inside ZSTD/XZ. Assuming you know the name of it , you will decide what to do with it. If it's APK, you can use it right away, and if it's a ZIP containing the APK files, you need to dig further there instead.

Why would a rooted device have any issues here? You get InputStream and you do whatever you wish with it. Create files, create folders for them, etc... You can run the app after it's installed, of course.

The APK content isn't touched. When you get an InputStream, it's after decompression, of course. That's true for ZIP file and every other archive file , whether it's compressed or not. That's true for both the Android framework and the library.