search

vgmoose / wiiu-hbas

[Wii U] Homebrew App Store - download apps for HBL (Legacy)
GNU General Public License v3.0
78 stars 11 forks source link

Download additional files for apps #10

Open vgmoose opened 8 years ago

vgmoose commented 8 years ago

Currently the app store only updates three things for an app: the meta.xml, the icon.png, and the actual elf. Some apps, however, may require additional files to be downloaded alongside them.

This wasn't implemented from the start, as it begs the question: what should happen to any conflicting existing files? Perhaps with #8 that would become a non-issue, as older files could be backed up and restored as well.

vgmoose commented 7 years ago

It'd be nice to look at how other package managers handle this problem. For example, yum provides RPMs which all the files packaged in the directories that they install into.

Without reinventing the wheel, I'd like to do a similar thing with .ZIP files. So for instance, a typical HBL elf app would look like: (all paths would unzip relative to sd:/)

spacegame.zip:

And an RPX app such as WiiU-Endless-Nuclear-Kittens could look as follows:

nuclear.zip

And then, for instance, the new haxchi HBL elf, which has a dependency to other files be stored on the SD card:

And this is all still operating on the assumption that HBAS only exists to serve HBL (aka, doesn't provide wupinstaller style, system menu installation at this time, although such installing would likely leverage this zip distribution style anyway)

I haven't looked into how other package managers do it, but one simple way I see to keep installation conflicts at a minimum, is to enforce strictly that the deepest folder cannot be shared between any two packages. Aka, if haxchi owns "sd:/haxchi", any other package that tries to install something to that directory will not be allowed. Similarly with "sd:/wiiu/apps/spacegame", only Space Game can use that folder.

These "deepest folders" can also be treated as dependencies, or things that can be depended on, for instance:

Retroarch Core Engine: provides: sd:/retroarch

Sega Emulator: depends: sd:/retroarch provides: sd:/wiiu/retro/segaemulator

Or in the case of useful style shared libraries:

IOSUHAX: provides: sd:/wiiu/apps/iosuhax

IOSU File Browser: depends: sd:/wiiu/apps/iosuhax provides: sd:/wiiu/apps/iosufilebrowser

By treating the deepest folders as mutually exclusively dependencies, and distributing zip archives of the folder, they can be safely depended on by other packages to possibly alert the user to install in addition to at download-time.

In this scheme, every package would be one zip file, analogous to an .rpm or a .deb, and metadata regarding the "deepest folders" would be exported. There could also be additional information about which deepest folders should be cleaned up when the app is uninstalled, and which ones should be left behind.

In order to support the scheme, in addition to providing the proper metadata when generating the repository (gen.py changes), a manner to download zip archives and recursively extract the files from them is also desired.

CreeperMario commented 7 years ago

If I have time this weekend, I'll look into ZIP extracting.

vgmoose commented 7 years ago

miniz may be able to be used for this, per woominstaller

Here's an example of the extraction: https://github.com/shinyquagsire23/woominstaller/blob/master/src/main.c#L374

vgmoose commented 7 years ago

@rw-r-r-0644 has added archive downloading in https://github.com/vgmoose/hbas/commit/3c65a18bc63e6563a16df75b9199698a59c7dc80, now the gen.py script needs to be updated to support packing other files into the folder.

So for instance, for haxchi, the zip that is downloaded (both on the web and in the app, this is what will be extracted onto the filesystem), is:

sd:
├── haxchi
│   ├── bootDrcTex.tga
│   ├── bootTvTex.tga
│   ├── config.txt
│   ├── iconTex.tga
│   └── title.txt
└── wiiu
    └── apps
        └── haxchi
            ├── haxchi.elf
            ├── icon.png
            └── meta.xml

However, this still doesn't address the issue of dependencies. For example, let's say a user installs Haxchi this way. It will properly download both the app folder and the extra sd:/haxchi folder. When that user goes to update or reinstall later, though, a naive approach would completely overwrite sd:/haxchi with whatever files are in the newly downloaded zip.

To address this, and combining some ideas I've mentioned above, there's a few options:

  1. discrete directories could versioned in the meta.xml, and only replace contents if the directory version changes, for eg: <directory version="1">sd:/haxchi</directory> would remain 1, and hbas would ignore replacing the directory even on new updates, unless that directory version is bumped.
  2. resolve to never replace any exist directories (disadvantage: cannot update any files besides the app folder once installed)
  3. rename existing directories to have _backup appended to them, so the new one can be downloaded without conflict (disadvantage: still basically replaces the files, just now they can be recovered via PC)
  4. ask the user what to do for each non-app folder that is about to be overwritten, whether they want to overwrite or ignore that directory.

The first option seems the most complicated to implement and keep track of the separate updates, but it may be the best use experience if done properly. Otherwise, I think 4 is also a great solution, the only issue being it requires the user to make a decision about something they might not know the answer to.

vgmoose commented 7 years ago

I keep waffling back and forth on this! (and talking to myself)

Ignoring dependencies, in order to properly handle upgrade and delete, it seems like I will need to install some type of manifest file per package to keep track of which files should be updated/deleted.

Here's a proposed .install file syntax. These would go in sd:/wiiu/hbas/

So this would be going somewhat with the second option from above ("resolve to never replace any existing directories") except it's now per-file, and the server can specify which ones should be updated or only installed at first GET.

This seems like the simplest way to move forward. Also worth noting that such .install files may themselves come bundled in the zip, otherwise they would be an additional download. The con of putting them in the zip is mostly that it would show up on the web as well.

opendata26 commented 7 years ago

You might want to take a look at https://wiki.archlinux.org/index.php/creating_packages as they have a similar format to the one you have proposed