Switching from single-file butler push to multi-file pushing seems to cause issues

[bjorn]

So far I have been pushing a single AppImage of Tiled on my linux-64bit-snapshot channel. This evening, I tried pushing the AppDir instead, so simply the directory containing all the files. I tried this because I thought it could reduce patch size and it would make the other executables shipping with Tiled more readily available.

But pushing the AppDir did something really strange on itch.io, where downloading the snapshot would produce a file called "Tiled-2017.06.27-x86_64.AppImage" (the name of the previous snapshot) but the contents was a ZIP of the new snapshot.

Unfortunately I couldn't test what the itch.io app would have done, because it appeared to be having networking issues (downloads weren't starting). In the meantime, I have pushed a single-file update again, which seems to be working fine.

In case it helps, the "wrong" version is the first "2017.07.11" in the build history.

[fasterthanlime]

But pushing the AppDir did something really strange on itch.io, where downloading the snapshot would produce a file called "Tiled-2017.06.27-x86_64.AppImage" (the name of the previous snapshot) but the contents was a ZIP of the new snapshot.

Ah, there's two things going on here:

• Browsers can't download folders, so we serve them a .zip (that's working as intended)
• That upload was previously a single-file, so the backend picked up the name of the single file to show on the web - but now that it's a folder, it's irrelevant (that's a bug, /cc @GranPC)

[bjorn]

I just tried to switch my macOS release from .dmg to a .zip file, and ran into the same bug.

With the .dmg release, after downloading there is just a non-functional Install button shown on the Downloads page in the itch.io app. Is that related to issue #1209?

[fasterthanlime]

(which doesn't seem to be supported by the itch.io app?)

original comment had that line - dmg is most definitely supported!

Can you post full app logs after trying to click Install? I'll also try to reproduce locally

[fasterthanlime]

(Note that dmg is much worse for the app than just pushing the .app folder with butler - you lose the patching, integrity verification, pretty much everything. If you want to supply a dmg I'd suggest doing it in addition to a zip, so that app users still get nice updates, and web downloaders get the pretty dmg thing)

[bjorn]

original comment had that line - dmg is most definitely supported!

I removed the line because I concluded support was intended from issue #1209. I'll see if I can find any logs, but nothing was popping up at least.

(Note that dmg is much worse for the app than just pushing the .app folder with butler - you lose the patching, integrity verification, pretty much everything. If you want to supply a dmg I'd suggest doing it in addition to a zip, so that app users still get nice updates, and web downloaders get the pretty dmg thing)

I've been distributing Tiled as a .dmg since years, but I am by no means attached to that. While it does provide a nice way of shipping stuff like readme and examples, it seems that these days it is more common to just distribute a zipped up app folder. And indeed I am aware of the advantages it would have when distributed with butler.

What I am a little concerned about though, in regards to recommending people to install Tiled through the itch.io app and using butler to push folders, is that I lose all system integration. Users won't get their .tmx files associated with Tiled and they won't get any shortcut for launching Tiled in their menu or in the Launcher on macOS. So I hope such things are planned to be addressed soon. :-)

[bjorn]

The logs of the failed .dmg install:

[2017-07-13 @ 22:21:49.673] [butler] butler: Downloading 17 MiB
[2017-07-13 @ 22:21:49.674] [butler] butler: Resuming at 0 B / 17 MiB
[2017-07-13 @ 22:21:53.773] [butler] butler: < 0 B + 17 MiB copied @ 4.2 MiB/s

[2017-07-13 @ 22:21:53.942] [download-watcher] Download ended, err: <none>
[2017-07-13 @ 22:21:53.970] [reactors/tasks] Download finished, starting install..
[2017-07-13 @ 22:21:54.366] [reactors/tasks] Starting install (f156f3b3-570d-46a8-9fff-4c40becced2c)...
[2017-07-13 @ 22:21:54.376] [download-watcher] Cancelling/clearing out last download
[2017-07-13 @ 22:21:54.410] [install] upgrading from build id undefined => build id 37124
[2017-07-13 @ 22:21:54.437] [install] installed mtime = NaN, archive mtime = 1499977313000
[2017-07-13 @ 22:21:54.443] [install/core] retrieving installer type of /Users/bjorn/Library/Application Support/itch/downloads/504366-37124.dmg from cache
[2017-07-13 @ 22:21:54.445] [install/core] invalid installer name stored: undefined - discarding
[2017-07-13 @ 22:21:54.455] [install/core] sniffed type {"ext":"dmg","mime":"application/x-apple-diskimage"} for /Users/bjorn/Library/Application Support/itch/downloads/504366-37124.dmg
[2017-07-13 @ 22:21:54.501] [install/dmg] Preparing installation of '/Users/bjorn/Library/Application Support/itch/downloads/504366-37124.dmg'
[2017-07-13 @ 22:21:55.116] [install/dmg] Found image undefined
[2017-07-13 @ 22:21:55.117] [install/dmg] Found image /Users/bjorn/Projects/tiled-release/Tiled-1.0.2.dmg
[2017-07-13 @ 22:21:55.117] [install/dmg] Done looking for previously mounted images
[2017-07-13 @ 22:21:55.117] [install/dmg] Trying to unlink /Users/bjorn/Library/Application Support/itch/downloads/504366-37124.dmg.cdr
[2017-07-13 @ 22:21:55.189] [install/dmg] Converting archive '/Users/bjorn/Library/Application Support/itch/downloads/504366-37124.dmg' to CDR with hdiutil
[2017-07-13 @ 22:21:55.362] [reactors/tasks] install threw, error: Error: Failed to convert dmg image, with code 1
[2017-07-13 @ 22:21:55.364] [reactors/tasks] Error in task install: Failed to convert dmg image, with code 1
[2017-07-13 @ 22:21:55.365] [reactors/tasks] Install failed for fresh cave, destroying
[2017-07-13 @ 22:21:55.397] [reactors/tasks] Error in install: Failed to convert dmg image, with code 1

[fasterthanlime]

I removed the line because I concluded support was intended from issue #1209. I'll see if I can find any logs, but nothing was popping up at least.

Ah, no need to guess :) https://itch.io/docs/itch/integrating/quickstart.html

What I am a little concerned about though, in regards to recommending people to install Tiled through the itch.io app and using butler to push folders, is that I lose all system integration. Users won't get their .tmx files associated with Tiled and they won't get any shortcut for launching Tiled in their menu or in the Launcher on macOS. So I hope such things are planned to be addressed soon. :-)

They are definitely planned, see this milestone

The logs of the failed .dmg install:

Well, I've never seen a .dmg that hdiutil couldn't convert to cdr (that's the app's way of allowing unattended install of .dmgs - mounting them sometimes pops up dialogs, it's annoying).

Just tried, it converts fine for me:

Mhh... I'll need to keep investigating this

[bjorn]

Well, I've never seen a .dmg that hdiutil couldn't convert to cdr (that's the app's way of allowing unattended install of .dmgs - mounting them sometimes pops up dialogs, it's annoying).

Just tried, it converts fine for me:

Hmm, strange. Two users commented at https://thorbjorn.itch.io/tiled about running into problems installing Tiled on macOS with the itch.io app, so at least I'm not the only one...

[fasterthanlime]

Ok so it's just the exact same issue as the original: the file is named .dmg but is really a .zip. That's a backend problem, we're on it.

[bjorn]

Ah! But that only explains why I am having issues trying to reproduce the issue those two users were facing. But when they wrote their comment, I had not tried to switch the download to a .zip yet...

Also, in the meantime I have put back a proper .dmg, so why isn't the itch.io app picking that up?

[fasterthanlime]

Ah! But that only explains why I am having issues trying to reproduce the issue those two users were facing. But when they wrote their comment, I had not tried to switch the download to a .zip yet...

I believe the issue started 30 days ago, when the first Tiled.dmg was pushed to the macos channel.

The single file feature backfired, and set the filename to Tiled.dmg, even for the app - which always gets a .zip, no matter what's pushed (single file or multiple files) - so the app downloads to a file named Tiled.dmg, then goes trying to figure out what format it is. For dmg in particular, magic numbers aren't enough - some .dmg files are actually bz2 compressed, and those cannot be identifier viat their four first bytes - so the app uses the file extension to guess that particular format instead.

Guessing that it's a .dmg, it does its normal routine (trying to convert it to cdr, mount it, copy files out of it, unmounting it) - but it fails at conversion time because it's not a .dmg, it's a .zip.

The fix is to make the backend use a different field for the "single filename" so that the itch app doesn't pick it up. (/cc @granpc - remember I felt uncomfortable overwriting the filename column? 😉)

That explains both comments and the trouble you're having now (even though you switched back to pushing the .app, the filename set when you first pushed a single file is still there). We'll fix it ASAP.

[GranPC]

The fix for this has been deployed!! Give it a shot and let me know if you run into any issues @bjorn

[bjorn]

@GranPC Thank you so much! I'm currently on holiday but I'll definitely give this a try next week!

[bjorn]

@GranPC Just to let you know, the above revert seems to have gone smoothly and so far I have not heard about any issues. Thanks!

[GranPC]

Awesome! Thanks for letting me know :)

Sent from my phone, please excuse typos

On Oct 31, 2017 11:51 AM, "Thorbjørn Lindeijer" notifications@github.com wrote:

@GranPC https://github.com/granpc Just to let you know, the above revert seems to have gone smoothly and so far I have not heard about any issues. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/itchio/itch/issues/1306#issuecomment-340726577, or mute the thread https://github.com/notifications/unsubscribe-auth/ABUkCJ2uPZWxtsY7V91BQWVY7q_LNkS_ks5sxvuegaJpZM4OUvec .