Fixes #195 and #125

[CalebQ42]

• Fixed an issue in #193 that caused #195 (the ini library apparently automatically adds quotes if there are trailing spaces)
• Fixed #125 by only calling the sudo command if the binfmt file is present.
• Added SquashfsReader to goappimage. This should help with getting fuse working for this.

[probonopd]

Thanks @CalebQ42. Highly appreciated. Happy Easter!

[probonopd]

Now that we can read squashfs, it would be interesting to see if we can get rid of all unsquashfs calls and how that would impact performance.

[CalebQ42]

As it is right now, there shouldn't be any calls to unsquashfs (and there hasn't been since #115). goappimage has the capabilities to fallback to unsquashfs if it has issues, but by default is using my library. I don't believe the check for unsquashfs was ever removed though.

As for performance, my library is 2x~2.5x slower then unsquashfs when completely extracting an image.

[probonopd]

As it is right now, there shouldn't be any calls to unsquashfs

Sorry, my bad. I just saw it is still mentioned in the code.

As for performance, my library is 2x~2.5x slower then unsquashfs when completely extracting an image.

Do you know why that is?

[CalebQ42]

As for performance, my library is 2x~2.5x slower then unsquashfs when completely extracting an image.

Do you know why that is?

I know at least part of it is that unsquashfs is written in C and seems to be very optimized C at that. I did, at one point, do my best to optimize it, and got it down to the ~2x slower point, before that I believe it was closer to ~4x slower. As it is, I would need to do research on how to optimize Go as it's already fully threaded and definitely uses ALL of the CPU when it's doing a full extract (just like unsquashfs does).

The test I've been doing is extracting the Firefox AppImage and the times are about .8 seconds for unsquashfs and 1.8 seconds for my library.

[probonopd]

Are the speed differences the same order of magnitude for zstd? Because that's probably where AppImage will go sooner or later.

[CalebQ42]

So doing some tests with zstd, my library comes to be about 3x~4x slower. My library is a bit slower but unsquashfs runs a bit faster (.5 seconds vs 1.9 seconds).

[probonopd]

I see. So maybe using Go for this purpose is not suitable, or the C code is super optimized?

[CalebQ42]

I'd have to do a bit more research into it to be sure. I am pretty sure that unsquashfs and zstd are both highly optimized C, which would give an inherent speed advantage vs Go, but I'd also like to see straight Go gzip vs Go zstd to see if it's just my library being a bottleneck.

[CalebQ42]

And again I haven't ever done any research optimization. There's probably a decent amount of things that I could do to speed it up.

[CalebQ42]

I wrote a quick test and it seems like the Go libraries for gzip and zstd have about equivalent performance to using command-line tools. In fact it seems the zstd library might be marginally faster (at least at the default compression level). So it definitely seems that my library is the main bottleneck.

[probonopd]

Thanks for the tests @CalebQ42. That sounds encouraging.

[probonopd]

Hi @CalebQ42 there seems to be another new squashfs implementation:

https://github.com/lawl/ayy/tree/master/squashfs

I read about it here:

So why an AppImage now? Because I got upset enough with packaging on linux that I wrote my own package manager for appimages, it only took minimal yak-shaving such as also writing my own squashfs driver.

Maybe you'd like to give this squashfs implementation a performance test?

[lawl]

Maybe you'd like to give this squashfs implementation a performance test?

It's not optimized for speed. ayy is "pretty fast" because it (proudly) does very little.

ayy is optimized for this: https://github.com/lawl/ayy/blob/master/go.mod

the absolute minimal amount of dependencies possible (which is one dependency: openpgp).

[probonopd]

@lawl thanks for chiming in here. We are considering to use a Go implementation of squashfs (with zstd) for a potential future AppImage runtime entirely written in Go. But so far we are not sure whether we can match the current runtime's squashfs performance levels.

[lawl]

I don't think ayy's squashfs library is currently a good fit for that, nor will it likely be in the near future. ayy tries to tackle other problems than writing a new runtime.

[probonopd]

OK, I see. anyhow, interesting project. Feel free to look around in this repo as well. Most likely you have better implementations for many things as this was my first contact to Go when I wrote this.