vasi / squashfuse

FUSE filesystem to mount squashfs archives
Other
288 stars 66 forks source link

Improve byteswap code. #35

Closed kevin-vigor closed 4 years ago

kevin-vigor commented 5 years ago

The generated byteswap macros are extraordinarily inefficient, especially on little-endian machines where they can be no-ops. Replace with nicely optimized linux byteswap macros wheen available.

Inline the __internal versions of the functions, use AC_C_INLINE in configure script to support ancient compilers if necessary.

Net result is that all the generated sqfsswapin* functions (e.g. sqfs_swapin_super_block) now compile down to a no-op on a little-endian linux machine, as one might hope.

vasi commented 4 years ago

Hi! This looks interesting, but also there's a lot more in this PR than it says on the tin. What do you think we should do about that?

kevin-vigor commented 4 years ago

Hi @vasi ! Sorry, I'm not sure how the pull request ended up including all those diffs; it was intended to include only the first one. I think the easiest thing to do would be to drop this pull request, I will resubmit if you think this is worth pursuing.

As you might guess, I have done some significant work on squashfuse to improve performance, most notably implementing multithreaded decompression. So I have a large number of fairly big and hairy changesets to offer. These have been running in production on millions of hosts for over a year, so I have high confidence. Are you interested in pursuing this?

vasi commented 4 years ago

Multi-threaded decompression sounds great!

My main constraint is that I'd like squashfuse to stay build-able on all kinds of systems, including old/weird ones—squashfs is basically an interchange format at this point, and there aren't many good ways of reading it except squashfuse. So ideally we can make threading optional. Does that sound reasonable to you?

vasi commented 4 years ago

(Also, I haven't looked at your whole branch, but really curious if you've tried speculative readahead.)

kevin-vigor commented 4 years ago

Need to clean up patch set, withdrawing this request, will resubmit,