Make things faster?

[LiquidFenrir]

Much much less copying of big structs (seriously, a single entry_s is 1 KB, and entry_list_s is 84 B), using pass-by-reference. Also, chunk loading directory entries instead of one by one, and preallocate entries with realloc instead of one by one. Finally, made a few more things const as I noticed them.

Functionality should be unimpacted by these changes, needs testing and benchmarks though. Note: this might cause merge conflicts in #277, depending on what gets merged first. Fixable for either.

[Zemogiter]

Has anyone successfully compiled Anemone with this PR? I'm getting these errors and I have no idea why since 3ds-lzma is installed.

C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: C:/devkitPro/portlibs/3ds/lib\libarchive.a(archive_read_support_format_zip.o): in function `archive_read_format_zip_cleanup':
archive_read_support_format_zip.c:(.text.archive_read_format_zip_cleanup+0x12c): undefined reference to `lzma_end'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_cleanup+0x140): undefined reference to `BZ2_bzDecompressEnd'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_cleanup+0x154): undefined reference to `ZSTD_freeDStream'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: C:/devkitPro/portlibs/3ds/lib\libarchive.a(archive_read_support_format_zip.o): in function `zip_read_data_zipx_lzma_alone.isra.0':
archive_read_support_format_zip.c:(.text.zip_read_data_zipx_lzma_alone.isra.0+0x58): undefined reference to `lzma_alone_decoder'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.zip_read_data_zipx_lzma_alone.isra.0+0x120): undefined reference to `lzma_code'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.zip_read_data_zipx_lzma_alone.isra.0+0x20c): undefined reference to `lzma_code'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.zip_read_data_zipx_lzma_alone.isra.0+0x30c): undefined reference to `lzma_end'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.zip_read_data_zipx_lzma_alone.isra.0+0x31c): undefined reference to `lzma_end'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.zip_read_data_zipx_lzma_alone.isra.0+0x360): undefined reference to `lzma_end'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: C:/devkitPro/portlibs/3ds/lib\libarchive.a(archive_read_support_format_zip.o): in function `archive_read_format_zip_read_data':
archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x160): undefined reference to `ZSTD_createDStream'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x168): undefined reference to `ZSTD_initDStream'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x170): undefined reference to `ZSTD_isError'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x18c): undefined reference to `ZSTD_DStreamOutSize'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x214): undefined reference to `ZSTD_decompressStream'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x21c): undefined reference to `ZSTD_isError'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x6c0): undefined reference to `BZ2_bzDecompressInit'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x768): undefined reference to `BZ2_bzDecompress'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0x780): undefined reference to `BZ2_bzDecompressEnd'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xa44): undefined reference to `lzma_stream_decoder'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xae4): undefined reference to `lzma_code'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xbac): undefined reference to `lzma_end'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xe50): undefined reference to `BZ2_bzDecompressEnd'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xe60): undefined reference to `lzma_end'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xeb8): undefined reference to `ZSTD_freeDStream'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xee0): undefined reference to `ZSTD_freeDStream'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xf78): undefined reference to `ZSTD_getErrorName'
C:/devkitPro/devkitARM/bin/../lib/gcc/arm-none-eabi/13.2.0/../../../../arm-none-eabi/bin/ld.exe: archive_read_support_format_zip.c:(.text.archive_read_format_zip_read_data+0xfac): undefined reference to `ZSTD_getErrorName'
collect2.exe: error: ld returned 1 exit status
make[1]: *** [/opt/devkitpro/devkitARM/3ds_rules:42: /home/user/Documents/GitHub/Anemone3DS/out/Anemone3DS.elf] Error 1
make: *** [Makefile:227: all] Error 2

[Helloman892]

@Zemogiter Just to double-check, can you compile the latest master?

[LiquidFenrir]

That came with the libarchive portlibs update, fixed (but affects master/the main repo) also, this pr doesn't work, in theory it's better, but tests showed no real improvement. 490003d did improve things but makes usability much worse, so it's not satisfactory.

[Zemogiter]

Thanks. I also had to run pacman -S 3ds-lz4 to make a successfull compilation. I do feel a huge improvement in "Loading themes, please wait" phase, which was always my biggest issue with Anemone. While I have to wait for thumbnails to load, I find it's still quicker than the master branch. However I encountered a crash when trying to access themeplaza from the app. I wasn't able to reproduce it since but the image icon (between the "exit themeplaza mode" and the theme/splash switch) is not working at all. On top of that, icons in themeplaza mode do not show up (I think they're still there just invisible). I was thinking - what if the quick-load of this PR was a special mode that would be activated if you start Anemone with a button pressed? Like when booting up Luma3DS. That way people can choose if they want fast-but-less-stable app. Here's the crash dump: crash_dump_00000002.dmp

@Helloman892 I haven't but given what @LiquidFenrir said, it probably wouldn't work either, I'll try to make a PR if I wont forget about it tomorow

[Helloman892]

@LiquidFenrir If you think this PR reduces usability, please close it; if you have any ideas to improve usability whilst still improving performance, I'm all ears.

[LiquidFenrir]

Well, 490003d is the only one that reduces usability, reverting the part in source/entries_list.c from line (new) 182 to (new) 190 would bring it back to equivalent but better performing as the current master.
It's all a pretty major change though, are you sure you want to go through with it? Oher devs would need getting used to quite a few things, and I don't remember if I tested everything (in the themeplaza browser part)

[Helloman892]

I think merging #290 caused a conflict; kindly resolve this @LiquidFenrir

[LiquidFenrir]

Seen it done, @Helloman892