BFLIM replacement in ALYT files results in a crash

[hlixed]

At the bottom of the ALYTtool repo, you mention "NOTE THAT ACTUALLY, IF YOU REPACK A BFLYT FILE IN AN ALYT FILE, THERE IS SOME ISSUES WITH THE ELEMENT DISPLAYING (or there is not displaying at all...)." For the last week or so, I've been trying to pack a BFLIM, replace it in an ALYT, and then pack that into a GARC. However, it looks like it's crashing every time the specific repacked BFLIM is being loaded. I'm not sure what the problem is; I've tried packing and repacking and it doesn't seem to work.

Other BFLIMS in the same ALYT file load fine - only when the game tries to load the one 180x360 BFLIM that was replaced does it cause problems.

Do you know what I could do to test or otherwise figure out what's causing this problem?

[Tyulis]

The warning was because of some issues in the ALYTtool's bflyt repacker, the 3DSkit's one works fine. Let me try.

[Tyulis]

OK... I think the non-multiple of 8 width is back. I'll must to search more... Did you really got a crash? I tested, and it only freezed on a black screen, with the music in background. Then, can you try with the -v option, and then check if width and height are not inverted? If it is the case, the last commit should fix the problem.

[Tyulis]

On my side, fix confirmed, even on padded textures. After reflexion, textures are always stored with power of two widths and heights, the padding is just to make the packer happy. What about you?

[hlixed]

It looks like the rotation bug is fixed.

I should mention: at https://github.com/Tyulis/3DSkit/blob/master/pack/BFLIM.py#L74, that 8 should be a zero; % 8 means the values go from 0-7, not 1-8.

What file are you editing? I'm editing files in 1.alyt of a/0/9/9 of US.

I also tested by opening the repacked BFLIM in Ohana3ds Rebirth. The original BFLIMs displayed properly, albeit glitchy. BFLIMs packed with 3DSkit do not display at all, citing some memory error, while BFLIMS packed with bflimtool do. Perhaps there's some data section missing that bflimtool is writing?

[Tyulis]

Oops, typo error. Then, your memory error can come from many things. Can you send the terminal output when you convert the bflim? It can always be helpful.

[Tyulis]

I hope this commit will fix the bug. It's looks good for me (tested on 0/6/4-5 in UM)

[hlixed]

I've just tested bflimtool -> pack alyt with 3DSkit -> pack garc with pk3DS, and it still crashed. Here's the ALYT I packed if you'd like to test it; it should go into 1.alyt in a/0/9/9. I'm starting to suspect 3DSkit's ALYT packing now.

[Tyulis]

Did you used the -d option to use paths relative to the given directory when you pack archives? I don't get any issues like this, only a glitchy (!!) BFLIM.

[Tyulis]

For me, it now works perfectly fine. It appears that the right swizzling is needed, look at the terminal output at extraction. I used python3 3DSkit.py -pf BFLIM -O{swizzle=8} <the bflim> (the swizzle value may change) and python3 3DSkit.py -pdf ALYT <folder>/dec_0

[hlixed]

Oh? So you tested this ALYT and it didn't crash? I'd love a screenshot, if possible!

I did use -d. I did this:

python3 3DSkit.py -p $ALYTDIR'/1/timg/intro_hakase_bg_01e.png' -o $ALYTDIR'/1/timg/intro_hakase_bg_01.bflim' -f BFLIM -O{swizzle=8;format=ETC1}
python3 3DSkit.py -p $ALYTDIR'/1/timg/intro_hakase_bg_01e.png' -o $ALYTDIR'/1/timg/intro_hakase_bg_00.bflim' -f BFLIM -O{swizzle=8;format=ETC1}

rm $ALYTDIR'/1/timg/intro_hakase_01smalle.png' $ALYTDIR'/1/timg/intro_hakase_00smalle.png' $ALYTDIR'/1/timg/intro_hakase_bg_01e.png'

python3 3DSkit.py -p -d $ALYTDIR'/1/' -o $ALYTDIR'/1edited.alyt' -f ALYT

(Isn't the directory to use supposed to have the __alyt__ directory in it anyways? Maybe the -d option could be set to use that directory by default?)

[Tyulis]

I know that, but else, the non archive formats cannot have the right output file name. Note that ETC1 is not actually supported for packing, i add an handler for that. I commit the fixes. EDIT: And use comma instead of semicolon for options, I saw that it causes problems on some systems

[hlixed]

Yeah, I saw that ETC1 packing was ignored, but I didn't remove it because the setting was ignored and it defaulted to RGBA8 anyways. Is there are reason you couldn't make -d optional if ALYT is used, but non-optional otherwise? That would be a low-priority nice-to-have feature.

Did you test my ALYT file? If so, could you take a picture of what the intro's graphics looked like when running on a 3ds?

[Tyulis]

I haven't tested this file (because it's the intro graphics, and I'm much farther in my save...), but I tested on 0/6/4, 0/6/5 and 0/7/5, and it appears to be working.

[hlixed]

Would you mind testing that file? You can use JKSM to copy your save file over to a safe location so you can restore it later.

[Tyulis]

I try that.

[hlixed]

(Minor english comment: to describe things you'll do in the future, AKA future tense, use 'I will try that' or 'I'll try that' instead of 'I try that'.)

Thank you!

[Tyulis]

(Oops, that's indeed true...)

[Tyulis]

And as you said, i got a freeze... Another bug! This module is mocking about me...

[hlixed]

Yup! That's what I've been testing this whole time. I wonder whether it's the ALYT packing that's causing the problem, or the BFLIMs causing the problem. That's what I wanted to show you.

Do you think you could help figure out what's causing that freeze?

[Tyulis]

It would be strange that the entire content of the ALYT file loads fine except the edited file. A freeze comes from a bad loading of the file, so it's can be almost anything. I'll do some tests, and try to see if I find the problem.

[hlixed]

Good idea. I wonder whether swapping intro_hakase_00 and intro_hakase_01 will work, for example, since they're the same size. One thing I haven't tried is simply extracting and repacking an ALYT, without ever touching the BFLIMS inside. If that test fails, we know the problem has to do with ALYT repacking instead of BFLIM repacking.

Let me know what tests you run!

[Tyulis]

I tried without touching anything, and it works. I'm actually comparing with ObsidianX's 3dstools, which produce a glitchy but working BFLIM

[hlixed]

Great! So we know the ALYT packing isn't the problem - just BFLIM generation.

What are the exact steps you ran with 3dstools? Could I have the GARC file?

[Tyulis]

I compare the output bflims of both programs. I use the UM a/0/6/5 and 0/9/9 garcs (probably identic in US) I cannot continue this evening, but I managed to get a (very) glitchy but working BFLIM, so that's hopeful.

[hlixed]

Can you upload your a/0/9/9 garc with the glitchy BFLIM and post the link here?

[Tyulis]

I send you the ALYT file (0/9/9, 1.alyt), because uploading the 140MB GARC with my internet connexion would take a while. 1.alyt.zip If you repack it into the GARC, you should THEORICALLY see a sympathic message at the bottom of the screen, saying that it is a test. But IN PRACTICE, it's bugged. Another fix to do in pack.BFLIM. Nice.

[hlixed]

No problem; thanks for the ALYT file.

I see commit 8afe9d8b8bd93f551 mentions GARC and ALYT packing. Have you tested it yet? Does it work without glitchiness? :smile:

[Tyulis]

Thanks to all gods of the world, it WORKS!! Currently, it works just fine if you pack the BFLIM without any swizzling (finally, it doesn't change anything), the edited BFLIM looks good. I commit it instantly!

[Tyulis]

The SARC packer had issues, but it still worked if not edited. That's strange. :

[hlixed]

Wow, great! But what do you mean by "if not edited"?

[Tyulis]

I the ALYT file was repacked without touching anything, it worked, even with a bad file length written in the file and a bad padding. Don't really know why.

[hlixed]

Ah. So you're saying that it's strange the ALYT packer worked even with the bad file length.

I'm trying to replace intro_hakase_00, _01, and _bg_01 with these files. skrelp_pictures.zip

Do you think you could help?

[Tyulis]

No -- It's strange that the game worked even with these issues. Anyway. I tested, and it works with swizzling too. Then if you edit them to make them the size of the originals (256x128 and 300x180 for bg), it should work

[hlixed]

Yeah, that seems necessary. It looks like intro_hakasebg* is 300x180 (the size of Large2.png, which I want to replace both of them) while the others are 256x128, and so they'll need to be cut down.

Thank you for all the help!

(did you commit the ALYT fixes in 9c9123e74c69e50992fbea15dba3cfb6? The message mentions them but I don't see the changed files.)

[Tyulis]

It seems to work ;) I commited all the changes. skrelp_pictures_e.zip

[hlixed]

Wow, amazing. Thank you!

[Tyulis]

No problem ;)

[hlixed]

Not quite; I'm getting a repack error now.

  File "3DSkit.py", line 178, in <module>
    result = main(args, opts)
  File "3DSkit.py", line 139, in main
    pack_files(files, args.out, args.compression, args.format, args.big, args.verbose, opts)
  File "3DSkit.py", line 34, in pack_files
    pack.pack(filenames, output, format, endian, verbose, opts)
  File "<...>/3DSkit/pack/__init__.py", line 10, in pack
    return func(*args)
  File "<...>/3DSkit/util/funcops.py", line 16, in __new__
    return self.main(*args, **kwargs)
  File "<...>/3DSkit/pack/BFLIM.py", line 100, in main
    data = self.repack_data(img)
  File "<...>/3DSkit/pack/BFLIM.py", line 157, in repack_data
    packed = self.pack_pixel(rgba)
  File "<...>/3DSkit/pack/BFLIM.py", line 167, in pack_pixel
    r, g, b, a = rgba
ValueError: not enough values to unpack (expected 4, got 3)

python3 3DSkit.py -p 'intro_hakase_01.png' -o 'intro_hakase_01.bflim' -f BFLIM Do you know what might be causing this?

[hlixed]

Nevermind: This error appears when a PNG file doesn't have an alpha channel. Added one via GIMP, and it packs fine.

[Tyulis]

Yep. Added a little img.convert('RGBA') to fix that.