kotiq / wt-tools

War Thunder resource extraction tools
29 stars 3 forks source link

blk_unpack_ng - IndexError: list index out of range #14

Closed PwaDesu closed 1 year ago

PwaDesu commented 1 year ago

I think this started happening after the latest Fire & Ice WT update, it was working as intended before. The .vromfs unpacked do work. It doesn't immediately fail as soon as blk_unpack_ng starts, it seems to be working fine until after a dozen or so of seconds it encounters a hurdle and this happens.


Error: Command failed: C:\Users\funxg\AppData\Local\Programs\Python\Python37\python.exe "D:\Apps\wt-tools\src\wt_tools\blk_unpack_ng.py" --format=strict_blk "C:\Program Files (x86)\Steam\steamapps\common\War Thunder\aces.vromfs.bin_u"
Traceback (most recent call last):

  File "D:\Apps\wt-tools\src\wt_tools\blk_unpack_ng.py", line 170, in <module>

    main()

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\click\core.py", line 829, in __call__

    return self.main(*args, **kwargs)

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\click\core.py", line 782, in main

    rv = self.invoke(ctx)

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\click\core.py", line 1066, in invoke

    return ctx.invoke(self.callback, **ctx.params)

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\click\core.py", line 610, in invoke

    return callback(*args, **kwargs)

  File "D:\Apps\wt-tools\src\wt_tools\blk_unpack_ng.py", line 166, in main

    process_dir(path, out_type, is_sorted)

  File "D:\Apps\wt-tools\src\wt_tools\blk_unpack_ng.py", line 126, in process_dir

    process_slim_dir(dir_path, names, out_type, is_sorted)

  File "D:\Apps\wt-tools\src\wt_tools\blk_unpack_ng.py", line 143, in process_slim_dir

    process_slim_dir(path, names, out_type, is_sorted)  # @r

  File "D:\Apps\wt-tools\src\wt_tools\blk_unpack_ng.py", line 143, in process_slim_dir

    process_slim_dir(path, names, out_type, is_sorted)  # @r

  File "D:\Apps\wt-tools\src\wt_tools\blk_unpack_ng.py", line 145, in process_slim_dir

    process_file(path, names, out_type, is_sorted)

  File "D:\Apps\wt-tools\src\wt_tools\blk_unpack_ng.py", line 86, in process_file

    root = bin.compose_slim_data(names, istream)

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\blk\binary\constructor.py", line 406, in compose_slim_data

    return BlockAdapter(BlockCon, names).parse_stream(istream)

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\construct\core.py", line 313, in parse_stream

    return self._parsereport(stream, context, "(parsing)")

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\construct\core.py", line 325, in _parsereport

    obj = self._parse(stream, context, path)

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\construct\core.py", line 710, in _parse

    return self._decode(obj, context, path)

  File "C:\Users\funxg\AppData\Local\Programs\Python\Python37\lib\site-packages\blk\binary\constructor.py", line 215, in _decode

    init = names[offset] if tag else self.parse_params_data(String, offset)

IndexError: list index out of range
kotiq commented 1 year ago

I unpacked the whole WarThunder/cache/binary.2.21.1 ver 2.21.1.30 tree using the v0.2.2.10 build and got no error.

Looks like a mismatch in the name map for the file. Please provide a sample of the data that is causing the error.

PwaDesu commented 1 year ago

I am not unpacking the .vromfs in the cache directory as those are not the latest, I unpack those directly in the WarThunder directory. In this case, it was trying to unpack blks from the newly created aces.vromfs.bin_u folder.

I just tried redoing the same process, but targeting the cache directory instead, and that just completed without errors. I take it the aces.vromfs.bin in the normal directory somehow is slightly different than the one in the cache directory, and thus some of the files it extracted out of it got corrupted? Here's the vromfs I had extracted and was trying to unpack the .blk's of. It's the most up to date one, the one in my cache directory is still on yesterday's patch. aces.vromfs.zip

kotiq commented 1 year ago

I again failed to reproduce the error.

sha1sum aces.vromfs.bin
7108f1c1730991f2e783ec177ab9bf13561d2fdd  aces.vromfs.bin

python -m wt_tools.vromfs_unpacker aces.vromfs.bin 
Unpacking files  [####################################]  100%
[OK] /tmp/dds/aces.vromfs.bin => /tmp/dds/aces.vromfs.bin_u

python -m wt_tools.blk_unpack_ng --format=strict_blk aces.vromfs.bin_u/ > aces.log

grep -Hn Error aces.log
PwaDesu commented 1 year ago

Right, I'm very sorry for wasting your time, the reason unpacking the cache directory worked was that I had never unpacked it before, so all of the files were from this update and this update only. I had unpacked the main directory many times before however and kept unpacking into the same folders, so I assume there were leftover, now deleted files from previous updates and those were the ones causing the error because they didn't come from the current .vromfs

I deleted the folders in order to unpack into clean, brand-new folders, and it worked as expected