Path Decoding - Githubissues

hexabits commented 2 years ago

Decoding for .pathtype, .pathmaterial, .pathextrusion, .pathsupport, .pathresource, .pathjoinpartresource, .supportset

TODO:

[X] A few other filetypes in other OVL related to paths
[x] Remove any unnecessary padding definitions
[x] Figure out injection tracebacks for .pathjoinpartresource

GAME STATUS:

[X] .pathextrusion injecting, no crashes
[X] .pathmaterial injecting, no crashes
[X] .pathresource injecting, no crashes
[X] .pathsupport injecting, no crashes
[X] .pathtype injecting, no crashes
[x] .pathjoinpartresource injecting, no crashes (see comments)
[X] .supportset injecting, no crashes (see comments)

hexabits commented 2 years ago

Edit: See https://github.com/OpenNaja/cobra-tools/pull/217#issuecomment-1179451455

4 .pathjoinpartresource files still do not inject on latest master, they appear to be the same bug.

The other 21 .pathjoinpartresource files inject fine and appear to work in game.

INFO | Creating facilityjoin_12x8.pathjoinpartresource in STATIC
ERROR | Could not create: facilityjoin_12x8.pathjoinpartresource
Traceback (most recent call last):
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\Pointer.py", line 108, in write_pointer
    self.write_template()
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\Pointer.py", line 116, in write_template
    self.frag.struct_ptr.write_instance(self.template, self.data)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl\compound\HeaderPointer.py", line 118, in write_instance
    cls.to_stream(self.pool.data, instance)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\path\compound\PointsList.py", line 59, in to_stream
    cls.write_fields(stream, instance)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\path\compound\PointsList.py", line 46, in write_fields
    stream.write_bytes(instance.padding)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\base\basic.py", line 83, in write_values
    instance = np.array(instance, dtype)
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl\__init__.py", line 721, in create_file
    loader.create()
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\modules\formats\BaseFormat.py", line 365, in create
    self.header.write_ptrs(self, self.root_ptr, self.file_entry.pool_type)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\MemStruct.py", line 161, in write_ptrs
    self.handle_write(prop, pointer, struct_ptr, loader, pool_type)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\MemStruct.py", line 144, in handle_write
    self.handle_write(prop, val.data, val.frag.struct_ptr, loader, pool_type, is_member=True)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\MemStruct.py", line 122, in handle_write
    val.write_ptrs(loader, struct_ptr, pool_type, is_member=is_member)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\MemStruct.py", line 164, in write_ptrs
    self.handle_write(prop, array, struct_ptr, loader, pool_type)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\MemStruct.py", line 125, in handle_write
    self.handle_write(prop, member, struct_ptr, loader, pool_type, is_member=True)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\MemStruct.py", line 122, in handle_write
    val.write_ptrs(loader, struct_ptr, pool_type, is_member=is_member)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\MemStruct.py", line 161, in write_ptrs
    self.handle_write(prop, pointer, struct_ptr, loader, pool_type)
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\MemStruct.py", line 142, in handle_write
    val.write_pointer()
  File "C:\Users\Jon\Documents\GitHub\cobra-tools\generated\formats\ovl_base\compound\Pointer.py", line 110, in write_pointer
    raise TypeError(f"Failed to write pointer data {self.data} type: {type(self.data)} as {self.template}")
TypeError: Failed to write pointer data PointsList [Size: 0, Address: 8688]
        * points = [Vector3 [Size: 12, Address: 8688]
                * x = -3.9000000953674316
                * y = -2.0
                * z = 5.849999904632568
        , Vector3 [Size: 12, Address: 8700]
                * x = -3.9000000953674316
                * y = -2.0
                * z = -5.900000095367432
        , Vector3 [Size: 12, Address: 8712]
                * x = 3.9000000953674316
                * y = -2.0
                * z = -5.900000095367432
        , Vector3 [Size: 12, Address: 8724]
                * x = 3.9000000953674316
                * y = -2.0
                * z = 5.849999904632568
        ]
        * padding = None
 type: <class 'generated.formats.path.compound.PointsList.PointsList'> as <class 'generated.formats.path.compound.PointsList.PointsList'>

hexabits commented 2 years ago

.supportset injects fine, and appears to work in game, but <connector_1> and <connector_2> still create Pointers on re-inject even if they are None.

<connector_1 pool_type="4" />

<connector_2 pool_type="4" padding="0" num_joints="0" unk_float_1="0.0" unk_int_1="0">
    <model_name />
    <joints />
</connector_2>

hexabits commented 2 years ago

@HENDRIX-ZT2 Re: the 4 .pathjoinpartresource files not injecting, it's related to the dynamic padding on two of my compounds:

<add name="padding" type="byte" arr1="(#ARG# % 2) * 8" />

<add name="padding" type="byte" arr1="(#ARG# % 4) * 4" />

I understand it's not necessary, and I've already removed them, but it highlights a bug with the current ndarray code, where an arr1="0" (0 being calculated or literal) of basic types will cause a NoneType exception.

hexabits commented 2 years ago

The game doesn't seem to mind the None pointers for these, but still bringing it up:

FILE [ 101 |  10400] (  16) parkentrancejoinpart.pathjoinpartresource
  PTR @ 0    -> SUB [ 99 |  12368] (  80)
    PTR @ 0    -> SUB [ 101 |   4544] (  32)
    PTR @ 8    -> SUB [ 101 |   4576] (  32)
    PTR @ 16   -> SUB [ 101 |  10416] (  16)
    PTR @ 24   -> SUB [ 101 |  10432] (  16)
    PTR @ 48   -> SUB [ 68 |  13152] (  16)

And after re-injection

FILE [ 103 |   6432] (  16) parkentrancejoinpart.pathjoinpartresource
  PTR @ 0    -> SUB [ 103 |   6448] (  80)
    PTR @ 0    -> SUB [ 103 |   6528] (  32)
    PTR @ 8    -> SUB [ 103 |   6560] (  32)
    PTR @ 16   -> SUB [ 103 |   6592] (  16)
    PTR @ 24   -> SUB [ 103 |   6608] (  16)
    PTR @ 32   -> SUB [ 103 |   6624] (  16)
      PTR @ 0    -> SUB [ 103 |   6640] (  80)
        PTR @ 0    -> SUB [ 103 |   6720] (  16)
        PTR @ 8    -> SUB [ 103 |   6736] (  16)
        PTR @ 16   -> SUB [ 103 |   6752] (  16)
        PTR @ 24   -> SUB [ 103 |   6768] (  16)
        PTR @ 32   -> SUB [ 103 |   6784] (  48)
        PTR @ 48   -> SUB [ 69 |   2867] (  17)
    PTR @ 48   -> SUB [ 69 |   2851] (  16)

hexabits commented 2 years ago

Also we already discussed this, but for posterity:

FILE [ 103 |    288] (  16) pathsupport_01.pathsupport
  PTR @ 0    -> SUB [ 69 |    297] (   9)

FILE [ 103 |    304] (  16) pathsupport_02.pathsupport
  PTR @ 0    -> SUB [ 69 |    306] (  11)

The .pathsupport zstr do not get padded to 16 bytes like in vanilla.

hexabits commented 2 years ago

Regarding https://github.com/OpenNaja/cobra-tools/pull/217#issuecomment-1179464822, these have a num field that I can use for forcing the pointers to be None, which I've done in c94e880

HENDRIX-ZT2 commented 2 years ago

For the none issue on the padding, this can be fixed by reverting the ndarray from_xml code to its older state.

hexabits commented 2 years ago

For the none issue on the padding, this can be fixed by reverting the ndarray from_xml code to its older state.

Wasn't the ndarray code needed for basic arrays that were non-empty? So then instead of 4 failing and 21 succeeding it would be reversed.

OpenNaja / cobra-tools

Path Decoding #217