sinbad / SPUD

Steve's Persistent Unreal Data library
MIT License
336 stars 51 forks source link

Support for FText and FMap property #30

Closed Nesquick0 closed 1 year ago

Nesquick0 commented 2 years ago

I tried to add support for FText which should be just copy of FName.

Second is support for FMap serialization. I tried to do it similar to FArray so there is a lot copy/paste and that it will fit into your systems without any major changes. I have to figure out how to save property type unique which will contain both key type and value type. I added ESST_Max = 64 and calculate it as "KeyType * ESSTMax + ValueType". I think it gives enough space for more ESST types and still be unique for any type of key/value types combination.

Also added tests for FText and some TMap types combination.

sinbad commented 2 years ago

Thanks again for the contributions!

I've merged the FText support initially since that's simple. I need to think about the map support a little more - the key/value type combination might be the best option - although I'd personally make ESST_Max = 128 to maximise the use of the range of a uint16 (max would be 127 * 128 + 127 = 16383).

However, I need to consider a bit more whether this is the correct approach since this is a numerical trick that can only be used once. Maybe there's a more explicit way to handle it that doesn't preclude other compound types / containers using the same approach later.

I'm stepping away from the computer for the holidays after today though so I don't have time right now to give it the thought it deserves, I'll come back to it in the new year. Of course if you have any alternative thoughts in the mean time please add them here.

Nesquick0 commented 2 years ago

Sure, thank you for your time. You are right that might be problem.

Without other changes I don't think ESST_Max can be 128 because there is already ESST_Unknown = 0x0F00 (3840). After I created this pull request I found out that currently it is fine but if there are 63 types then 6463 = 4032 (0x0FC0) so without changing ESST_Unknown there can only be ESST_Max = 61 so 6061+60 = 3720. Also when doing those calculation I found out that I missed that types begin with 0 (I thought that it starts with 1). It is probably not common to have key in map as ESST_UInt8 but I'm not sure about Enum. And whether it can cause problems that "0*64+something" will have same value type as just something.

sinbad commented 2 years ago

Really good points - I think this reinforces the thinking that maybe we shouldn't abuse this enum as a cheat to encode a map. I'm thinking that it really needs a specific ESST_Map entry, with its own nested encoding of the key/value types - it doesn't match the ESST_ArrayOf modifier but with good reason I think.

sinbad commented 1 year ago

As of a441974dbe8b8a34d69d5ca2ee19a5592249e217, Maps are supported as well as arrays of custom structs. I took a different approach and wrapped "difficult" types in a UE serialisation record, meaning we can now support everything - although because the record is opaque to us, things like cross-references between objects aren't restored in these cases.