More templates, store raw parameter data (Colors)

[SKmaric]

This allows for retaining of the unchanged bytes relating to parameters in the case of the entire template not yet being completed, simply by knowing how long this data is for each object, and then reinserted into the .orc when saving without risk of losing data. Additionally this makes it easier to research the parameters for making of complete templates.

[blueskythlikesclouds]

Reading raw data and using them to save object is a neat idea, but what about objects that have uint arrays and strings in them? They have offsets.

[SKmaric]

I've already thought about that, for the time being I've put in dummy lengths that should work temporarily, at least as far as preventing loss of data. We'll need to look into it a bit more and add new parameter data types for HedgeLib to read from templates.

[thesupersonic16]

Do we need to delete all the objects with no templates? And I know SCSE deletes them anyways. I honestly don't like the changes in SOBJ but we will see what Rad would say.

[SKmaric]

Oh no, that's only for when saving files. Actually what I've done is allow objects with no templates to be loaded even without parameter data, so I added the Purge function as to not cause problems when these are saved. This is also why I got the idea to store raw data without proper templates, so these objects could remain untouched before work on their templates was started.

As for the other changes to SOBJ.cs it's just git being weird since I put a few existing functions in If brackets. Maybe I should comment them better...

Edit: alright, I changed this function to skip instead of delete entirely, apologies for the confusion.

[thesupersonic16]

Atleast with your changes we can now complete the first stage. :)

[blueskythlikesclouds]

I would wait for Rad to review this.

[thesupersonic16]

same.

[SKmaric]

I still find it odd the way git treated some of my changes to SOBJ.cs, since I've now put a few existing bits of code inside If brackets (related to different behaviours for objects with/out templates and templates which only define data lengths rather than actual parameters). It thinks I completely replaced the code and makes the diff look like a mess.

[thesupersonic16]

There must be something wrong with HedgeEdit.

[thesupersonic16]

I think i found the issue.

[thesupersonic16]

Should be fixed now.

[thesupersonic16]

You might want to merge the last commit unless you got a better solution.

[SKmaric]

On a different note I've made a spreadsheet to more clearly document the progress of Colors' templates: Google Docs

Let me know what you think and if it's alright and can be incorporated somehow I'll open it up for editing.

[thesupersonic16]

Looks nice.

[Radfordhound]

Sorry for the delay, I've been really busy lately and just didn't have the time to sit down and go through all of this.

I really like the idea actually, though yeah, there's definitely a few things I'd want to change about it's execution.

Right now I see it's set to get the raw length (in bytes) of all of the object's parameters from the template, then read all of them, then jump back to the beginning of the parameters and read any known parameters also defined in the template again.

phew here goes

While this definitely seems to work fine, I see it as a little inefficient. I think a better solution, if this isn't too much to ask, would be to have the template instead define an integer like "AdditionalBytes" that could be used to read all remaining bytes in the object's parameter list that aren't already defined by the template.

This way HedgeLib wouldn't have to jump around and keep re-reading data just because the template isn't done, but the system would still work the same in the end.

[thesupersonic16]

It seems most/all of the requested changes are been solved.

[SKmaric]

Thanks for reviewing my PR. I've now done my best to make these changes as requested.

My solution to the "AdditionalBytes" function is to read the full length integer as provided by the .xml, as before, but to then calculate how many bytes are remaining to be read/written based on how many known parameters have been defined, using the values provided by BaseStream.Position. I have also left in the original method as a boolean toggle in case there is any need to have the complete set of bytes loaded, which now also supports editing and saving like the new method.

[SKmaric]

It is worth noting that one of my other key changes is that HedgeLib can now load objects even if they don't have a template, where it will continue to load everything excluding parameter data. The "IsTemplateExists" was an easy solution to identifying which objects lacked proper parameter information, so they could be excluded from saving to prevent errors in-game.

I did remove this skipping function entirely for now, because I was admittedly having difficulty getting the Write subprogram to access the objectTemplates dictionary without significantly altering other parts of the lib (inb4 some easy solution because I'm still relatively unfamiliar with C#)

[thesupersonic16]

Doesn't HedgeLib already read objects without templates? EDIT: Nevermind.

[thesupersonic16]

Also can't you check if there is no params? like if (obj.Parameters.Count == 0)

[thesupersonic16]

Note sure if the problem is in SCSE or in HedgeLib or if it just me, But It failing to read data for "RawParamData".

[SKmaric]

It might be because there are no "remaining" bytes left, and they're all loaded properly as parameters. If you check the raw data for another object type there may be some with 2 or 3 bytes.

[thesupersonic16]

It seems to be a problem with reading the templates. Which could be a problem in SCSE.

[SKmaric]

Alternatively search for "bool rawDataMode = false" in SOBJ.cs and change it to true, this will cause it to load the complete data regardless of which parameters are defined in the template.

[SKmaric]

But yeah, make sure the templates are updated since I changed the attribute name to "RawByteLength" from "RawBitLength" in all template xmls.

[thesupersonic16]

ok, Found the issue.

[thesupersonic16]

Its looking for RawByteLength, But the templates doesn't contain an element with the type of RawByteLength, But there is RawBitLength

[thesupersonic16]

I'll have to get VS to copy the new templates.

[thesupersonic16]

wow you are good at creating templates, I struggle to make one.

[thesupersonic16]

We should really merge this PR.

[Radfordhound]

My apologies for another late response. Just like with last time, I've been busy and just wanted to really make sure I could sit down and look through all of the changes.

Thanks a ton, I've looked through the changes and this all looks great! I've merged it. :)