search

Norbyte / lslib

Tools for manipulating Divinity Original Sin and Baldur's Gate 3 files
MIT License
732 stars 135 forks source link

Support for Elder Scrolls Online GR2 #34

Open CactusBack opened 4 years ago

CactusBack commented 4 years ago

Hey Norbyte, could you, please, take a look at the GR2 files from the Elder Scrolls Online and see if LSLib could possibly support converting them?

There seems to be 3 types of GR2 files in this game: one for meshes, one for skeletons and one for animations.

Here are examples of all three types of files - https://www.mediafire.com/file/fh7u6jx4jvp3qth/GR2_examples.rar/file.

Not sure if it will be of any help but here's also the granny2_x64.dll from the game folder - https://www.mediafire.com/file/kfwifp1suh81oe7/granny2_x64.rar/file.

Finally, here's a screenshot of the error message I get trying to import ESO GR2 files into GR2 Converter (LSLib v1.14.1) - https://monosnap.com/file/yM7LNNmKXCS63gsX8meiXUNZtvayyC

fireundubh commented 4 years ago

Why? What is the point?

CactusBack commented 4 years ago

Why? What is the point?

I found this thread - https://github.com/Norbyte/lslib/issues/13 - and from Norbyte's replies there I assumed that they could work on support for other games.

If that is not the case, I apologize and this thread could be closed.

fireundubh commented 4 years ago

ESO is an online-only game; it can't be modified. You offered an objection to my query, so in the absence of an answer, I can only come to one of two conclusions: you're looking for a tool to help you steal assets, or you're looking for shallow exploits. I don't speak for Norbyte or LaughingLeader, but I doubt we're very much interested in facilitating. Correct me if I'm wrong.

CactusBack commented 4 years ago

ESO is an online-only game; it can't be modified. You offered an objection to my query, so in the absence of an answer, I can only come to one of two conclusions: you're looking for a tool to help you steal assets, or you're looking for shallow exploits. I don't speak for Norbyte or LaughingLeader, but I doubt we're very much interested in facilitating. Correct me if I'm wrong.

Oh, I misunderstood your question then.

I'm not interested in exploits and if by stealing you mean extracting models to sell them, then no, I'm not interested in that either.

The initial reason why I started extracting ESO models was just to look at them. I love that game and the Elder Scrolls universe and wanted to look at the characters in detail out of game. Later I thought other players might want a closer look at the models as well, so I started uploading pictures and videos of them online for everyone to see - https://esomodelviewer.com/

I can export meshes already but unfortunately they are without skeletons. So my main goal now is to find a way to export them with skeletons and to convert GR2 animations. That would allow me to showcase ESO characters as close as possible to how they appear in-game.

Obviously, you are not obligated to help me in any way and might not even be able to do it even if you wanted since ESO GR2 format might be very different from Divinity GR2. I do understand that so it's all good. But thought I would ask here anyway just in case.

fireundubh commented 4 years ago

Someone else will need to look at the file format because I don't have 125 GB of free space to spare right now.

CactusBack commented 4 years ago

Someone else will need to look at the file format because I don't have 125 GB of free space to spare right now.

Sure, totally understandable. Just curious - why 125 GB though? The current game size is 84.5 GB.

Norbyte commented 4 years ago

Although this is mainly a tool for Divinity, if it is possible to add GR2 support for other games with relatively low amount of work then I'm not against it. This one looks like a vertex format mismatch issue, i.e. a format base GR2 already supports but I haven't added that particular configuration yet, so it should be relatively easy I think. I'll look at it.

CactusBack commented 4 years ago

Great, thank you for looking into this, Norbyte!

fireundubh commented 4 years ago

Just curious - why 125 GB though? The current game size is 84.5 GB.

The ESO installer requires 125 GB of free space to install the game. I freed up 200 GB last night; however, I'm near my ISP's bandwidth cap. But it looks like Norbyte is on this. Cool!

CactusBack commented 4 years ago

The ESO installer requires 125 GB of free space to install the game.

Ah right, I see then.

CactusBack commented 4 years ago

Hey Norbyte, any progress on this?

thunderysteak commented 4 years ago

I have submitted a pull request that should give it a compatibility with ESO. I was able to import Yaghra Spider, Molag Bal and piece of armor from ESO into 3DS Max to confirm the fixes. It exports with correct UVs and functional skeleton.

3dsmax_2020-04-26_03-10-21 3dsmax_2020-04-26_03-20-57 3dsmax_2020-04-26_03-22-36

CactusBack commented 4 years ago

I have submitted a pull request that should give it a compatibility with ESO. I was able to import Yaghra Spider, Molag Bal and piece of armor from ESO into 3DS Max to confirm the fixes. It exports with correct UVs and functional skeleton.

That's amazing, thank you! Can't wait to try it.

Norbyte commented 4 years ago

Thanks for the update, I've merged the PR. I've also created a v1.14.2 release which contains all changes up to this version.

thunderysteak commented 4 years ago

Thanks

Also just discovered that there are still big incompatibilities in ES:O that will throw an error. For example, the normal mapping being in real16 with two arrays so I'll try to see if its an easy fix as the last one. Didn't catch it yesterday due of a small sample of models out of 85k.

There are also animation files that throw "Curve does not contain position data" exceptions or cause crashes due of how the files are constructed. Some specific bones do not have curve position data like "root" but others have, which I think might cause this issue since I don't have any other non-ESO GR2s to compare against. I have an example for the data:

Data sample from 1002407.gr2 which is an "DB_KneelDead_Fall_Loop" animation file, located in TrackGroups -> TransformTracks

"Bip01" contains this data for PositionCurve.CurveData:

"mshBn_spine02" on the other hand contains this data for PositionCurve.CurveData:

Call stack when an animation file causes a crash:

>   LSLib.dll!LSLib.Granny.Model.CurveData.DaConstant32f.GetMatrices() Line 29  C#
    LSLib.dll!LSLib.Granny.Model.CurveData.AnimationCurveData.GetQuaternions() Line 233 C#
    LSLib.dll!LSLib.Granny.Model.CurveData.AnimationCurveData.ExportKeyframes(LSLib.Granny.Model.KeyframeTrack track, LSLib.Granny.Model.CurveData.AnimationCurveData.ExportType type) Line 273 C#
    LSLib.dll!LSLib.Granny.Model.TransformTrack.ToKeyframes() Line 607  C#
    LSLib.dll!LSLib.Granny.Model.ColladaExporter.ExportKeyframeTrack(LSLib.Granny.Model.TransformTrack transformTrack, string name, string target) Line 480 C#
    LSLib.dll!LSLib.Granny.Model.ColladaExporter.ExportTrack(LSLib.Granny.Model.TransformTrack track) Line 571  C#
    LSLib.dll!LSLib.Granny.Model.ColladaExporter.ExportTracks(LSLib.Granny.Model.TrackGroup trackGroup) Line 581    C#
    LSLib.dll!LSLib.Granny.Model.ColladaExporter.ExportAnimations(LSLib.Granny.Model.Animation animation) Line 604  C#
    LSLib.dll!LSLib.Granny.Model.ColladaExporter.Export(LSLib.Granny.Model.Root root, string outputPath) Line 643   C#
    LSLib.dll!LSLib.Granny.Model.Exporter.SaveDAE(LSLib.Granny.Model.Root root, LSLib.Granny.Model.ExporterOptions options) Line 218    C#
    LSLib.dll!LSLib.Granny.Model.Exporter.Save(LSLib.Granny.Model.Root root, LSLib.Granny.Model.ExporterOptions options) Line 231   C#
    LSLib.dll!LSLib.Granny.Model.Exporter.Export() Line 790 C#
    LSLib.dll!LSLib.Granny.GR2Utils.SaveModel(LSLib.Granny.Model.Root model, string outputPath, LSLib.Granny.Model.Exporter exporter) Line 86   C#
    LSLib.dll!LSLib.Granny.GR2Utils.ConvertModels(string inputDirectoryPath, string outputDirectoryPath, LSLib.Granny.Model.Exporter exporter) Line 119 C#
    ConverterApp.exe!ConverterApp.GR2Pane.GR2BatchConvertBtn_Click(object sender, System.EventArgs e) Line 438  C#
    [External Code] 
    ConverterApp.exe!ConverterApp.Program.Main() Line 20    C#

IMO a quick band-aid fix could be added to add a toggle to just make the exporter ignore any file that has zero skeletons or meshes and just the "GrannyRootBone" model and animation for non-Divinity model exports.

I literally started playing around with ES:O files only yesterday so I'm still trying to figure out things on my own.

@CactusBack is there a way how to contact you directly on Discord, IRC, Steam or similar? You might be able to help me understand the ESO file structure better (if there exists one even).

CactusBack commented 4 years ago

@CactusBack is there a way how to contact you directly on Discord, IRC, Steam or similar? You might be able to help me understand the ESO file structure better (if there exists one even).

Sure, my Discord username is Cactus_Back#0088, my Steam is https://steamcommunity.com/id/cactusback and you can also email me at nikita@esomodelviewer.com, whatever you prefer.

It does seem like ESO doesn't have the defined file structure on our end though. At least the tool I use to extract files (https://www.esoui.com/downloads/info1258-EsoExtractData.html) only creates folders and files with numbers as names and many folders share the same type of assets (e.g. .gr2 files are spread through a number of folders) so it's hard to make a sense of that.