SeanPesce
commented
2 years ago Thanks for the interest in the project!
My current understanding of the game files is extremely limited - I wrote custom tooling to extract inner files from the *.archive files, and subsequently from the *.pc_resourcelib files (mapped by the *.pc_headerlib files). From there I extracted the bulk of the English data from TextList files and the cutscene subtitles from a few .pc_resourcelib files that were quite puzzling to me. I got my tools working just enough to get the job done, so the code is a mess and there are plenty of parsing issues. The only tool that I feel has reached a releasable state is my archive extractor. A lot of the magic for this project is done by in-game code hooks that replace string data at run-time.
I haven't attempted to extract non-English strings from the game yet, though I agree that it's a good idea and I'd like to have JSON data sets for all the officially-supported languages. The thing is, I'm not actually sure if my copy of the game shipped with the non-English data; I'll have to take a look at some point.
As for the relationships between different files (video -> localized audio/subtitles, etc.) - I don't actually have an understanding of how that works yet. I believe the relations are defined inside scene/blueprint/entity/localization files, but I haven't done any reversing of these sub-formats yet. I'm currently using a brute-force approach with some custom in-game hooks to map video runtime IDs to their respective subtitles ~; I still have 24 cutscene subtitles that I need to find the video IDs for before cutscene translation will work 100%~.
~My immediate goal is to manually map out the last few video IDs to have a "complete" source data set for people to create new translations with. Once that's done,~ I need to create more run-time hooks to load extended fonts (e.g., to support the full Turkish alphabet). Once all that is done I plan to go back to reversing the game files to get a better understanding of the formats and relationships so that I can create tooling and/or documentation for other modders.
As far as dialog flow/state machine data, unfortunately I haven't looked at that at all yet so I don't have any helpful information. Hopefully one day I'll have the time.
Something to keep an eye on (and admittedly I haven't been myself) - the 2021 Guardians of the Galaxy game released by Square Enix runs a newer iteration of the Dawn Engine, so I imagine the game files will have many shared formats. GotG also probably has a bigger modding community than DXMD at the moment, so they might have already surpassed my knowledge of these formats (especially seeing as I've been doing all my work alone). It would probably be helpful to look into any tools/research that GotG modders have released to use them as a reference (and feel free to use this GitHub issue as a place to keep track of any such related projects).
I wish I could be of more help, but unfortunately that's all I know for now. These game files are extremely complex, and due to my limited schedule it takes me a long time to make progress on reversing them. Feel free to add me on Discord (SeanP#5604) if you have binary analysis/reverse engineering experience and plan to start working on DXMD - it's a lonely road out here haha.
KillerBeer01
Hi!
First, I must congratulate you on successfully breaching ( :D ) MD's data file formats. As someone who spent a lot of time trying to figure what's what in HR's .drm files and how to make sense of randomly scattered and mismatched game texts - trust me, I can appreciate the feat. Seeing as even less people (practically noone) makes mods for MD, I believe that its engine is even more convoluted for someone without a native SDK, and my hopes to try my hand at MD's translation were nearing zero... and then I saw your project.
So, there are a few questions I have. First, wouldn't it be too hard to generate strings.json for other languages already present in game? Although the official localization for my language does exist, my previous experience says that there's no such thing as a game translation without glaring blunders, and there are always things to fix.
Second one perhaps would be harder - do you happen to know how to extract voice audio from game's data files into a playable format, and more important, how to link them to texts? So, if I see a string with content.id 1100392264 and text "If this was the only information we had, we probably would call it off.", I could find the respective file and listen to the way this line sounds in game. I firmly believe that hearing the original actors' voice makes a key difference between doing a more or less decent translation and a really "live" one.
Finally, the question that is probably hardest and time consuming, and I'd completely understand if you don't want to go in there, but I have to ask: any insights on where the information about game dialogues is stored and how does it work? That is, I can see in the .json that all lines for a particular dialogue are grouped with the same resource name and id, but not the flow of the dialogue itself. My ultimate goal would be to make something that provides not just the left view but also something closer to the right one, so to speak: https://ibb.co/fqLVFzT
Thanks in advance. You're my only hope, Obi-Wan Kenobi :D .