WoW mpqs not fully supported

[Warkdev]

Hello, I'm trying out this library to open archive for World of Warcraft Vanilla. However, it seems not all the files are correctly read from the archives. Is there something different ? Thanks, Warkdev

[Frotty]

World of Warcraft archives currently aren't fully supported/tested. The high block table(for very big mpqs that wc3 didnt have) isn't implemented and WoW might use an unsupported compression format. You can help by providing more specific examples of what doesn't work including a stacktrace, then we might be able to implement it. @DrSuperGood also knows more about WoW mpqs I believe

e: The same basically goes for starcraft (2) mpqs. Feel free to contribute for their support.

[DrSuperGood]

I am working on rewriting the hash table and block table code. When it is all done it should support both deleted hash table entries and extended block table entries (the high block table).

If it is a missing compression algorithm then it might be easy to implement if one can identify which.

It is also possible that not all of the code is big file aware currently. I believe some WoW MPQs exceeded 2 GB so such code might not work correctly currently. An on going effort is being made to fix such code.

[Warkdev]

Hello, thank you for the answer. I've no error as such but I do have different results for the list of files and the extracted content between this library and Stormlib. For instance, the DBC file LiquidType.dbc isn't shown from the dbc.mpq file with this library while it does with Stormlib. As well, the extracted AreaTable.dbc does 97kb while the one from Stormlib is 125kb.

I'm open to contribute to this if you let me know where I could start digging :)

[Warkdev]

Well, that's my bad, for Vanilla it seems working properly. I've tested with another editor. :)

But I'm still willing to help if needed !

[DrSuperGood]

The files being different sizes might be the result of a missing compression algorithm. The missing files are either a result of incorrect list file parsing, or that Stormlib uses a more expansive standard list file (includes hidden WoW files).

[Frotty]

I think he was simply using one of the other jmpqs, judging from his forks 🌴 If you find other non working mpqs feel free to report/upload smallest possible example.

[Warkdev]

Hello, well the fact is that I was not using the right MPQ file. In WoW, you have MPQ containing DBC (Database Client) files which may be overwritten by other MPQ.

For instance, AreaTable.dbc is 97kb weight in dbc.MPQ but 125kb in patch-2.MPQ. LiquidType.dbc was found inside patch.MPQ.

Still available if you need help as I'm interested by this :)

[Frotty]

Ah that makes sense. I believe MPQEditor automatically searches for patch mpqs (either always or for known mpqs) and replaces them in the view. I suppose there could be an additional open-option that searches for patch mpqs and overwrites files accordingly, but it's not high priority for now.

If you want to help you could try if other mpqs work with jmpq, e.g. diablo 1/2, starcraft, warcraft 2, WoW the burning crusade (i think the high block table came into existance here), etc. We already know that the pkware exploder, sparse and bzip2 decompression are missing, so those files won't be extractable. Ideally you would upload one mpq per not-working file-type, containing 1 example file. Then we can use it as a test object in the testing suite.

[Warkdev]

Well, no it doesn't but I did it manually :)

I'm currently working with terrain.MPQ and there I do have a difference. It should contain WDT and ADT files (as indicated by MPQeditor) but it doesn't show WDT files. Is the last PR dealing with this eventually ? Where can I upload the terrain.MPQ ?

[Frotty]

MPQEditor has the function tho, on opening an mpq you can specify patch files.

I believe those mpqs are quite big and probably shouldn't uploaded copyright-wise. Maybe you can make the mpq smaller by deleting files in mpqedit, so that it's small, but jmpq still can't read it. Then upload to some hoster. Unless of course the big size is what breaks it..

[Warkdev]

I should compare but I did not see any issue reading patch.MPQ (1 889 591kb) while terrain.MPQ seems incomplete (1 073 977 Kb).

I'll compare patch.MPQ content to see whether the size is the problem but I've my doubts.

[Warkdev]

Guys, I should sleep more.. The file is complete but I had an ending '0' in my String which made it not comparable.

Sorry to disturb you...

[Frotty]

No problem. It would be nice if you would leave a star if jmpq helped you :)

[Warkdev]

Yeah, definitely ! I fork, I star, I love ! Thank you guys :) Do you plan to have other libraries to read & manage files extension that are in there ?

Tal'

[Frotty]

I'm not sure how much wc3 and WoW share, but for some things like mdx models, blp textures there are libraries yeah