search

CHollingworth / Lampray

Linux Application Modding Platform. A native Linux mod manager.
https://www.nexusmods.com/baldursgate3/mods/2169
The Unlicense
178 stars 19 forks source link

[BG3] Only one mod is enabled and installed #48

Closed airtonix closed 11 months ago

airtonix commented 11 months ago

Describe the bug

with these mods enabled and deployed

image

These .pak files are copied: 

Larian Studios/Baldur's Gate 3/Mods 
x ls -al 
total 3936
drwxr-xr-x. 1 zenobius zenobius     102 Oct 15 10:56 .
drwxr-xr-x. 1 zenobius zenobius     138 Aug 12 19:18 ..
-rw-r--r--. 1 zenobius zenobius 4004924 Oct 15 10:56 BagsBagsBags.pak
-rw-r--r--. 1 zenobius zenobius    1546 Oct 15 10:56 NoIntroPak.pak
-rw-r--r--. 1 zenobius zenobius   18573 Oct 15 10:56 NoPressAnyKeyMenu.pak

Larian Studios/Baldur's Gate 3/Mods 
> pwd
/home/zenobius/.local/share/Steam/steamapps/compatdata/1086940/pfx/drive_c/users/steamuser/AppData/Local/Larian Studios/Baldur's Gate 3/Mods

this is the Lamp_Data/Mod_lists/BG3.mdf

<?xml version="1.0"?>
<root>
    <Profile Name="Default">
        <Mod ArchivePath="Lamp_Data/Archives/Baldur's Gate 3/No Intro (pak)-680-1-0-1696093119.zip" modType="1" enabled="true" timeOfUpdate="15/10/2023 10:48" />
        <Mod ArchivePath="Lamp_Data/Archives/Baldur's Gate 3/Bags Bags Bags-880-11-1693217745.zip" modType="1" enabled="true" timeOfUpdate="15/10/2023 10:42" />
        <Mod ArchivePath="Lamp_Data/Archives/Baldur's Gate 3/No Press Any Key Menu-745-1-63-1695398869.zip" modType="1" enabled="true" timeOfUpdate="15/10/2023 10:49" />
    </Profile>
</root>

but this is the Lamp_Data/Deployment/Baldur's Gate 3/PlayerProfiles/Public/modsettings.lsx

<?xml version="1.0"?>
<save>
    <version major="4" minor="0" revision="10" build="100" />
    <region id="ModuleSettings">
        <node id="root">
            <children>
                <node id="ModOrder">
                    <children>
                        <node id="Module">
                            <attribute id="UUID" value="3e158fb8-686e-4153-9430-e27f4b9fb7f8" type="FixedString" />
                        </node>
                    </children>
                </node>
                <node id="Mods">
                    <children>
                        <node id="ModuleShortDesc">
                            <attribute id="Folder" type="LSString" value="GustavDev" />
                            <attribute id="MD5" type="LSString" value="" />
                            <attribute id="Name" type="LSString" value="GustavDev" />
                            <attribute id="UUID" type="FixedString" value="28ac9ce2-2aba-8cda-b3b5-6e922f71b6b8" />
                            <attribute id="Version64" value="36028797018963968" type="int64" />
                        </node>
                        <node id="ModuleShortDesc">
                            <attribute id="Folder" type="LSString" value="BagsBagsBags.pak" />
                            <attribute id="MD5" type="LSString" value="" />
                            <attribute id="Name" type="LSString" value="BagsBagsBags.pak" />
                            <attribute id="UUID" type="FixedString" value="3e158fb8-686e-4153-9430-e27f4b9fb7f8" />
                            <attribute id="Version64" value="36028797018963968" type="int64" />
                        </node>
                    </children>
                </node>
            </children>
        </node>
    </region>
</save>

but if i go through a process of deploying each one individually in order to get the right xml values for each mode i can end up with a manually created modesettings.lsx : 

<?xml version="1.0"?>
<save>
    <version major="4" minor="0" revision="10" build="100" />
    <region id="ModuleSettings">
        <node id="root">
            <children>
                <node id="ModOrder">
                    <children>
                        <node id="Module">
                            <attribute id="UUID" value="12a1fc9f-e39d-43f2-ab49-25026f073e46" type="FixedString" />
                        </node>
                        <node id="Module">
                            <attribute id="UUID" value="5c8badce-e6a2-48f4-ac9c-55ba2c33a7bc" type="FixedString" />
                        </node>
                        <node id="Module">
                            <attribute id="UUID" value="3e158fb8-686e-4153-9430-e27f4b9fb7f8" type="FixedString" />
                        </node>
                    </children>
                </node>
                <node id="Mods">
                    <children>
                        <node id="ModuleShortDesc">
                            <attribute id="Folder" type="LSString" value="GustavDev" />
                            <attribute id="MD5" type="LSString" value="" />
                            <attribute id="Name" type="LSString" value="GustavDev" />
                            <attribute id="UUID" type="FixedString" value="28ac9ce2-2aba-8cda-b3b5-6e922f71b6b8" />
                            <attribute id="Version64" value="36028797018963968" type="int64" />
                        </node>
                        <node id="ModuleShortDesc">
                            <attribute id="Folder" type="LSString" value="NoPressAnyKeyMenu.pak" />
                            <attribute id="MD5" type="LSString" value="" />
                            <attribute id="Name" type="LSString" value="NoPressAnyKeyMenu.pak" />
                            <attribute id="UUID" type="FixedString" value="5c8badce-e6a2-48f4-ac9c-55ba2c33a7bc" />
                            <attribute id="Version64" value="36028797018963968" type="int64" />
                        </node>
                        <node id="ModuleShortDesc">
                            <attribute id="Folder" type="LSString" value="NoIntroPak.pak" />
                            <attribute id="MD5" type="LSString" value="" />
                            <attribute id="Name" type="LSString" value="NoIntroPak.pak" />
                            <attribute id="UUID" type="FixedString" value="12a1fc9f-e39d-43f2-ab49-25026f073e46" />
                            <attribute id="Version64" value="36028797018963968" type="int64" />
                        </node>
                        <node id="ModuleShortDesc">
                            <attribute id="Folder" type="LSString" value="BagsBagsBags.pak" />
                            <attribute id="MD5" type="LSString" value="" />
                            <attribute id="Name" type="LSString" value="BagsBagsBags.pak" />
                            <attribute id="UUID" type="FixedString" value="3e158fb8-686e-4153-9430-e27f4b9fb7f8" />
                            <attribute id="Version64" value="36028797018963968" type="int64" />
                        </node>
                    </children>
                </node>
            </children>
        </node>
    </region>
</save>

and then do one final enable all and deploy to get mods copied.

Error message

no error messages

To Reproduce Steps to reproduce the behaviour:

  1. Build Lamp 1.0.8
  2. Run Lamp 1.0.8
  3. set paths to bg3
  4. download these mods
  5. Drag the downloaded zip files on the Lamp UI
  6. click enable on all
  7. set all rows mod type to Standard Mod
  8. Click on Deploy
  9. ls -al ~/.local/share/Steam/steamapps/compatdata/1086940/pfx/drive_c/users/steamuser/AppData/Local/Larian Studios/Baldur's Gate 3/Mods to confirm .pak files are there
  10. cat ~/.local/share/Steam/steamapps/compatdata/1086940/pfx/drive_c/users/steamuser/AppData/Local/Larian\ Studios/Baldur\'s\ Gate\ 3/PlayerProfiles/Public/modsettings.lsx
  11. See that only one of the desired mod is listed

Expected behaviour

All deployed mods should be listed in ~/.local/share/Steam/steamapps/compatdata/1086940/pfx/drive_c/users/steamuser/AppData/Local/Larian\ Studios/Baldur\'s\ Gate\ 3/PlayerProfiles/Public/modsettings.lsx

Screenshots If applicable, add screenshots to help explain your problem.

System Info (please complete the following information):

Lamp 1.0.8 (built from this commit: https://github.com/airtonix/Lamp/commit/66d5f73b61c07661bc1fbaa6e70fb05803113711 using ./setup.sh && act push

> inxi -S -G
System:
  Host: brass Kernel: 6.4.4-200.fc38.x86_64 arch: x86_64 bits: 64
    Desktop: GNOME v: 44.3 Distro: Fedora release 38 (Thirty Eight)

Graphics:
  Device-1: NVIDIA GA102 [GeForce RTX 3090 Ti] driver: nvidia v: 535.54.03
  Display: x11 server: X.Org v: 1.20.14 with: Xwayland v: 22.1.9 driver: X:
    loaded: modesetting,nouveau,nvidia unloaded: fbdev,vesa
    gpu: nvidia,nvidia-nvswitch resolution: 1: 3440x1440~60Hz
    2: 3440x1440~60Hz 3: N/A
  API: OpenGL v: 4.6.0 NVIDIA 535.54.03 renderer: NVIDIA GeForce RTX 3090
    Ti/PCIe/SSE2

Additional context Add any other context about the problem here.

CHollingworth commented 11 months ago

Unfortunately this is due to the way these mods are packaged, they dont contain the required metadata in the places we expect, while bg3mm can extract them with the divinity tools there is sadly no linux equivalent. I am however working on an extraction method with quickbms which works with the v18 packfiles but nothing older than that. 1.0.9 should be able to pull this information as currently we can only pull from files that include the metadata (info.json) or earlier pak files that keep their metadata in plaintext within the pak.

GoNZooo commented 11 months ago

Unfortunately this is due to the way these mods are packaged, they dont contain the required metadata in the places we expect, while bg3mm can extract them with the divinity tools there is sadly no linux equivalent. I am however working on an extraction method with quickbms which works with the v18 packfiles but nothing older than that. 1.0.9 should be able to pull this information as currently we can only pull from files that include the metadata (info.json) or earlier pak files that keep their metadata in plaintext within the pak.

Having this in mind, has Larian/someone documented the format used for the PAK files anywhere? I'd like to write parsers for them myself but I can't seem to find any documentation on them written down. Am I correct in thinking that lslib would be the place to read this in source format if it's not explicitly documented anywhere?

airtonix commented 11 months ago

Of the libraries you're using for Lamp. the only library I'm even a little bit familiar with is ImgUI (but from the golang end).

So I don't understand any of what you said or why it affects the fact that deploying three mods results in only one of them randomly ending up in the modsettings.lsx.

If it's the library causing the issue of clobbering the generated modsettings.lsx, then my peasant brain would look to solve this with generating a modsettings.lsx for each mod and then writing our own bit of glue to merge them.

maybe I completely misunderstand the problem?

CHollingworth commented 11 months ago

So the problem right now is unless the metadata is provided alongside the mod, we have to extract it, See lampParse.h, we're scrapping it out via plaintext which only sort of works.

There is no documentation persay but here's what I've scraped together, it's lz4 frames and you can unpack current v18 using this quickbms script https://github.com/smacx250/BG3/blob/main/misc_script/getBg3PakMeta.bms

There is also the windows c# libary called lslib which can be built on linux (with the linux port fork) but I couldn't get functioning https://github.com/Norbyte/lslib

https://github.com/pimzero/lslib/tree/linux-port

On Sun, 15 Oct 2023, 22:56 Zeno Jiricek, @.***> wrote:

Of the libraries you're using for Lamp. the only library I'm even a little bit familiar with is ImgUI (but from the golang end).

So I don't understand any of what you said or why it affects the fact that deploying three mods results in only one of them randomly ending up in the modsettings.lsx.

If it's the library causing the issue of clobbering the generated modsettings.lsx, then my peasant brain would look to solve this with generating a modsettings.lsx for each mod and then writing our own bit of glue to merge them.

maybe I completely misunderstand the problem?

— Reply to this email directly, view it on GitHub https://github.com/CHollingworth/Lamp/issues/48#issuecomment-1763514712, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARGLQVOYAIJ6Y7UQUIWILXLX7RL2TANCNFSM6AAAAAA6ATSHV4 . You are receiving this because you commented.Message ID: @.***>

CHollingworth commented 11 months ago

Unfortunately this is due to the way these mods are packaged, they dont contain the required metadata in the places we expect, while bg3mm can extract them with the divinity tools there is sadly no linux equivalent. I am however working on an extraction method with quickbms which works with the v18 packfiles but nothing older than that. 1.0.9 should be able to pull this information as currently we can only pull from files that include the metadata (info.json) or earlier pak files that keep their metadata in plaintext within the pak.

Having this in mind, has Larian/someone documented the format used for the PAK files anywhere? I'd like to write parsers for them myself but I can't seem to find any documentation on them written down. Am I correct in thinking that lslib would be the place to read this in source format if it's not explicitly documented anywhere?

If you do please let me know, id love to use it in Lamp isntead of my current idea of downloading quickbms in the background

CHollingworth commented 11 months ago

Well. Issue closed, i was going to give up but ive written the parser this gave me the kick to get it done and done right. Thank you all. v1.0.9 will have this fixed

CHollingworth commented 11 months ago

Ive also released the canary for those not wanting to build, as always its on the discord.