CHollingworth / Lampray

Linux Application Modding Platform. A native Linux mod manager.
https://www.nexusmods.com/baldursgate3/mods/2169
The Unlicense
184 stars 19 forks source link

Mods without `info.json` are not deployed at all #14

Closed alterNERDtive closed 1 year ago

alterNERDtive commented 1 year ago

If an archive contains just a naked .pak file with no info.json, Lamp ignores it.

CHollingworth commented 1 year ago

That's a strange one, that seems to not be the case for me. I'll check the predeployment step.

On Fri, 15 Sept 2023, 08:28 alterNERDtive, @.***> wrote:

If an archive contains just a naked .pak file with no info.json, Lamp ignores it.

— Reply to this email directly, view it on GitHub https://github.com/CHollingworth/Lamp/issues/14, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARGLQVJ4ZVGTAQJUM5YKT5TX2P7SPANCNFSM6AAAAAA4ZMHF3Y . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Azurion42 commented 1 year ago

I'm noticing this too. It'll send the .pak files to the right folder, but the modsettings.lsx file only has the files with .json files in the zips with them.

LAMP Terminal Output.txt Game Folder modsettings.txt LAMP Folder modsettings.txt

LT-Batman commented 1 year ago

If an archive contains just a naked .pak file with no info.json, Lamp ignores it.

While you wait for a fix for this. You can rename the .pak file to .txt and open it. (May take a while to completely open/load depending on your CPU / RAM) Once it's fully loaded, ignore the text it fails to show due to encryption, and simply use the find tool (search) for " UUID. " It will show you the JSON script within the .pak file that you'll need to manually change the .modsettings file yourself. If youre unsure of how to rewrite the modsettings file or unsure of how to force a load order inside the modsettings file just ask.

edit: Alternatively you could use one of the pak reading tools that are available to simply unpack, read it, and make your own json file or you could use the method i mentioned before to just copy the information to make your own json file so that the management tool can load the pak file for you. Dont forget to change it back to a .pak file and not to edit anything within it. If you do be sure to NOT SAVE when you close it out.

(Also keep in mind that after it completes loading it may take a little while again for the search to work properly. Also, that some people's coding can be weird or messy so they may end up encrypting the json information but you should still be able to gather the key info needed such as the uuid, name and folder in most cases)

CHollingworth commented 1 year ago

If an archive contains just a naked .pak file with no info.json, Lamp ignores it.

While you wait for a fix for this. You can rename the .pak file to .txt and open it. (May take a while to completely open/load depending on your CPU) Once it's fully loaded, ignore the text it fails to show due to encryption, and simply use the find tool (search) for " UUID. " It will show you the JSON script within the .pak file that you'll need to manually change the .modsettings file yourself. If youre unsure of how to rewrite the modsettings file or unsure of how to force a load order inside the modsettings file just ask.

edit: Alternatively you could use one of the pak reading tools that are available to simply unpack, read it, and make your own json file or you could use the method i mentioned before to just copy the information to make your own json file so that the management tool can load the pak file for you. Dont forget to change it back to a .pak file and not to edit anything within it. If you do be sure to NOT SAVE when you close it out.

(Also keep in mind that after it completes loading it may take a little while again for the search to work properly. Also, that some people's coding can be weird or messy so they may end up encrypting the json information but you should still be able to gather the key info needed such as the uuid, name and folder in most cases)

Thank you, ill have to create something to search for it, i was under the assumption that pak files without would not need this as the mods that i use dont, ill add it to the next hotfix.

CHollingworth commented 1 year ago

Im reopening this issue for anyone with experience with Quick BMS, or if anyone's found a way to decompile pak files successfully on linux as this will be the only way i can extract the needed information. The windows platform already has plenty of tools for extracting pak files. i think its time i build one in. ive got the out of date version of BG3.bms however it seems to stumble on post early access mod files.

Azurion42 commented 1 year ago

Im reopening this issue for anyone with experience with Quick BMS, or if anyone's found a way to decompile pak files successfully on linux as this will be the only way i can extract the needed information. The windows platform already has plenty of tools for extracting pak files. i think its time i build one in. ive got the out of date version of BG3.bms however it seems to stumble on post early access mod files.

I've been using a tool called ExportTool , I just added it to steam and start it using proton experimental. ConverterApp.exe is the file to start.

LT-Batman commented 1 year ago

Im reopening this issue for anyone with experience with Quick BMS, or if anyone's found a way to decompile pak files successfully on linux as this will be the only way i can extract the needed information. The windows platform already has plenty of tools for extracting pak files. i think its time i build one in. ive got the out of date version of BG3.bms however it seems to stumble on post early access mod files.

I use ExportTool as well when the .pak file is too messy to simply convert to .txt and read it. It's pretty self-explanatory. I use Wine to utilize it. Granted I could use about any Windows application with Wine(other mod managers)-- I still manually installed my mods instead of using the other mod managers. Im super weird about memory and storing things I hardly use when I can do it myself.

pepper-jk commented 1 year ago

@CHollingworth The lslib, which is linked above as ExportTool, appears to work on moving to .Net Core, which would enable native linux support down the road. https://github.com/Norbyte/lslib/issues/137 https://github.com/Norbyte/lslib/pull/139 For now, I also used the latest release on wine and it worked fine for extracting .pak archives.

I would also suggest in looking at their code for this and maybe implement a way for your mod manager to extract the UUID automatically. Being able to add .pak mods without info.json would be the main reason - for me at least - to consider using a mod manager. Extracting UUIDs manually is the biggest hassle of manual mod management right now.

CHollingworth commented 1 year ago

im currently trying to extract the pak files headder as it should contain the data needed without unpacking the entire pak, Using the QuickBMS extraction script as inspiration, ive looked through the linux fork of lslib and if all else fails ill be trying to make a c++ rewrite of the extraction tools

alterNERDtive commented 1 year ago

I've been using a tool called ExportTool , I just added it to steam and start it using proton experimental. ConverterApp.exe is the file to start.

F me. Just tried getting it to run manually through wine (no dice), then remembered it was mentioned here. Why does Proton just work™? -.-

CHollingworth commented 1 year ago

Little Update on this as its my primary issue right now, i made a build of @pimzero's Linux Port of LSLib, im not sure if ive made a mistake but ive only ended up with windows libraries however the source code has shown me ive been going the wrong way about it. If possible through i would very much like to get in contact with @EKey as they made the original QuickBMS script which would be alot easier to implement,

CHollingworth commented 1 year ago

Unfortunately whilst i can extract UUIDs from the file, which is the fix i am going to implement, this fails to get the correct uuid some of the time, for now im going to close this until i can get LSLib working on linux,

alterNERDtive commented 1 year ago

Hmm. I would definitely not close this, as it’s a deal breaker for me (and I’d assume lots of other people). I’d even pin it, probably.

While I’m here, I’m excited for the next release :)

Edit: just realized it is pinned. Still think closing it will lead to confusion / send the wrong message though.

pepper-jk commented 1 year ago

Unfortunately whilst i can extract UUIDs from the file, which is the fix i am going to implement, this fails to get the correct uuid some of the time, for now im going to close this until i can get LSLib working on linux,

Could you share your code? or setup? for extracting the UUIDs? Maybe another set of eyes might help spotting the mistake.

CHollingworth commented 1 year ago

It's just searching through the pak files as text looking for anything that matches a uuid using regex. The first one about 8 times out of ten is the one we want.

On Wed, 20 Sept 2023, 17:28 Jens Keim, @.***> wrote:

Unfortunately whilst i can extract UUIDs from the file, which is the fix i am going to implement, this fails to get the correct uuid some of the time, for now im going to close this until i can get LSLib working on linux,

Could you share your code? or setup? for extracting the UUIDs? Maybe another set of eyes might help spotting the mistake.

— Reply to this email directly, view it on GitHub https://github.com/CHollingworth/Lamp/issues/14#issuecomment-1728074465, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARGLQVPGVZFOW6YSP2MJRXTX3MKTXANCNFSM6AAAAAA4ZMHF3Y . You are receiving this because you modified the open/close state.Message ID: @.***>

CHollingworth commented 1 year ago

In addition I've currently tried, QuickBMS, but that only works on v15 pak files and patch 4 uses v18. Lslib linux port but it doesn't yield linux usable Libaries and its written in c# so I can't do I direct inclusion. I've also tried direct extraction with lz4 frames however this hasn't yielded any useable results.

On Wed, 20 Sept 2023, 18:29 Charles H, @.***> wrote:

It's just searching through the pak files as text looking for anything that matches a uuid using regex. The first one about 8 times out of ten is the one we want.

On Wed, 20 Sept 2023, 17:28 Jens Keim, @.***> wrote:

Unfortunately whilst i can extract UUIDs from the file, which is the fix i am going to implement, this fails to get the correct uuid some of the time, for now im going to close this until i can get LSLib working on linux,

Could you share your code? or setup? for extracting the UUIDs? Maybe another set of eyes might help spotting the mistake.

— Reply to this email directly, view it on GitHub https://github.com/CHollingworth/Lamp/issues/14#issuecomment-1728074465, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARGLQVPGVZFOW6YSP2MJRXTX3MKTXANCNFSM6AAAAAA4ZMHF3Y . You are receiving this because you modified the open/close state.Message ID: @.***>

CHollingworth commented 1 year ago

Once i update the master branch, it will be under BG3.cpp findJsonData

LT-Batman commented 1 year ago

Once i update the master branch, it will be under BG3.cpp findJsonData

Im terribly late but .json files with the UUID needed are only manifested if people use the multitool to pack their mods. If they use LSLib to do so, it simply converts it to a .pak and calls it a day. So in order to 100% get the correct UUID needed every time you need to direct your code to follow this path: %ModName% / Mods / %ModName% / meta.lsx 95-98% of all mods need a meta.lsx (meta.lsf as well but lsx is readily readable) file for the game to read.

People are also taught that its best to name your mod & mod folder the same so there should hardle ever be an issue with directing this tree.

If something still fails then a broad search for meta.lsx should do or an intricate search of reading available lsx files whom's code starts like this:

<version major="4" minor="0" revision="0" build="5" />
    <region id="Config">
        <node id="root">
            <children>
                <node id="Dependencies" />
                <node id="ModuleInfo">
                    <attribute id="Author" type="LSWString" value="MWaR" />
                    <attribute id="CharacterCreationLevelName" type="FixedString" value="" />
                    <attribute id="Description" type="LSWString" value="Aziza's Blade'" />
                    <attribute id="Folder" type="LSWString" value="SL_W" />
                    <attribute id="GMTemplate" type="FixedString" value="" />
                    <attribute id="LobbyLevelName" type="FixedString" value="" />
                    <attribute id="MD5" type="LSString" value="" />
                    <attribute id="MainMenuBackgroundVideo" type="FixedString" value="" />
                    <attribute id="MenuLevelName" type="FixedString" value="" />
                    <attribute id="Name" type="FixedString" value="SL_W" />
                    <attribute id="NumPlayers" type="uint8" value="4" />
                    <attribute id="PhotoBooth" type="FixedString" value="" />
                    <attribute id="StartupLevelName" type="FixedString" value="" />
                    <attribute id="Tags" type="LSWString" value="" />
                    <attribute id="Type" type="FixedString" value="Add-on" /> _**### (unique to the majority of lsx files in a mod to alert you of this being the source UUID that follows)**_
                    <attribute id="UUID" type="FixedString" value="5e017dc5-da42-4923-b4d7-742138a739b0" />**_### (The source UUID needed to add to the modsettings file)_**
                    <attribute id="Version" type="int32" value="1" />
                    <children>
                        <node id="PublishVersion">
                            <attribute id="Version" type="int32" value="1" />
                        </node>
                        <node id="Scripts" />
                        <node id="TargetModes">
                            <children>
                                <node id="Target">
                                    <attribute id="Object" type="FixedString" value="Story" />
                                </node>
                            </children>
                        </node>
                    </children>
                </node>
            </children>
        </node>
    </region>
</save>
alterNERDtive commented 1 year ago

Slightly off topic, but wtf even are .pak files? re-inventing the tarball wheel?

pimzero commented 1 year ago

Hi, quick unrelated remark (given I had a notification, but do not have context on this project/issue)

Little Update on this as its my primary issue right now, i made a build of @pimzero's Linux Port of LSLib, im not sure if ive made a mistake but ive only ended up with windows libraries however the source code has shown me ive been going the wrong way about it.

Note that the libraries are dll files, but .Net, not native PE files. It should be possible to use them in a linux native binary with Mono, although I haven't tried.

Note that my lslib port to linux does 2 main things: allow building lslib with a Linux system, and remove dependency on native libraries, to be able to run with Mono (otherwise the project only works under Wine)

LennardF1989 commented 1 year ago

re-inventing the tarball wheel?

Aren't all games ;)