Request: Modding wineapps using phoenicis

Kreyren commented 5 years ago

no it dosent but get you fivem work for you

Assumig that we have GTA V script in phoenicis try that one, else file a new issue i will contrib it -> Do not resolve GTA V issues here.

Originally posted by @Kreyren in https://github.com/PhoenicisOrg/scripts/issues/914#issuecomment-487298908

Requesting phoenicis to implement mod manager function to allow modding of phoenicis-apps.

ABSTRACT

Modding of phoenicis-apps on demand using GUI simmilar to ModOrganizer/Webiums WoT modpack/Vortex, etc..
Crowdsource what mods are compatible with what configuration to ensure compatibility and possible fixes from upstream or forks.
Ability to make modpacks using phoenicis.
Provide platform for modding of phoenicis-apps that encourages people to contribute with open-source modifications that is sanitized to ensure compatibility.
Resolve issues with engine missing per-modification functions in engine (wine).

Current

Has to be done using non-standart and frankenstained configuration of phoenicis script to work.
Crowdsourcing is problematic and so is compatibility

ISSUES

Modular mods are mandatory

We need a way to enable/disable mods for demand so that end-user can cherrypick which mods to use.

Wine compatibility is important

We can't really use steamplay (excluding that it's property of wine instead of FOSS) for everything for example league of legends currently needs patched glibc and patched wine (https://bugs.winehq.org/show_bug.cgi?id=47198) + DXVK through dgvoodoo wrapper or DXV9 to get performance supperior to native neither of which is possible in proton.

Assuming that simmilar or worse issue will be present with mods like FiveM (https://github.com/PhoenicisOrg/scripts/issues/914) how do we resolve it?

Expecting a method to deliver wine, wine patches, workarounds on demand based on which mods are used.

Workarounds (https://github.com/PhoenicisOrg/scripts/tree/master/Engines/Wine/Verbs) are handled by phoenicis, but can not be uninstalled on demand which is issue assuming scenario where game with mod breaks if previously installed workaround is present.
Wine Patches League Of Legends example above needs glibc and wine patches where glibc patches likely break whole system (broke mine untill gcc changed.) and wine patches which needs to be modular so that they can be applied based on game and mod configuration.

How this could be implemented?

Sandboxing?

TODO

Compiling engine?

TODO

Providing binnaries?

TODO

~~Would personally contrib sh!@$# ton of mod configuration for wineapps..~~

Kreyren commented 5 years ago

@plata @qparis I believe that this can be easily done using new includes with checkbox-like options for mods simmilar to World Of Tanks mods alike:

Meaning checkbox-like menu + preview

qparis commented 5 years ago

I don't really understand the goal of the issue. What is the problem having one script per app/mod?

Kreyren commented 5 years ago

@qparis wanna maintain +- 12000 scripts just for skyrim and it's different configurations instead of having one master script that can rule them all?

qparis commented 5 years ago

The master script will be 12000 harder to maintain so I do not see the point. The more we separate the responsibilities, the easier the maintenance is

Kreyren commented 5 years ago

not really you can just define that file A goes to path A (change downloaded file if needed) -> and thats it..

mod maintainance is not hard task + you can usually use standartized builds which doesn't even need those definitions.

qparis commented 5 years ago

Then, all the mods scripts will be very simple, I really don't see how it can help to have everything in the same script. You are just loosing metadata for nothing.

Kreyren commented 5 years ago

@qparis Scenario:

GTA V == GAME FiveM == Modification

Expected GUI which asks the user if he/she wants modding this game -> if yes:

Output GUI like

with description for available mods with checkbox-like menu to make modding super easy..

Mod scripts won't be easy if you have to download game all the time.. you can in theory make one sub-directory in game with scripts to install mods from which phoenicis would read.

qparis commented 5 years ago

Mod scripts does not mean that you have to download the game all the time. A script is perfectly allowed to require and patch an existing prefix. So I don't see the advantage.

Kreyren commented 5 years ago

@qparis example?

qparis commented 5 years ago

Why do you want an example? A script can use an existing wineprefix, this is a fact

Kreyren commented 5 years ago

@qparis What if end-user wants two games with different mods?

Kreyren commented 5 years ago

Assuming that skyrim is practically game engine for new games at this point alike: https://enderal.com/

qparis commented 5 years ago

I really don't understand the point, sorry... If a user wants two games, they install two games. If they want two mods, they install two mods. In each situation, a game or a mod = a script. This is as simple as it

Kreyren commented 5 years ago

@qparis example then:

mod A B C D E F with GAME

end-user wants A B E F for GAME then A B C F because E is not compatible with C.

where end-user wants one game with two custom modpacks and where B depends on vcrun2017 like https://github.com/PhoenicisOrg/scripts/issues/914 with

Why can't you just make it easy for everyone to mod their wineapps using known user-friendly gui instead of frankenstaining scripts together assuming that i understand your proposal.

qparis commented 5 years ago

Currently, a game can’t be installed twice in different wineprefix. If a mod has compatibility conflicts (which should be a rare case), we consider it a standalone game and not an extension

Kreyren commented 5 years ago

If a mod has compatibility conflicts (which should be a rare case)

Very very very common..

Currently, a game can’t be installed twice in different wineprefix.

Is not expected assuming that we can make those mods modular to be activated on demand for one game that is used as source which shoudn't be hard to do..

we consider it a standalone game and not an extension

Which is what this issue is about, standalone game is not sufficient for those usecases! -> allow modular moding using custom GUI

qparis commented 5 years ago

If a mod has compatibility conflicts (which should be a rare case)

Very very very common..

Any concrete example? Why would there any conflict with wine and not on Windows?

Currently, a game can’t be installed twice in different wineprefix.

Is not expected assuming that we can make those mods modular to be activated on demand for one game that is used as source which shoudn't be hard to do..

If a given mod has serious conflict with a game, I do not see how it could be done. And in any case, if it is that simple, I would simply deactivate every mod except the one that is being run, which convinces me that we should have 1 mod per script.

we consider it a standalone game and not an extension

Which is what this issue is about, standalone game is not sufficient for those usecases! -> allow modular moding using custom GUI

Kreyren commented 5 years ago

Any concrete example? Why would there any conflict with wine and not on Windows?

Not just on wine this is also issue on windows where two mods are not compatible with each other and require patching AND/OR changes. -> Very common in game modding.

If a given mod has serious conflict with a game, I do not see how it could be done. And in any case, if it is that simple, I would simply deactivate every mod except the one that is being run, which convinces me that we should have 1 mod per script.

There are patches or changes in configuration of said mod to work around this issue that can be incorporated in phoenicis to make this process just mashing button instead of wasting hours, days to fix it..

In theory you can provide metadata to check which mod is compatible with what

qparis commented 5 years ago

In this case I think you have two possibilities: -> A mod should run alone : in this case, we need a shortcut and a dedicated script -> The user wants/needs to run several mods at the same time. In this case, we need to make configuration panel extendable.

So it depends on what we are talking about

Kreyren commented 5 years ago

-> The user wants/needs to run several mods at the same time. In this case, we need to make configuration panel extendable.

this

assuming that for enderal which is game build on skyrim and end-user wants to apply additional mods to it (which is possible) which standalone script will not suffice.

Kreyren commented 5 years ago

Something alike

If end-user wants to mod his/her wineapp

qparis commented 5 years ago

Enderal needs its own script. Like skyrim, it can be moded. The panel extension can be shared between these two apps

Kreyren commented 5 years ago

@qparis enderal is special case so using it as standalone might be sufficient assuming that end-user can add mods on top of it.

plata commented 5 years ago

This issue basically requests to implement a mod manager. Normally, I expect this to be available for Windows already (like it's the case for Skyrim, WoT etc.). Any example where manual mod management is really required?

Kreyren commented 5 years ago

Any example where manual mod management is really required?

@plata everytime available and everytime when it's not

plata commented 5 years ago

I can understand your point but I have two issues with it:

If a mod manager is missing, it would make more sense to implement it separately of Phoenicis such that other users can also benefit from it.
Can we even know how the mods should be handled? E.g. in your WoT example, there are groups of mods which can only be used exclusively (like crosshairs). I cannot imagine how this could be implemented in a generic way.

Kreyren commented 5 years ago

If a mod manager is missing, it would make more sense to implement it separately of Phoenicis such that other users can also benefit from it.

Disagree, it beeing a phoenicis function would be much better..

Can we even know how the mods should be handled? E.g. in your WoT example, there are groups of mods which can only be used exclusively (like crosshairs). I cannot imagine how this could be implemented in a generic way.

Example: $GAMEROOT/image/kreyren.png

Mod options: $GAMEROOT/image/kreyren.png with option to add random rainbows $GAMEROOT/image/kreyren.png with option to add something else $GAMEROOT/image/kreyren.png with option to implement new function that handles abcd..

plata commented 5 years ago

Where are the images?

Kreyren commented 5 years ago

Where are the images?

Example of files that can be modded. -> You are just adding/replacing files.. whats hard about that?

plata commented 5 years ago

Sorry, I don't get your point.

Kreyren commented 5 years ago

@plata https://en.wikipedia.org/wiki/Sandbox_(computer_security)

I'm suggesting to make all verbs including mods and patches to wine sandboxed so that we can remove them and install on demand for solution of this issue.

plata commented 5 years ago

I know what a sandbox is... Nevertheless, I do not see how this could be applied for a verb/patch. You cannot know which files will be installed or modified by a patch nor is it possible to separate them from a Wine prefix. Unless you can prove otherwise, I believe this is technically impossible.

Kreyren commented 5 years ago

Option A

@plata You cannot know which files will be installed or modified by a patch nor is it possible to separate them from a Wine prefix.

What's difficult on making two IDENTICAL wine builds.

Apply patch on either of them

diff it, results are definitions for sandbox.

Option B

@plata You cannot know which files will be installed or modified by a patch nor is it possible to separate them from a Wine prefix.

Make definitions for original wine build, everything that doesn't match those definitions are files to be sandboxed.

Option C

@plata You cannot know which files will be installed or modified by a patch nor is it possible to separate them from a Wine prefix.

Make a new mount directory with option --rbind

That won't apply changes by patch into a source.

Option D

@plata You cannot know which files will be installed or modified by a patch nor is it possible to separate them from a Wine prefix.

Make phoenicis to execute portage or paludis that slots required wine, it's configuration and dependencies for engine usage and apply patches accordingly.

plata commented 5 years ago

What's difficult on making two IDENTICAL wine builds.

You're confusing me. Are we talking about patches for an app inside a Wine prefix or about patches to Wine itself?

Kreyren commented 5 years ago

@plata

I'm suggesting to make all verbs including mods and patches to wine sandboxed (@Kreyren )

Both depending on wineapps used. Same logic could be applied to wineapp itself.

plata commented 5 years ago

When talking about patches to Wine itself, you normally refer to patches to the source code which means they must be compiled. Therefore we cannot apply the same logic.

Kreyren commented 5 years ago

@plata ....

you can precompile them..

plata commented 5 years ago

Proof of concept please.

Kreyren commented 5 years ago

@plata

# Create new directory
mkdir $HOME/kreyren 

# Example source
echo "ABCDE" >> $HOME/kreyren/SOURCE" 

# Example patch 
[ $(ls $HOME/kreyren/SOURCE) == "ABCDE" ] && echo "ABCDEFGH" >> $HOME/kreyren/SOURCE" 

# The difference is "FGH" -> if FGH remvoved from SOURCE it will remove patch.

Real world example: These are differences between my wine-staging-4.[6-7]-rc_p1 : https://gist.githubusercontent.com/Kreyren/dc7054e16a99730dd63d04dfc14e963b/raw/0cd1833b49ff6d351ffa2d5899a89ab7856664ee/fgjfghjfghk

If all these are revoked/ppplied -> wine-staging-4.[6-7]-rc_p1 on demand.

We can also work around rewriting by making .backup files if patch tries to rewrite something.

Sufficient?

plata commented 5 years ago

Most of the diff you posted is binary. Like I said: you cannot patch that.

Kreyren commented 5 years ago

Most of the diff you posted is binary. Like I said: you cannot patch that.

thats why am i suggesting sandboxing affected pathnames..

plata commented 5 years ago

Can anybody else please have a look at this? Obviously, I can't wrap my head around it...

qparis commented 5 years ago

@plata I agree with you, this conversation is going all over the place.

@Kreyren you need to provide a real proof of concept because so far, it is not clear where you want to go.

As far as I understand from the comments

you are mixing two different concepts: sandboxing and prefix versioning. The latter has already been suggested here: #11782
I don't see why you suggest sandboxing to implement your idea ;
I don't see how prefix versioning would help uninstalling a verb (see the comment below) ;
I don't see how verb uninstallation would help to implement your idea ;
I don't see how prefix versioning would help to implement your idea either ;
The proof of concept you've described does help me to see how sandboxing could be implemented ;
The proof of concept you've described does help me to see how prefix versioning could be implemented either.
Same remark for wine version diffing. Apart the fact that it would avoid someone having the upstream version to download a whole package, I don't see how it helps.

So at this point, you need to put all the pieces together because honestly it is not clear so far. Please describe a end-to-end story.

@plata If you see anything else, please add it :-)

Why container versioning does not solve verb uninstallation problems? Even if we were able to versioninze containers, it would not solve at all the problem of verb uninstallation. Let's say you have three files and two verbs

lib1.dll
lib2.dll
lib3.dll
verbA patches lib1.dll and lib2.dll
verbB patches lib1.dll and lib3.dll

If you install verbA and then verbB, you would not be able to uninstall verbA without uninstalling verbB with your method since a file was patched by the two verbs. The only way we could ensure that we can uninstall a verb is that the underlying installer is reversible, which is most of the time not the case.

Kreyren commented 5 years ago

The latter has already been suggested here: #11782

Is not hyperlink, provide valid reference

I don't see why you suggest sandboxing to implement your idea ;

Since that or simmilar concepts are beeing used by 90% of apps that allows modding like vortex mod manager and ModOrganizer so that changes to file can be revoked any time -> Modular moding

Meaning that it's beeing expected for phoenicis to be able to manage mods on go alike:

Mods A B C D E F Where script is using A B E F so that end-user could add A B C D E and remove F on demand. (which is mandatory for mod managers)

I don't see how verb uninstallation would help to implement your idea ;

since modding on linux is not as simple as on windows and so changes in engine will likely be required to make engineapp working.. for example:

Mods A B C D E F While script is using B C D with verb1

if end-user wants to add mod E F he needs to remove verb1 for compatibility (and use verb2 instead) Which is common issue in linux modding based on my experience depending on the engineapp used.

Why container versioning does not solve verb uninstallation problems?... ...

Depending on the verb used we can put lib2.dll and lib3.dll into a .bak files or store them assuming that we have definitions for said verb to know with which files it's going to interact to make sure it woudn't make permanent changes to engine or engineapp -- OR -- We can make logic in which we symlink said engine files into a new directory and set them as read only, IF verb would try to overwrite symlink it would have to ask phoenicis for permission which would make a track of file changes alike: "This verb wants to replace lib2.dll so i removed symlink and defined change by said verb" alike

verb1
- Changed $SOME_DIR/lib2.dll 
- Changed $SOME_DIR/tmp/log
- Changed $SOME_DIR/icon.png
...

If verb1 would be removed we know exactly what to remove to revoke the changes.

-- OR --

We can define engineapp and engine into a separate directory in phoenicis and then copy them into a new container with said changes which is probably easiest to implement, but wastes system resource (storage).

-- OR --

~~We can go the gentoo/exherbo way and handle engine changes using compilation on demand~~

Or let me present the idea and issue:

IDEA

Allow modding of engineapps using phoenicis

ISSUES

Some modifications require changes in engine dependencies
Some modifications require special patches in engine
Some modifications are not compatible with each other
End-user has to have the ability to change modifications on demand like mod organizer:

Notice that modorganizer allows loading of mod based on priority and allows mods to be enabled/disabled on demand which is expected from any mod manager.

Kreyren commented 5 years ago

EDIT: The order of mod loading is also important.

qparis commented 5 years ago

Still not clear, sorry

Kreyren commented 5 years ago

@qparis Will update OP then i try to make it easier to understand with examples.

Kreyren commented 5 years ago

Updated OP with info about issue presented, working on theory for fix still.

plata commented 5 years ago

Closed until you can provide a proof of concept implementation.

PhoenicisOrg / scripts