ModOrganizer2 / modorganizer

Mod manager for various PC games. Discord Server: https://discord.gg/ewUVAqyrQX if you would like to be more involved
http://www.nexusmods.com/skyrimspecialedition/mods/6194
GNU General Public License v3.0
2.19k stars 163 forks source link

Identifying .esp that conflict with each other #1111

Closed Shine753 closed 4 years ago

Shine753 commented 4 years ago

As it's done for the files, would it be possible to add a marker that shows is .esp files touch the same things, and will conflict with each other if both activated. If this is possible, that would be a great improvement. In that case, being able to tell where / on which parts the confict is would be really great.

Al12rs commented 4 years ago

No, this would be too expensive. See load times of xEdit for an indication. xEdit already exists and offers this functionality so I'm not inclined in adding this to Mo2.

isanae commented 4 years ago

This is way outside the purview of MO, I'm closing this as it's very unlikely we'll ever do it.

Shine753 commented 4 years ago

MO2 is a great piece of software in many aspects, and deserves the best. The load time is not an issue if you ask manually for a report. But being able to see the same kind of information into a single software is a real benefit.

If I can understand that you don't plan to develop something like this, perhaps the developer of xEdit could share a lib with you to achieve this, the same way you call LOOT even if it's also here a different software. This way you will both benefit f th ebest of the two softwares and community.

Al12rs commented 4 years ago

Consider this, this is too heavy to consider running automatically even as an option so you would in any case need to manually request it. Doesn't this start to sound like you having to press the xEdit toolbar shortcut? Then the plugin list by itself isn't enough we would need to have a new view so you can actually see the conflicts and understand if they can be ignored or not, something like the xEdit record view. Then you probably would like to be able to make changes to the files to fix the conflicts or even better create a patch, which is yet another xEdit functionality we would need to reimplement.

Even if all of these were packaged in a library we would still need to create all the views and ui interactions to make it work, which is a lot of work.

So basically use xEdit. If you feel like it isn't as convenient, give feedback to the xEdit Devs so that the experience can be more seamless.

Shine753 commented 4 years ago

Hello,

I don't agree with you. This evolution seems more logical in MO2 for many reasons :

1- MO2 already does that, in a very efficient way, for the mod list. At least, I don't wait more than if I click on one ESP, it says me which other ESP are in conflict with the one selected. This does not seem hard to implement, does not require a lot of views as far as I know, if xEdit allows to give this surface information to MO2 when called

2- calling this is not more intrusive than clicking on the button sorting the ESPs by calling LOOT. And this is VERY convenient to be able not to have to open multiple softwares to achieve this goal.

3- The final purpose of xEdit is to analyse, deeply, and to... edit. I'm just asking at least for the surface analysing part, that could be shared between MO2 and xEdit : which mods conflict with others. If you want to know why, then ok, just open xEdit. The same thing as LOOT : if you want to add a priority : open LOOT. If you just want to sort the mods, just click on the button.

4- Yes, this could be achieved into xEdit. But the way xEdit is made seem to make this a bit more complicated here, not necessary in the actual logic of the application. So I'm asking this surface feature in the software that seems made for that, has a logic for that, and whichalready do that very efficiently.

LostDragonist commented 4 years ago

As far as performance goes, it takes about a minute for me to load 5 plugins in xEdit without a cache and 6 seconds to load the same plugins with a cache. Considering that users can have hundreds of plugins, this is simply too slow to be updated as part of the main GUI.

There is nothing about this operation that is efficient. There is nothing about this operation that is the same as looking at file conflicts.

Al12rs commented 4 years ago

To understand whether two mods are conflicting we just need to ask the filesystem which file names are inside the active mods and check if there are files that have the same name. This is actually pretty expensive already for setups with 100k plus files (which is pretty common). We spent a lot of time trying to optimize this to the max for the next release.

Now for plugins this is different, my knowledge on plugins is actually only superficial, but basically each plugin contains inside "records" that need to be changed in the game. Thing is that each plugin can contain hundreds of thousands of records (like any plugin containing many word edits).

So now what we would need to do is to actually read the contents of each plugin (which will require a parsing library specialized), list all the records touched by each one and then do this for all the plugins you have. After that use the plugin priority to determine which plugins are conflicting with who. Now we need to actually show this on ui, so adding new icons, new events for selection, new colors for highligting etc. Now we need some way to show which records are the one causing the conflict, then which mods provide which records, so basically something like the conflicts tab of modinfodialog but for plugins.

Apart from the fact that actually reading the contents of a file on disk is much more expensive that simply querying the filesystem to know which files are there, you can see how this would be a lot of work. Then we don't even consider stuff like esl, esm, maybe future games with different formats.

It is simply a lot of work for something that ultimately was already done by another program. Use that program. We aren't getting payed to work here so it's not like we can just take on long projects like it's nothing. We would instead prefer to concentrate on things that aren't already offered by other tools.

Zarggg commented 4 years ago

The best method of comparing the contents of plugin files (ESM/ESP/ESL) is xEdit. Period.

In order to effectively do this, you must load the contents of each plugin and then compare the values of all records present in each plugin. To do this iteratively for every single plugin within a given profile would be exponentially more time and computation costly than what the program does now.

I don't think you fully grasp the requirements of what you are asking.

Shine753 commented 4 years ago

Sorry again, but I still don't agree with you. I answer to you both below. I understand you don't want to implement that, but the arguments you advance seem wrong. So I continue to argument, if perhaps we misunderstood each other.

For this feature, you just need to : 1- wait for the user to click on a button. 2- call the xEdit shared library (if they are ok to make one of course) to ask it which mod is in conflict with which one, just making it able to access the mods. 3- Wait for a return, synchronously or asynchronously 4- display the result of the return into the list

This is just a matter of wanting or not to discuss with the xEdit developers, or not.

Arguments like the following ones are visibly out of purpose :

But again I can understand that you don't want to discuss with the xEdit developers. Or you don't want to add new features. In that case, ok, I'm ok with that, this is the end of the story. But this is not what you advance and I feel that you didn't synchronise with what I suggested, or that you analysed it perhaps a bit quickly before rejecting it.

Shine753 commented 4 years ago

The best method of comparing the contents of plugin files (ESM/ESP/ESL) is xEdit. Period.

Well, the best method for sorting plugins is LOOT. So what... ? The button in MO2 is extremely convenient.

The best method for playing games is a joypad. You don't need to think about adding things to this. Or using a mouse and a keyboard. Period. We can play long with this ;)

But, can you please tell me what is the best method just to simply know, for all activated plugins, which one conflict with other ones when it's selected in the list (as MO2 highlights the esp associated with a mod when the mod is selected) ? Perhaps I missed something with xEdit.

In order to effectively do this, you must load the contents of each plugin and then compare the values of all records present in each plugin. To do this iteratively for every single plugin within a given profile

Yes, but this can be delegated. And not developed by the MO2 team. As for LOOT. And you don't need to compare everything inside each plugin, just find at least one conflict for a surface render inside MO2.

would be exponentially more time and computation costly than what the program does now.

I'm sure that the computation time of the actual MO2 is far more than what it was in its very first releases, isn't it ? ;)

I don't think you fully grasp the requirements of what you are asking.

Be sure I do

BinaryMisfit commented 4 years ago

I do think the continued discussion around this is mute. The developers have already declined and closed this request as they have other priorities.

It's their software at the end of the day and they are allowed to do with it what they wish. The xEdit developers has made it clear that a library to call xEdit won't be developed and it makes the whole idea of doing this impossible anyway.

Shine753 commented 4 years ago

I do think the continued discussion around this is mute. The developers have already declined and closed this request as they have other priorities.

It's their software at the end of the day and they are allowed to do with it what they wish.

For sure, it's their softwares, but they are not Youtube robots, and any non closed mind can reopen something that has been closed if discussing about the subject shows in conclusion that it can be a good idea. It's not because one don't see something in the first place that it doesn't exist.

I also understand they have other priorities. I don't expect them an answer in the minute or a development for the next version. As you said, it's their software. They also have have their lives, their constraints, and knowing that, I only report to let them know about something (a bug, a feature request, ...).

The xEdit developers has made it clear that a library to call xEdit won't be developed and it makes the whole idea of doing this impossible anyway.

Ok, but no one has advanced that as an argument in our discussion. If confirmed, this definitely closes this subject.

fireundubh commented 4 years ago

If confirmed, this definitely closes this subject.

Hi, I'm one of the xEdit devs. AL (@Al12rs), one of the MO2 devs, brought this topic to our attention to assess the feasibility of this request.

  1. Conflict filtering, which requires a whole load order analysis, is far more complex than discussed in this thread. It is not easy, simple, or anything short of hard work.
  2. Conflict filtering outcomes for load orders that have not been meticulously crafted with The Method and xEdit mod groups are, in ElminsterAU's words, "completely meaningless." These outcomes (e.g., tens of thousands of conflicts) are not actionable for nonexpert xEdit users who can identify neither which conflicts are unimportant nor what specific conflicts mean for the player experience.
  3. With respect to performance, consider: with an AMD Ryzen 9 3900x processor, xEdit's Very Quick Show Conflicts filter processed the 624,966 records in my load order in 33 seconds. In MO2, every time you change a plugin or shift the order of plugins, this filter would need to run again. That would necessarily mean 33 seconds per run on my machine per plugin action. I won't speculate on how exactly xEdit-powered conflict filtering would work within MO2, but whatever shape that could take would be severely detrimental to MO2.
  4. Speaking for the xEdit team, we will not be creating a library to support the feature you've requested. It is out-of-scope for our project, and ElminsterAU does not wish xEdit to be used as middleware.
  5. If you would like to as-quickly-as-possible see the conflicts (as xEdit defines them) in your load order, you can run xEdit from MO2 with the -vqsc argument.
Shine753 commented 4 years ago

Hello fireundubh,

Thank you for your answer. If ElminsterAU does not wish xEdit to be used as middleware, it's obvious that this feature request won't exist.

Of course, it's obvious that conflict filtering has always had a lot of considerations to play with and is not easy, and xEdit is very impressive for that.

About the performance aspect, I imagine that Al12rs contacted you early in this discussion, what is nice and appreciable, so you probably didn't have all the informations when you thought about this answer. Because it's a non-subject. The conflict research does not need to be done each time an esp is touched but only when a user ask for a search. If the user can wait at least as long as you say to launch the game to see if it's mod configuration is working well or not, or wait this same time to launch xEdit to apply the filter and take hours to check each item individualy, he also can wait this same 30 seconds after clicking a button into MO2 to see where are the conflicting esp and act on its load order.

For the rest, I propose to continue if needed the discussion on the xEdit page as this won't concern MO2 if xEdit don't want to share datas with MO2, wether as a shared library, or as a resume file that MO2 could access.

Al12rs commented 4 years ago

From Elminster:

Conflict reports on setups that's haven't been created using The Method with mod groups to hide false positives are completely meaningless You end up with 10000s of conflicts

Basically the point is that even if we got the list of which plugins conflict with which plugins, it would be completely useless because all of them will have some sort of conflict with almost all of the others. If the entire pluginlist has the same +- conflict icon and if when selecting a plugin the entire pluginlist gets highlighted it becomes useless and that is why there is no point in this.