search

ModOrganizer2 / modorganizer

Mod manager for various PC games. Discord Server: https://discord.gg/ewUVAqyrQX if you would like to be more involved
http://www.nexusmods.com/skyrimspecialedition/mods/6194
GNU General Public License v3.0
2.22k stars 162 forks source link

No longer check mods that are not found on nexus for updates, delay checks for hidden mods. #712

Closed hagar-qim closed 4 years ago

hagar-qim commented 5 years ago

The problem:

when requesting an update of all modules, MO2 reports every single module as either missing, or outdated, until it chokes on the communication with the nexus.

To Reproduce:

select "Check for Updates" for Skyrim

Environment:

MO 2.2.0, right afetr install Windows 10

Details:

after re-starting MO2, every subsequent request is met with the error message "you've exceeded the Nexus API rate limit and requests are now being throttled. Your next batch of requests will be available in approximately xxx minutes" (over 40 minutes in my case)

USVFS:

MO Interface:

https://gist.github.com/hagar-qim/272605377c77b19347b0df1b13c1bcea

Zarggg commented 5 years ago

I did a quick check of a few mods listed in your logfile -- trying to browse them manually by ID on the site -- and all the ones I looked at are indeed hidden or deleted from Nexus.

hagar-qim commented 5 years ago

weird that all these mods collectively decided to hide and/or delete just coinciding with the upgrade. Also, what's up with being shut out from the nexus for almost an hour ? I don't think that's correct behavior

LostDragonist commented 5 years ago

They were hidden/deleted before now and you just weren't notified. The "being shut out part" is due to request limits enforced by the Nexus servers. We don't have any say in that.

Silarn commented 5 years ago

The old API allowed you to retrieve data from deleted and hidden mods, while the new one does not.

I'm addition, it sounds like you have a large (>1000) modlist. I'd recommend picking up the latest alpha build. There was an oversight where large modlists which exceed the API limit will end up requeueing every mod. Alpha 2 remedies this by only queueing mods without recently tracked update data.

Once all mods have been checked by the new API, future update checks should only require a handful of requests, provided you check for updates more often than once a month.

hagar-qim commented 5 years ago

might consider updating to the alpha build when I figure out how to do that. I'm still not convinced there isn't a problem though : MO2, amongst other things, marks USLEP (nexus id 71214) as "deleted", while the page of that mod still exists and the files downloadable. Also, how is it that the API request limit is suddenly a problem, when it wasn't before ? Did the API change ? is that why USLEP is erroneously marked as "deleted" ?

LostDragonist commented 5 years ago

Arthmoor deletes every version of his mods when he uploads a new version. The specific file you downloaded was deleted therefore MO tells you about it. He recently deleted the latest version of USLEEP and uploaded a new file with the same version. MO is telling you about this. You do want to know about the latest version of files, right?

2.2.0 uses a completely new API. The old API has been shut down. A lot of the Nexus connected functionality has changed as a result of that.

https://www.nexusmods.com/news/14028

Silarn commented 5 years ago

Right, well there are two things. If the actual mod was locked or deleted, you will get a message in the console telling you that. If the file was deleted, the update tooltip will tell you that. Usually this also means that the version would be increased, but there are cases when files are deleted and reuploaded with the same version.

There's a significant difference between a file being deleted and a mod being deleted. That being said, after initially automatically clearing the Mod ID when a mod was detected as being no longer available, we reverted that. But in these sorts of cases, leaving all of these mods untouched can just cause more problems as MO2 will continue to query them as if they still exist. With this many removed mods in your list that's probably going to end up causing a lot of extra requests for no reason.

Might be we should automatically clear out the Not Found entries as these have been, I believe, permanently removed, leaving just the Forbidden ones alone since they have a chance to return. Might be we should create an additional flag for this that will delay a recheck a full day, maybe longer.

hagar-qim commented 5 years ago

@Silarn : well, that sounds like a constructive proposal. @LostDragonist: thanks for the info. As a test, I downloaded the USLEP "new" file and installed it. The file is still marked as "deleted". I can give other examples of clearly erroneous feedback, but I suspect the fault is with the new API, and not so much MO2, processing the erroneous feedback.