Pickysaurus commented 2 years ago

This ticket will document the investigation into the issue occasionally reported by Stardew Valley players where the manifest.json file for one or more of their mods installed via Vortex is reported as invalid by SMAPI during startup.

Symptoms

After installing the mod with Vortex and starting Stardew Valley via SMAPI, the SMAPI console window will show some red error messages like this:

22:50:05 | ERROR | SMAPI | These mods could not be added to your game.
-- | -- | -- | --
22:50:05 | ERROR | SMAPI | - [BL] Desert because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - [CP] DaisyNiko's Tilesheets because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - [CP] Date Night 1.3.4 because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - [CP] Elle's Old Coop Animals because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - [CP] If It Fits, I Sits because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - [CP] More Buildings for Cute Valley Pink 1.2-Slime Hutch Interior Extended because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - [CP]8 Colors Shipping Bin because it's an empty Vortex folder (is the mod disabled in Vortex?).
22:50:05 | ERROR | SMAPI | - [PFM] Bonster PPJA intergration because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - Adventurer's Guild Expanded 1.0.15\[CP] Adventurer's Guild Expanded because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - Adventurer's Guild Expanded 1.0.15\[JA] Adventurer's Guild Expanded because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - AlternativeTextures because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - AtraCore because its manifest is invalid.
22:50:05 | ERROR | SMAPI | - Bonster's Fruit & Veggies because its manifest is invalid.

As the log implies, these mods will not be loaded by the game.

Obervations

The following appears to be consistent with this issue, but it is currently unverified.

Users experiencing this issue appear to have the game and mod staging folder on two different drives, meaning they are using Symlink deployment in Vortex.
The file(s) appear to be present in the correct location but are unreadable by SMAPI.
Installing the same file manually appears to resolve the problem.

Theories

Here are the possible causes based on the available information.

1 - Vortex is corrupting the manifest (Unlikely)

Vortex has been blamed for corrupting the manifest during mod install or deployment, however, Vortex only reads and copies the file. It never directly writes to it or saves it.

2 - SMAPI is having issues following Symlinks (Unlikely)

This issue appears to be limited to Symlinked manifests so it may be that SMAPI is encountering some kind of error when attempting to follow the link.

3 - SMAPI is resaving the manifest at some point, corrupting it (Requires investigation)

In a recent example, @pathoschild was able to verify that the symlink did indeed exist and was pointing to the right place, however, the resulting file contained nothing but NULL characters. manifest.zip

The file is not edited at any point by Vortex, so the working theory there is that at some point SMAPI re-saves the manifest file and this is silently causing the file to be overwritten with null characters.

4 - The downloaded copy of the mod is corrupted (Requires investigation)

This would require a user to provide the archive from their Vortex downloads folder, but it's possible the file from the Nexus Mods CDN was corrupted. In this instance, redownloading the file would probably fix the issue.

5 - User error

You can never rule out user error!

Actions

[ ] Add the Vortex version to the vortex.deployment.json file to assist with debugging. @TanninOne
[ ] SMAPI to read the vortex.deployment.json file (if present) and log the version, deployment type and staging folder keys to the SMAPI log @pathoschild
[ ] Users experiencing this issue should be directed to @pathoschild and @Pickysaurus for troubleshooting.

Pickysaurus commented 2 years ago

Example log: SMAPI log (7ddc6c74a14245df85a88c681727ff43).txt

TanninOne commented 2 years ago

@Pathoschild Out of curiosity: Is that file that's entirely NULs the expected size? As in: was the original, valid manifest also 279 bytes?

Pickysaurus commented 2 years ago

The original file is here: https://www.nexusmods.com/stardewvalley/mods/10844?tab=files 279 bytes, confirmed. manifest.zip

Pathoschild commented 2 years ago

The file is not edited at any point by Vortex, so the working theory there is that at some point SMAPI re-saves the manifest file and this is silently causing the file to be overwritten with null characters.

SMAPI only reads the file as-is; it's never moved, edited, or resaved in any way.

Pickysaurus commented 2 years ago

The file is not edited at any point by Vortex, so the working theory there is that at some point SMAPI re-saves the manifest file and this is silently causing the file to be overwritten with null characters.

SMAPI only reads the file as-is; it's never moved, edited, or resaved in any way.

So the next thing to test is if the downloaded archive was corrupted.

TanninOne commented 2 years ago

Also: can we pin down when exactly the file gets corrupted? Is it immediately after the mod got installed, after deployment or after the game was run?

Pickysaurus commented 2 years ago

Currently talking to Discord user mols (she/her)#5748 who reported the issue recently. I have asked them to run a few tests to see if we can determine when the file gets corrupted.

Pickysaurus commented 2 years ago

Currently, reinstalling the mod seems to continue to result in a blank manifest. I've asked for the zip file to check for corruption, but she's not been able to send it to me yet.

Pickysaurus commented 2 years ago

I've managed to do some more troubleshooting with a user called Nebula on Discord.

They had a similar problem when installing this: https://github.com/Annosz/UIInfoSuite2/releases/download/2.2.5/UIInfoSuite2.zip

Before adding it to Vortex they performed the following actions:

So with the first install I opened it with Winrar and when I went to drag the file into vortex at the bottom and it popped up in notifications that the file doesn't exist. I then tried a couple more times to drag it into vortex, and after that, instead of dragging the mod into my mods folder for Stardew, I went into a folder I made for mods that weren't multiplayer compatibale that was simply on my computer and not attached to anything specific, and I dragged the UI file into that file. From the Non multiplayer file I dragged the UI file into vortex where it downloaded seemingly just fine.

The resulting output of that was this file: UIInfoSuite2_2.zip - double zipped because GitHub won't let me upload 7z

When trying to install the file, this is the Vortex log. Failed install log.zip

Fri, 12 Aug 2022 15:23:08 GMT - debug: mod id for newly installed mod archivePath=[[APPDATA PATH - REMOVED FOR PRIVACY]]\Vortex\downloads\stardewvalley\UIInfoSuite2.7z, modId=UIInfoSuite2
Fri, 12 Aug 2022 15:23:08 GMT - debug: installing to modId=UIInfoSuite2, destinationPath=[[APPDATA PATH - REMOVED FOR PRIVACY]]\Vortex\stardewvalley\mods\UIInfoSuite2
Fri, 12 Aug 2022 15:23:08 GMT - info: using install path id=UIInfoSuite2, installPath=[[APPDATA PATH - REMOVED FOR PRIVACY]]\Vortex\stardewvalley\mods\UIInfoSuite2, fileName=UIInfoSuite2
Fri, 12 Aug 2022 15:23:08 GMT - debug: extracting mod archive archivePath=[[APPDATA PATH - REMOVED FOR PRIVACY]]\Vortex\downloads\stardewvalley\UIInfoSuite2.7z, tempPath=[[APPDATA PATH - REMOVED FOR PRIVACY]]\Vortex\stardewvalley\mods\UIInfoSuite2.installing
Fri, 12 Aug 2022 15:23:08 GMT - debug: done removing mods game=stardewvalley, mods=[UIInfoSuite2]
Fri, 12 Aug 2022 15:23:08 GMT - info: [new-file-monitor] file list snapshot saved filePath=[[APPDATA PATH - REMOVED FOR PRIVACY]]\Vortex\stardewvalley\snapshots\snapshot.json, duration=260
Fri, 12 Aug 2022 15:23:08 GMT - debug: extraction completed
Fri, 12 Aug 2022 15:23:08 GMT - debug: [installer] test supported
Fri, 12 Aug 2022 15:23:08 GMT - debug: [installer] test supported result {"supported":false,"requiredFiles":[]}
Fri, 12 Aug 2022 15:23:08 GMT - debug: invoking installer installer=stardew-valley-installer, enforced=false
Fri, 12 Aug 2022 15:23:08 GMT - info: determine mod type gameId=stardewvalley
Fri, 12 Aug 2022 15:23:08 GMT - info: determined mod type id=UIInfoSuite2, modType=
Fri, 12 Aug 2022 15:23:08 GMT - debug: installer instructions [{"type":"copy","source":"assets\\LevelUp license.txt","destination":"UIInfoSuite2\\assets\\LevelUp license.txt"},{"type":"copy","source":"assets\\LevelUp.wav","destination":"UIInfoSuite2\\assets\\LevelUp.wav"},{"type":"copy","source":"i18n\\de.json","destination":"UIInfoSuite2\\i18n\\de.json"},{"type":"copy","source":"i18n\\default.json","destination":"UIInfoSuite2\\i18n\\default.json"},{"type":"copy","source":"i18n\\es.json","destination":"UIInfoSuite2\\i18n\\es.json"},{"type":"copy","source":"i18n\\fr.json","destination":"UIInfoSuite2\\i18n\\fr.json"},{"type":"copy","source":"i18n\\hu.json","destination":"UIInfoSuite2\\i18n\\hu.json"},{"type":"copy","source":"i18n\\ja.json","destination":"UIInfoSuite2\\i18n\\ja.json"},{"type":"copy","source":"i18n\\ko.json","destination":"UIInfoSuite2\\i18n\\ko.json"},{"type":"copy","source":"i18n\\pl.json","destination":"UIInfoSuite2\\i18n\\pl.json"},{"type":"copy","source":"i18n\\pt.json","destination":"UIInfoSuite2\\i18n\\pt.json"},{"type":"copy","source":"i18n\\ru.json","destination":"UIInfoSuite2\\i18n\\ru.json"},{"type":"copy","source":"i18n\\th.json","destination":"UIInfoSuite2\\i18n\\th.json"},{"type":"copy","source":"i18n\\tr.json","destination":"UIInfoSuite2\\i18n\\tr.json"},{"type":"copy","source":"i18n\\uk.json","destination":"UIInfoSuite2\\i18n\\uk.json"},{"type":"copy","source":"i18n\\zh.json","destination":"UIInfoSuite2\\i18n\\zh.json"}]
Fri, 12 Aug 2022 15:23:08 GMT - debug: removing temporary path [[APPDATA PATH - REMOVED FOR PRIVACY]]\Vortex\stardewvalley\mods\UIInfoSuite2.installing

As you can see from the log, Vortex somehow ignores the manfiest.json and DLL files at the top level but installs everything else.

Downloading a fresh copy of the file appeared to resolve the problem. So that would imply something Nebula did prior to adding it to Vortex may have confused things?

My Discord Ref

TanninOne commented 1 year ago

@Pickysaurus any progress on this? Is this still even a problem?

Pickysaurus commented 1 year ago

I'm not sure if it's still a problem. I've not had much feedback either way since. I'm going to assume it does still happen as I gave steps to reproduce it in my last comment.

TanninOne commented 1 year ago

I followed those steps to reproduce and it installs fine so if there are no other leads we have to conclude this is fixed.

Nexus-Mods / Vortex

Stardew Valley: Empty Manifest Investigation #12548

Symptoms

Obervations

Theories

1 - Vortex is corrupting the manifest (Unlikely)

2 - SMAPI is having issues following Symlinks (Unlikely)

3 - SMAPI is resaving the manifest at some point, corrupting it (Requires investigation)

4 - The downloaded copy of the mod is corrupted (Requires investigation)

5 - User error

Actions