[Enhancement] MODS environment variable

[mbround18]

Acceptance Criteria:

• installmod odin command.
• Ability to detect if the mod is a dll or valheim plus.
• staging the mod information.
• installed mods configuration file.

[neoKushan]

Thunderstone has a good documented API we could probably take advantage of to pull mods in https://valheim.thunderstore.io/api/docs/

[mbround18]

Thunderstone has a good documented API we could probably take advantage of to pull mods in https://valheim.thunderstore.io/api/docs/

A good amount of these mods are windows client only tho. Thoughts on how we could get peeps to not install those than get confuzzled?

[mbround18]

Also, I already have the nexus, unfortunately nexus is paywalled. If you want to install any nexus mods you will have to use their premium rolls eyes

[neoKushan]

It's a good question, there doesn't seem to be any kind of tags to indicate server vs client even on nexus mods which is a bit of a bugger as that would be a natural way to filter.

The bigger elephant in the room is that ValheimPlus is one of the few not on Thunderstone (though I believe most of the mods are compatible with it).

I guess it's a case of how helpful we want to be. We could theoretically do all the heavy lifting of installing bepinex/ValheimPlus (Maybe this is a top-level decision as to which "version" of bepinex you want, vanilla or ValheimPlus), then pull, unzip and install any additional mods from some kind of config file to keep it nice and easy. Or we could just install ValheimPlus and let users copy the right DLLs to the right folders.

[mbround18]

a lot of other docker repos do that heavy lifting but honestly they are hard coding that. I am taking the approach of doing an analysis on any zip downloaded and extracting where needed. Similar to how minecraft does it :) that way it can be agnostic of what url you add and all you have to do is keep track of which link is in which order. (mod loader list) Than if you want a new version or an older version just change the link to match.

Thats how https://github.com/itzg/docker-minecraft-server handles it and I must say its a very clean approach

[ouvoun]

The bigger elephant in the room is that ValheimPlus is one of the few not on Thunderstone (though I believe most of the mods are compatible with it).

Requests for them to join Thunderstore seem to be pretty much ignored. Frankly I'm not sure why. Regardless, it's an excellent mod repository with a lovely client to boot (r2modman). I've used it extensively for Risk of Rain and it's been quite nice.

Unfortunately as you've discovered, @mbround18, there isn't metadata describing platform support. In my humble opinion that responsibility should just fall on the users to validate whether their desired mod is supported within this container.

I love the idea of an environmental variable for this, and will watch this eagerly for progress. :)

[mbround18]

I love the idea of an environmental variable for this, and will watch this eagerly for progress. :)

Its getting there <3 the hurtle i am at now is how do you differentiate between a mod that needs to be extracted to the root of the server dir or extracted into the bepinex/plugins directory?

[neoKushan]

Can you go off of the presence of a bepinex/plugin tree? Presumably that folder combination would imply the .zip must be extracted to a particular root folder (which you can align based off of where the bepinex folder is), whereas the absence of one you can probably assume it just goes into bepinex/plugins.

[mbround18]

@neoKushan sadly not all mod developers do that :P some create it in a sub folder of various names and others include various additional files to be extracted in other locations.

It might be, I just need to throw up a guide on whats an acceptable zip file to install.

[ouvoun]

@mbround18 I realize a single ENV variable doesn't give you a lot of room to play with, but I wonder how feasible it'd be to define the structure of each mod?

So for each mod to install, you'd list:

• The zip URL
• The dll location within the zip file (e.g. BepInEx/plugins/ValheimPlus.dll)
• Config locations within the zip file (e.g. BepInEx/config/valheim_plus.cfg)
• Other files follow the same approach (if needed)

This would work for Valheim Plus, for example, as long as BepInEx can be installed automatically if mods are enabled.

With that said, that's a little bit janky. It'd be nicer if you could just specify a Thunderstore package and have it use the latest version instead. I don't know enough about the Thunderstore manifest to know if that's possible though. It doesn't seem to have information defined about the top-level vs nested structure, so maybe you'd be in the same situation you are in now?

[neoKushan]

Thunderstone does have the mod pack details on its github: https://github.com/thunderstore-io/Thunderstore though it doesn't list specifics for valheim. It does list dependencies though, which is helpful.

Looking at a selection of mods, there's no consistency to how they're packaged and nothing in the metadata to help. It seems bepinex will recursively search the plugins folder for additional plugins to load though, so if I had to guess you can just dump everything into the plugins folder and it'll work. Seems messy though.

It'd probably make sense to make that entirely ephemeral , that is to start from a clean plugins folder and dump the mods in when the container starts.

[mbround18]

Currently, i am taking a non atomic approach. If you want to remove mods its up to the user to clean up files remaining on the system. As for mods I am currently testing with the following zip files:

ValheimPlus: https://github.com/valheimPlus/ValheimPlus/releases/download/0.9.4/UnixServer.zip BepInEx from Thunderstone: https://cdn.thunderstore.io/live/repository/packages/denikson-BepInExPack_Valheim-5.4.800.zip

[neoKushan]

I'm sure you've checked this already, but it's worth knowing that the contents of both of those are almost identical, just ValheimPlus has the ValheimPlus.dll and Valheim.DisplayBepInExInfo.dll DLLs in the plugins folder and valheim_plus.cfg in the config folder.

The start_game_bepinex.sh / start_server_bepinex.sh files are different but I don't think there's anything importantly different about those files. All other files are identical (binary/checksum).

Going by the minecraft example earlier that you've been going off of, perhaps this is really just a different "type" of game server?

Nice simple ENV variable to decide which base to start from:

TYPE=Vanilla (default ofc) TYPE=BepInEx TYPE=ValheimPlus

Then all the user needs to do is add their additional mods however they want.

[mbround18]

Thats not a bad idea :)

[ouvoun]

Nice simple ENV variable to decide which base to start from:

TYPE=Vanilla (default ofc) TYPE=BepInEx TYPE=ValheimPlus

Then all the user needs to do is add their additional mods however they want.

I like this idea, but the downside is that it wouldn’t be able to support automatic updates for non-V+ mods.

[mbround18]

The other issue is, there is no reliable way to get valheim plus updates without hitting the github api and that has a pull rate that with the amount of people who use this container would surpass.

[ouvoun]

As long as V+ isn't bundled with the container, each instance should be fine with hitting the releases endpoint without hitting any sort of limit, right? I assume the API limits are per-IP rather than per-repo.

https://api.github.com/repos/valheimPlus/ValheimPlus/releases/latest

[mbround18]

Good idea, i have it pulling the latest zip file with

curl https://api.github.com/repos/valheimPlus/ValheimPlus/releases/latest | jq -r '.assets[] | select(.name=="UnixServer.zip") | .browser_download_url'

[mbround18]

Hey @ouvoun I am going to be tossing up a dev image in a little. Are you interested in giving it a whirl?

[ouvoun]

@mbround18 Sure, happy to give it a spin!

[mbround18]

Alrighty :) Its ready <3

Dev Image: mbround18/valheim:development-mods
New Environment Variables: 
TYPE=Vanilla | ValheimPlus | BepInEx
MODS= List of mods separated by command new line

Example docker-compose:

version: "3"
services:
  valheim:
    image: mbround18/valheim:development-mods
    environment:
      - PORT=2456
      - NAME="Creative Update"
      - PASSWORD="12345"
      - TZ=America/Los_Angeles
      - DEBUG_MODE=1
      - AUTO_UPDATE=1
      - AUTO_UPDATE_SCHEDULE="*/5 * * * *"
      - UPDATE_ON_STARTUP=0
      - TYPE=ValheimPlus
      - "MODS=
          https://cdn.thunderstore.io/live/repository/packages/abearcodes-SimpleRecycling-0.0.10.zip,
          https://cdn.thunderstore.io/live/repository/packages/abearcodes-CraftingWithContainers-1.0.9.zip
        "
    ports:
      - "2456:2456/udp"
      - "2457:2457/udp"
      - "2458:2458/udp"
    volumes:
    - ./tmp/saves:/home/steam/.config/unity3d/IronGate/Valheim
    - ./tmp/server:/home/steam/valheim
    - ./tmp/backups:/home/steam/backups

[ouvoun]

@mbround18 This is awesome to see, love it!

First tried a combination of Vanilla + MODS, which gave me a few installation failures as expected. Might be worth just skipping the attempt at installation if the server type is Vanilla.

Next tried the BepInEx type with a single mod. Worked fine. Added a second mod, also worked. Love it!

Few things that came to mind while running this. It seems like the config directory isn't overwritten when installing new mods (which is great!) but I wonder if it could be symlinked into the saves directory for easier management? I've seen another container do this and it seems pretty convenient. Just a thought.

Secondly, unzipping the mod into the plugins directory ends up leaving it a bit of a mess. I just tested this, and it seems like you can actually nest the unzipped plugin into a folder, and BepInEx is still able to load it successfully. Maybe namespacing the mods at the plugins folder level could make them easier to uninstall?

This is legit though, super happy to use this.

[mbround18]

Hey @ouvoun if you nest the plugin does it load just fine? 0.o

[ouvoun]

@mbround18 Seems to, yeah! Maybe check yourself too just in case. :)

[mbround18]

It looks like it chain loaded the 6 mods I am using for test. Ill toss it up as a rebuild for ya to take a gander at :)

[mbround18]

New update is out where it extracts it to a sub folder

[ouvoun]

@mbround18 Nice, works great. Also tried it with a mod that has a nested "plugins" folder structure, and it seemed to load fine.

Next up is to get Valheim Plus onto Thunderstore, then no ValheimPlus type would be necessary and it could be installed like any other Thunderstore plugin. :)

[mbround18]

Lol true, for now tho it should be all good. I pushed a new update to the image which should fix the cron jobs :P (which i broke in the last push)

[ouvoun]

@mbround18 One quick suggestion for a future improvement. Thunderstore has a nice API to get the latest version for a mod. For example, hitting this API: https://valheim.thunderstore.io/api/experimental/package/abearcodes/SimpleRecycling/

Gives you this response:

{
  "namespace": "abearcodes",
  "name": "SimpleRecycling",
  "full_name": "abearcodes-SimpleRecycling",
  "owner": "abearcodes",
  "package_url": "https://valheim.thunderstore.io/package/abearcodes/SimpleRecycling/",
  "date_created": "2021-03-01T02:16:02.136885Z",
  "date_updated": "2021-03-09T22:52:48.070132Z",
  "rating_score": 170,
  "is_pinned": false,
  "is_deprecated": false,
  "total_downloads": 128588,
  "latest": {
    "namespace": "abearcodes",
    "name": "SimpleRecycling",
    "version_number": "0.0.10",
    "full_name": "abearcodes-SimpleRecycling-0.0.10",
    "description": "Adds a tab to the crafting station that recycles (uncrafts) items from player inventory",
    "icon": "https://cdn.thunderstore.io/live/repository/icons/abearcodes-SimpleRecycling-0.0.10.png",
    "dependencies": [
      "denikson-BepInExPack_Valheim-5.4.600"
    ],
    "download_url": "https://valheim.thunderstore.io/package/download/abearcodes/SimpleRecycling/0.0.10/",
    "downloads": 501,
    "date_created": "2021-03-09T22:52:47.760653Z",
    "website_url": "https://github.com/abearcodes/Valheim",
    "is_active": true
  },
  "community_listings": [
    {
      "has_nsfw_content": false,
      "categories": [],
      "community": "valheim"
    }
  ]
}

It would be neat if mods could be supplied as versioned full names instead of the direct URL, so auto updates could work. For example:

namespace-modname@latest
namespace2-anothermod@0.0.1

[mbround18]

Ill wait until they solidify their api, for now you can also just copy the manual download link :P

I added code last push to follow redirects :P

[ouvoun]

Fair enough! By the way, I see in your PR that the URL for BepInEx is hardcoded which might cause some issues down the road once it updates and breaks dependencies. :) Future problems I suppose lol.

[mbround18]

Thats an easy enough change :P for just one mod its no biggie :p check out my latest push

[ouvoun]

Legend :D

[neoKushan]

You're a machine @mbround18 !

[spannerman79]

This looks very interesting.

One thing that I don't know if its been mentioned yet or not - because of how easy it is to cheat as Valheim is not an authoritative server model the only Anti Cheat I've came across is this one. Down side is that it needs this package of BepInEx Full to be on the server. Doesn't need to be on client side for either.

I did try to see about giving that Full BepInEx package a spin but I need more sleep as I must be missing something simple as Odin doesn't seem to pick it up even after setting the correct locations for the env vars.

[mbround18]

This looks very interesting.

One thing that I don't know if its been mentioned yet or not - because of how easy it is to cheat as Valheim is not an authoritative server model the only Anti Cheat I've came across is this one. Down side is that it needs this package of BepInEx Full to be on the server. Doesn't need to be on client side for either.

I did try to see about giving that Full BepInEx package a spin but I need more sleep as I must be missing something simple as Odin doesn't seem to pick it up even after setting the correct locations for the env vars.

Odin looks for doorstop to be in the root of the server as thats most common but you can override with an env variable. Additionally it wont recognize this full package and you would have to mess with some env vars to get it to extract properly. Good call out tho! whats the tru difference between

This: https://valheim.thunderstore.io/package/1F31A/BepInEx_Valheim_Full/

And this: https://valheim.thunderstore.io/package/denikson/BepInExPack_Valheim/

???

[spannerman79]

Well I managed to come across https://valheim.thunderstore.io/package/1F31A/BepInEx_Valheim_Full_Updater/#reasoning:~:text=Reasoning,GitHub%20if%20you%20encounter%20any%20problems @mbround18

Interesting read as to why. And if its the actual cause then I would tend towards former over the latter.

And as a native linux user I tend to avoid mod managers. Usually on linux (ie Vortex) they are borked or broken and usually need work arounds in order to make them work.

[mbround18]

done, I have a commit which supports this package now. Will toss it up in a bit.

[mbround18]

The change should be live on the mbround18/valheim-odin:development-mods latest push :P

[spannerman79]

I did some testing and here is some feedback for you.

First of all - great work!

Now the nitty gritty

---

My compose is the following;

version: "3"
services:
  valheim:
    image: mbround18/valheim:development-mods
    environment:
      - PORT=2456
      - NAME="Creative Update"
      - PASSWORD="12345"
      - TZ=Australia/Sydney
      - DEBUG_MODE=1
      - AUTO_UPDATE=1
      - AUTO_UPDATE_SCHEDULE="*/5 * * * *"
      - UPDATE_ON_STARTUP=0
      - TYPE=BepInEx
      - "MODS=
          https://cdn.thunderstore.io/live/repository/packages/1F31A-BepInEx_Valheim_Full_Updater-1.0.2.zip
        "
    ports:
      - "2456:2456/udp"
      - "2457:2457/udp"
      - "2458:2458/udp"
    volumes:
      - /share/Valheim_Docker/testing/saves:/home/steam/.config/unity3d/IronGate/Valheim
      - /share/Valheim_Docker/testing/server:/home/steam/valheim
      - /share/Valheim_Docker/testing/backups:/home/steam/backups

The TYPE env var will fail when using ValheimPlus and BepInEx_Valheim_Full_Updater - could be their side too not picking up on the pack difference between ValheimPlus and the standard BepInEx package.

After the first initial setup/install of the container with that compose, starting up the server again and checking logs I see;

Info   :BepInEx Valheim Full Updater] Comparing DLL checksums...
[Info   :BepInEx Valheim Full Updater] BepInEx Valheim Full is installed and up-to-date.

So BepInEx Valheim Full is in there now :smile:

Also while Odin will state that BepInEx doesn't listen to SIGINT if you run the container with Console Interactive & TTY (-i -t) (using portainer - i know lazy but meh) attaching and doing a Ctrl+c will send SIGINT and the server will terminate correctly without needing to join the server (as admin) and in console doing a save.

To automate adding V+ to a setup like this(old fashoned/non automated way);

unzip -j "/path/to/UnixServer.zip" "/BepInEx/plugins/ValheimPlus.dll" -d "/path/to/server/BepInEx/plugins/ValheimPlus.dll"
unzip -j "/path/to/UnixServer.zip" "/BepInEx/config/valheim_plus.cfg" -d "/path/to/server/BepInEx/config/valheim_plus.cfg"

That would only be good on a fresh, new creation as valheim_plus.cfg contents/sections/options will change over time and then its up to the one running the container to update V+ maunually. Which isn't hard, just replace ValheimPlus.dll and do a diff on valheim_plus.cfg to enable/change whats been added/removed.

[mbround18]

You can just use type set to BepInExFull 0.o

Than add valheim plus to your mods variable, errm that wont work doe. I would say ValheimPlus is incompatible with running BepInEx Full unless you want to manually copy the files. This gets kind of into the trenches with mods how there is no standard way of doing things.

Also thats an old message I originally included :P BepInEx originally did not listen to a SigINT but now it does 🤷

[spannerman79]

You can just use type set to BepInExFull

Oh well, I did the long way around of debugging/testing :stuck_out_tongue:

Than add valheim plus to your mods variable, errm that wont work doe. I would say ValheimPlus is incompatible with running BepInEx Full unless you want to manually copy the files. This gets kind of into the trenches with mods how there is no standard way of doing things.

Your telling me.

They use to have just the plugin only over on the nexus. Looks like its been hidden. Maybe too many people had NFI what they were doing :shrug:

Edit: Well I made a request for just the plugin & config alone to be added into their Assets.

[spannerman79]

From 0.9.5 onwards V+ will have a compatible package @mbround18

[mbround18]

From 0.9.5 onwards V+ will have a compatable package @mbround18

Dannng!! Nice!!! I saw your issue you posted, thank you for being such a go getter for this ❤️

[ouvoun]

Looks like 0.9.5 was released. valheim_plus.cfg and ValheimPlus.dll are both listed as separate files that'd need to be installed. I guess they'd need to be copied to the config and plugins folders, respectively.

Maybe if we remind them gently enough times that Thunderstore would be a huge boon for everyone they'll do it and make it easier for everyone. At this point I can't help but think their lack of doing so is related to monetization.

[mbround18]

Looks like 0.9.5 was released. valheim_plus.cfg and ValheimPlus.dll are both listed as separate files that'd need to be installed. I guess they'd need to be copied to the config and plugins folders, respectively.

Maybe if we remind them gently enough times that Thunderstore would be a huge boon for everyone they'll do it and make it easier for everyone. At this point I can't help but think their lack of doing so is related to monetization.

Nahhh it should generate the config on launch :P so extracting the config to the config folder is extra work ;)

[ouvoun]

Nahhh it should generate the config on launch :P so extracting the config to the config folder is extra work ;)

FWIW I tried that in one of the past releases and it failed to launch due to a missing config, so if they've fixed it then that would be great. :)

[spannerman79]

it should generate the config on launch

It does, but it would be "easier" for first time setups. After that yes, manual updating V+ will be needed as the config would need to be modified.

if only V+ had a smart config uptate/checker built in. I do miss how when botania (minecraft mod) updated it only removed config coptions when they didn't apply and it appended the config file with any new config options

Edit: seems like someone already suggested and they are working on it ( source https://github.com/valheimPlus/ValheimPlus/issues/239#issuecomment-791888908 )

Once the above issues mentioned has been done next thing to look at is a way to auto update ValheimPlus.dll.

[mbround18]

@ouvoun and @spannerman79 I just pushed a fix for this :)

- TYPE=BepInExFull

- "MODS=
       https://github.com/valheimPlus/ValheimPlus/releases/download/0.9.5.5/ValheimPlus.dll,
       https://github.com/valheimPlus/ValheimPlus/releases/download/0.9.5.5/valheim_plus.cfg
   "

[spannerman79]

Edited: Please ignore, looks like DockerHub was being dumb and whatever CDN they use didn't update for my country until literal 10 mins after I posted this.

I did check dockerhub to see when the image was there but yeah.. their CDN....