Cause of Remote API not working

[underdogest]

The VRage Remote Api uses the HttpListener class from dotnet, which in turn uses http.sys windows kernel driver. The current wine implementation of http.sys is spotty at best, so it wont work as it is. One possibility would be to decompile VRage.Dedicated.dll and recompile it against a HttpListener implementation that uses native tcp sockets.

Edit: After changing the HttpListener prefix from "http://*:" + this.Port.ToString() + "/vrageremote/" to "http://+:" + this.Port.ToString() + "/vrageremote/" and adding a context.Response.ContentLength64 = bytes.Length; at two places before context.Response.OutputStream.Write(bytes, 0, bytes.Length); the remote api actually works under docker

[Devidian]

Ok, good to know - but i don't think its the right solution for a docker image.

[Netbulae]

Have been looking trough the wine commits and there may be some patches in 8.8 that could impact this. I'm not that into wine and dotnet to be able to confirm. Will try to setup a test environment with the latest dev wine to test when I have the time

http.sys: Add support for adding multiple urls to request queues. https://github.com/wine-mirror/wine/commit/0f139a04b110c222dfca05831d8263c912919c1b

http.sys: Add support for relative URLs. https://github.com/wine-mirror/wine/commit/0ba9567a5651418b2f373643e70895fc2de6d9fc

http.sys: Use SO_EXCLUSIVEADDRUSE on listening socket. https://github.com/wine-mirror/wine/commit/0ba9567a5651418b2f373643e70895fc2de6d9fc

[Devidian]

Ok, i am currently using wine 8.8 devidian/spaceengineers:winestaging but did not try the remote api yet

[Devidian]

I've updated staging to wine 8.9 still no response when i try to connect to Remote API

[erladev]

Hi @underdogest! Since your changes are currently the only solution, can you briefly describe how you made them? I'd like to implement them for my docker SE..

• was the change made to code "within" VRage.Dedicated.dll ?
• can you recommend tools for disassembly and reassembly? your help is much appreciated!

[Devidian]

I've tested the latest wine, still not working.

[underdogest]

Hi @underdogest! Since your changes are currently the only solution, can you briefly describe how you made them? I'd like to implement them for my docker SE..

• was the change made to code "within" VRage.Dedicated.dll ?
• can you recommend tools for disassembly and reassembly? your help is much appreciated!

yes in the binary itself using dnsspy. Open the DLL, find the class and click edit class, then do the code changes and click compile. Afterwards "save module". Here you change * to + Here you add "context.Response.ContentLength64 = bytes.Length;"

Use the newest version of dnspy: https://github.com/dnSpyEx/dnSpy SpaceEngineers calls the winapi wrong, but the winapi still works correctly on windows, but not in wine.

Afterwards it should look like this:

Since its decompiling and recompiling the names might be chosen randomly on your dnspy.

[Braste]

Hi @underdogest! Since your changes are currently the only solution, can you briefly describe how you made them? I'd like to implement them for my docker SE..

• was the change made to code "within" VRage.Dedicated.dll ?
• can you recommend tools for disassembly and reassembly? your help is much appreciated!

yes in the binary itself using dnsspy. Open the DLL, find the class and click edit class, then do the code changes and click compile. Afterwards "save module". Here you change * to + Here you add "context.Response.ContentLength64 = bytes.Length;"

Use the newest version of dnspy: https://github.com/dnSpyEx/dnSpy SpaceEngineers calls the winapi wrong, but the winapi still works correctly on windows, but not in wine.

Afterwards it should look like this:

Since its decompiling and recompiling the names might be chosen randomly on your dnspy.

Tried this, did not work.

[underdogest]

Tried this, did not work.

Just tried it again with newest version and it works. Using docker in debian12 in VirtualBox:

make sure the server is actually starting and you can join!. That took the longest for me, if there are spaces in ANY filepath it will crash and restart. Check in /appdata/space-engineers/instances/TestInstance/SpaceEngineers-Dedicated.cfg for <LoadWorld>Z:\appdata\space-engineers\instances\TestInstance\Saves\AlienPlanet</LoadWorld> I removed the spaces in the save name, after that it worked.

Full walkthrough: My docker-compose.yaml:

services:
  se-server:
    image: devidian/spaceengineers:winestaging
    container_name: se-ds-docker
    restart: unless-stopped
    volumes:
      # left side: your docker-host machine
      # right side: the paths in the image (!!do not change!!)
      - /appdata/space-engineers/plugins:/appdata/space-engineers/plugins
      - /appdata/space-engineers/instances:/appdata/space-engineers/instances
      - /appdata/space-engineers/SpaceEngineersDedicated:/appdata/space-engineers/SpaceEngineersDedicated
      - /appdata/space-engineers/steamcmd:/root/.steam
    ports:
      - target: 27016
        published: 27016
        protocol: udp
        mode: host
      # very important for the remote client
      - target: 8080
        published: 8080
        protocol: tcp
        mode: host
    environment:
      - WINEDEBUG=-all
      - INSTANCE_NAME=TestInstance
      - PUBLIC_IP=192.168.99.111
      # public ip required for healthcheck

Started up the server with docker-compose up -d checked the logs with docker logs 7c46ced700ed till I saw it crash-looping. Then stopped the container docker stop 7c46ced700ed

I created a world on Windows with X:\SteamLibrary\steamapps\common\SpaceEngineersDedicatedServer\DedicatedServer64\SpaceEngineersDedicated.exe Called the instance "docker", saved and started it until "Game ready", then stopped the server. Copied everything in C:\ProgramData\SpaceEngineersDedicated\docker\* to vbox:/appdata/space-engineers/instances/TestInstance/ Renamed the folder vbox:/appdata/space-engineers/instances/TestInstance/Saves/Alien Planet lots of numbers to vbox:/appdata/space-engineers/instances/TestInstance/Saves/AlienPlanet

Opened up /appdata/space-engineers/instances/TestInstance/SpaceEngineers-Dedicated.cfg Changed the entry <LoadWorld>C:\ProgramData\SpaceEngineersDedicated\docker\Saves\Alien Planet lots of numbers\Sandbox.sbc</LoadWorld> to <LoadWorld>Z:\appdata\space-engineers\instances\TestInstance\Saves\AlienPlanet</LoadWorld>

Copied my patched VRage.Dedicated.dll to (remember to backup the original to VRage.Dedicated.dll.bak just in case) vbox:/appdata/space-engineers/SpaceEngineersDedicated/DedicatedServer64/

Restarted the docker container docker start 7c46ced700ed

Waited for the game to be ready and just connected with VRageRemoteClient.exe to 192.168.99.111:8080 using the RemoteSecurityKey in the SpaceEngineers-Dedicated.cfg

Common errors:

• You forgot to expose port 8080 in the container
• You have something already listening on port 8080 on the docker host
• You didnt check if the server was actually ready for joining players

You can try my attached patched VRage.Dedicated.dll, but once the dedicated server gets updated you need to redo my steps in dnspy like described above. Steam will probably overwrite the dll. Also it might become incompatible with the other updated dlls. VRage.Dedicated.dll.zip

[underdogest]

In the current wine without my patch I get the following error: wine: Call from 00006FFFFF463E67 to unimplemented function httpapi.dll.**HttpSendResponseEntityBody**, aborting Which happens because HttpSendResponseEntityBody is only a stub in wine: https://source.winehq.org/WineAPI/httpapi.html This method is called when a http server sends the response chunked (contentlength is not known when sending). When the contentlength is known, it sends it via httpapi.dll.HttpSendHttpResponse, which is implemented fine in wine.

So for wine to support the remote client natively someone has to implement httpapi.dll.HttpSendResponseEntityBody in wine here: https://gitlab.winehq.org/wine/wine/-/blob/master/dlls/httpapi/httpapi_main.c (very hard) and add a case for http://* next to http://+ here: https://gitlab.winehq.org/wine/wine/-/blob/master/dlls/http.sys/http.c#L388 (literally just a line)

[Devidian]

So it could be fixed in wine in the future. Good to know.

[underdogest]

Writing a contribution for wine right now

[Braste]

In the current wine without my patch I get the following error: wine: Call from 00006FFFFF463E67 to unimplemented function httpapi.dll.**HttpSendResponseEntityBody**, aborting Which happens because HttpSendResponseEntityBody is only a stub in wine: https://source.winehq.org/WineAPI/httpapi.html This method is called when a http server sends the response chunked (contentlength is not known when sending). When the contentlength is known, it sends it via httpapi.dll.HttpSendHttpResponse, which is implemented fine in wine.

So for wine to support the remote client natively someone has to implement httpapi.dll.HttpSendResponseEntityBody in wine here: https://gitlab.winehq.org/wine/wine/-/blob/master/dlls/httpapi/httpapi_main.c (very hard) and add a case for http://* next to http://+ here: https://gitlab.winehq.org/wine/wine/-/blob/master/dlls/http.sys/http.c#L388 (literally just a line)

I use a native windows httpapi.dll and do not get this error. Instead I don't get ANY message, with or without the patch.

[underdogest]

I use a native windows httpapi.dll and do not get this error. Instead I don't get ANY message, with or without the patch.

you would also need the native http.sys driver, but that one cant be loaded into linux or wine.

[Braste]

I use a native windows httpapi.dll and do not get this error. Instead I don't get ANY message, with or without the patch.

you would also need the native http.sys driver, but that one cant be loaded into linux or wine.

That makes total sense, my bad. I reverted to native. I still can't connect, but at least when I close the remote client I get "02d4:err:http:receive_data Got error 10054; shutting down connection." in my server console.

[underdogest]

Looking good boys, my wine's httpapi.dll and http.sys work with an unmodified spaceengineers dedicated server now!

Now I just need to get my changes into wine main lol

[erladev]

@underdogest Very nice! Could you link the wine change? Apart from being curious how you implemented it, i might compile wine myself until the new wine version is released.

[PhysicsMD]

@underdogest Very nice! Could you link the wine change? Apart from being curious how you implemented it, i might compile wine myself until the new wine version is released.

https://gitlab.winehq.org/wine/wine/-/merge_requests/6216

[underdogest]

@underdogest Very nice! Could you link the wine change? Apart from being curious how you implemented it, i might compile wine myself until the new wine version is released.

https://gitlab.winehq.org/wine/wine/-/merge_requests/6216

Oh dear, they just copy pasted code from my branch and merged it into master. There was still an issue I was too lazy to fix regarding keep-alive http connections not working after the first connection ended.

[Art-Shpiller]

if you have host network mode, then you don't need to specify ports, just network_mode: "host"

[jpcapone]

Can anyone confirm that this is working and I can download, install and run Vrage remotely?

[Devidian]

i've just pushed a new staging branch update to wine staging 9.20 (was still on 9.9). Image should be ready in a hour maybe, dont know how fast dockerhub builder is today. Took 20 Minutes on my pc. Maybe someone can test then and report

[jpcapone]

i've just pushed a new staging branch update to wine staging 9.20 (was still on 9.9). Image should be ready in a hour maybe, dont know how fast dockerhub builder is today. Took 20 Minutes on my pc. Maybe someone can test then and report

Thanks! I'll keep my eye on this thread.