Linux version

[Ape]

Is it possible to port this game to Linux?

[Ape]

The game seems to have several binary dependencies in the 3rd folder. These would need a Linux version, too.

RakNet, SharpDX, and SteamSDK are available for Linux already. I'm not sure about HavokWrapper. Is that required or just optional or is there a way to replace it? I guess VRage.Native is made by the same developers as SpaceEngineers itself. Is the sources for that available?

[Spartan322]

Well first, Havok has (in the past at least) been on Linux (and other Unix-based systems) before though not sure of the most recent versions (btw, it is a somewhat expensive license to get), and you can not replace that Havok Wrapper, it is kind of what makes the physics work.

[crosbymichael]

I think a good first step is to get the dedicated server working on linux. I plan on trying to get this to work over the weekend and if anyone feels like joining in that would be great.

[OndrejPetrzilka]

We can provide VRage.Native code, that's the smallest problem. We cannot provide HavokWrapper code (licensing). Also we cannot provide HavokWrapper.dll/so for linux, because the wrapper is mixed mode assembly (written in C++/CLI) and mono does not support mixed mode assemblies. It seems it would be necessary wrap Havok in different way (I'm not even sure how at the moment), we cannot use SharpDX way of wrapping, because Havok is not COM object (DirectX is).

We're open to suggestions and happy to provide support.

[mccorkle]

@crosbymichael I'm considering working on the same. Direct message me.

[mccorkle]

@OndrejPetrzilka Is HavokWrapper the only thing stopping a native Linux compile? Havok still claims to support Linux in their marketing message on the bottom of http://www.havok.com/physics/.

Would you guys be interested in adding someone at Valve's SteamOS team to this conversation? SteamOS is Linux based (http://store.steampowered.com/steamos), and the success of this work would define whether Space Engineers could be sold on SteamOS based machines.

[Ape]

I wrote above that SharpDX would be available for Linux, but now that I read it I don't know where I got that. DirectX may be really problematic here.

[mccorkle]

It also looks like Wine runs the client at "silver" status, as reported by the community: https://appdb.winehq.org/objectManager.php?sClass=version&iId=30180

I'll be trying that out on my rig first, the trying to wine wrap the game to run on SteamOS. With the source, I should be able to trace back to anything that Wine can't deal with and post that if anyone else is interested.

[Spartan322]

uh DirectX is a Microsoft-only product, it was made to erase the non-MS products from gaming so, the problem is to start using OpenGL (which I feel would be a better option anyway, it appears to work faster then DirectX IIRC), which requires a major rewrite (even though they kind of both support the same stuff, even if though it takes DirectX a lot longer to get advanced features that OGL already got years ago)

[mccorkle]

@Spartan322 Perhaps Valve's DirectX to openGL converter (which they open sourced) could help with that.

https://github.com/ValveSoftware/ToGL

It isn't 100%, and only supports DX9, so there might be some more effort needed over on that project to catch up with DX11. What are your thoughts?

[Ape]

One possibility is to replace SharpDX with MonoGame (and use it also on Windows). Porting any low-level DirectX is going to require a lot of effort.

[OndrejPetrzilka]

Writing OpenGL render should possible, because render is separate project and communicates only by sending messages. (You can even try to switch to DirectX11 render, it's probably one line somewhere in MyProgram)

[Spartan322]

@mccorkle I'd suggest converting it by hand, while that is great and all, that could create a bit of overhead, which we seriously don't need in SE (or VRage, simple it isn't ready for it) plus we need to convert every single DLL (by that I mean we need to hope most DLL owners of the 3rd Parties here are willing to produce .so files) we also have to massively change how the game does 90% of the things it does (most of the game is center on using Windows only libs or parts)

P.S: Wouldn't mind making a repo devoted to reworking this for Unix systems but I would like permission from Keen before doing so (also I'm more of a C++ developer, so I wouldn't be as active on it plus I prefer C and C++, C# is kind of ridiculous to me)

[fabian-marquardt]

I am also very interested in a Linux port, especially for the dedicated server. If someone is working on it and needs help, please message me.

[elmat0]

Likewise. I looked at space engineers many months ago and saw it was windows only and moved on. Unfortunately there isn't much I could contribute to a porting effort, though would certainly appreciate any efforts to this effect.

[sjsoft2015]

will surely show all possible assistance in porting dedicated server

[bfosberry]

Super excited about this!

[admalledd]

Ryan Gordon did a talk a while ago about starting with linux game porting/development. Decent place to try and wrap heads around how to approach the problem.

https://www.youtube.com/watch?v=Sd8ie5R4CVE

My recommendation would be two approaches side-by-side:

1. People with windows machines, focus on moving to cross-platform libraries (OpenGL, re-do parts of the Havok wrapper?). Some one with more knowledge on the current pain points willing to write a bit on what possible replacements/migration paths there are? My understanding is that all of the current pain points besides DirectX has direct linux versions to look at, can we get confirmation on that and what they are? Licensing?
2. People with only linux machines, focus on the mantra "When in doubt stub it out", your/our goal is to get the game building on linux, even if it never can run. This lets us find the problematic areas for the windows devs to look at, as well as the starting point for replacing with linux/platform specific libraries and code paths.

Thoughts?

[Spartan322]

@admalledd The worst problem is that half the used libraries for SE is windows-only (or the Unix/POSIX versions are super outdated and are very bad to use today) also seeing as SE (and VRage) seem to never been built to adapt (I think KSWH need to slow down some development btw, they rush things and some of the code is a little sloppy) itself, which means we need to directly break parts to convert it to Linux. The issue is they built this without abstraction of anything, which would make this job of easy translation to Unix (Mac and Linux + more, this is what this really is) much easier. I would do this if I were a C# developer and had the time but both things are not true (I know a good amount of C# but I barely have the time to waste on this alone and I see C#, in my eyes, as dirty, just me)

[syberphunk]

"Also we cannot provide HavokWrapper.dll/so for linux, because the wrapper is mixed mode assembly (written in C++/CLI) and mono does not support mixed mode assemblies. It seems it would be necessary wrap Havok in different way (I'm not even sure how at the moment), we cannot use SharpDX way of wrapping, because Havok is not COM object (DirectX is)."

Is this required for a linux dedicated server, though?

[Spartan322]

@syberphunk Kind of, it depends on how intimate the server gets with the clients, and lets not forget that the server being cross-platform needs to send and receive information without the OS messing with it (which may be a little more difficult then it should)

[syberphunk]

@Spartan322 Raknet handles network communication cross platform really easily. The OS will not be 'messing with it'. Usually all data is changed into 'network byte order' and then back to be uniform.

The problem I foresee is whether or not the server needs to calculate the physics using Havok and then communicate that to the client, or if it's done client side and the data's just thrown through the server. If it's the latter then that's not really an issue if the server doesn't have to calculate it, it just has to pass on the data.

[Spartan322]

Well, we can't actually talk about Raknet yet though, we can only really deal with the current system for now, when we get Raknet implements 100%, then we can talk about it.

[cryptiklemur]

That seems counter intuitive @Spartan322. We should definitely be talking about the future system. No point writing something that big, that will just need to get rewritten in the next few months.

[Spartan322]

@aequasi The issue is we are not sure how Keen plans to do everything, which means we have to wait for them to implement it, unless given inside info and an interface for it. Otherwise we have no choice but to wait.

[cryptiklemur]

The point of having a discussion here, is to get their input on it too, not just the implementation.

[Spartan322]

I'm just pointing out an issue with discussing it like this, plus this is talking about implementation, this isn't just a discussion, so that's why I am saying this, if anything, we shouldn't really discuss anything other then the implementation here, the other stuff should be on the forums.

[cryptiklemur]

I can't really see why this shouldn't be discussed here. Its a more technical discussion than is normally seen on the forums.

My 2c

[Spartan322]

Github issues are not made for this type of discussion, you should only use github for discussion on code and implementation or problems, the forums should be used otherwise

[cryptiklemur]

While i dont entirely agree, I went ahead and set up a chat room for more real-time discussions.

http://forum.keenswh.com/threads/modding-chat-room.7359969/

[croxis]

In the reddit AMA they said the server will be 100% authoritative when the new netcode rolls out. The server and clients will both be running the physics simulation with the server overriding any differences in the client.

Could always gut havok out and make a branching using a cross platform library like bullet. Don't think Keen would be too pleased or supportive of that ;)

[Spartan322]

@croxis can't do that (unless you'd like rubber-banding and bad performance, Havok and Bullet are not made to synchronous running, cause the client does a lot of physics to make it look pretty (or it should, or you are frozen and not able to move until the server responds))

[croxis]

I thought the client also runs the physics code for client side prediction to prevent that? The client adds lerp (or other graphical only smoothing) in the event of discrepancies with the server? Looks like Gaffer has a new talk on the GDC vault as his original article is a bit dated now (2004). Regardless I'll stop with the off topicness :)

[Spartan322]

@croxis I was talking about prediction (which seems more broken for SE then some other games), if you try to run Bullet on a server and Havok on the client it doesn't go well.

• 1: Havok and Bullet do things hugely different, so you have to redesign the server around that
• 2: The difference systems work at different speeds and do things so different, which means calculations show different results, resulting in rubber-banding and maybe worse things

But now that I explained that, how about we talk about something else on this now.

[croxis]

Sorry you misunderstood my joke. When I mentioned switching to bullet I meant both client and server. I thought that was obvious.

A good step for the dedicated is developing a command line only interface (unless there is one, I haven't found it). This would at least make it more feasible running it through wine without having to fiddle with off screen x server buffers. A further abstraction of making a server api (with the in game api) would open up the possibility of better integrated 3rd party interfaces (web, ncurses, qt, irc bots, whatever suits your fancy). It is a far cry from porting the whole game to linux but it is something that would need to be done for headless machines.

[Spartan322]

Maybe if we wrote an abstraction layer for SE, so we could separate many parts of it, but most of our issues reside with the fact that there isn't really a GUI system (we'd/Keen need to roll our/their own) making cmd more feasible, but then comes the fact that we still are missing the libraries. Sooo, yeah, we could start doing this but we would have to wait to receive the Havok .so file (not to mention the other files that we need for the server {just for those new guys, .so is the linux version of dll in this case})

[mccorkle]

If anyone needs a Linux server to be able to work on this issue, let me know. Amazon has a 1 year long "free tier" and I can help anyone who needs one to start working on this.

As for us building a Linux dedicated server, I think that there are going to be two distinct efforts: 1) using Wine to wrap the current dedicated server, and using our awesome new access to the source to debug issues to make this a stable and supported platform 2) getting the Space Engineer code to run on Mono and then either swap libraries for Linux versions or replace them with Linux equivalents for physics, netcode, UI, sound, and AI.

I'm going to focus my energies on the first one for now, as I expect it to be the shortest path to a working headless dedicated server. I'll be doing that on a Debian Wheezy server so that any distro specific things I learn or build might help to later get the client up and running on a SteamOS box.

I'm joining the Slack space above, so look for me on there.

[Sekuraz]

I would like to say that we should first fix those minor wine issues and then work on a full linux version, what do you think about this?

[TehSomeLuigi]

I think it would be nice if we could get SE running in Wine in 64-bits mode, but a Linux version would be ideal.

[Sekuraz]

There are almost all libraries missing for 64 bit (I am not exactly sure which ones, but I think: Havok, DirectX, etc.)

[SaphireLattice]

@TehSomeLuigi, 64-bit wine is broken, veeeery broken. At least from what i know.

[jlitewski]

The way I see this happening is this:

First, people get together to get the game working decently in Wine (it currently works in wine, but it still needs a lot of help). After that is done, start working on an abstraction layer for the graphics code, and move all the DirectX code (DX9 and DX11) to use that. Then add OpenGL support (OpenGL3.x, OpenGL4.x, Vulcan). After that, OS abstractions (like path names and junk), and then it should only need platform natives (and a lot of testing).

[Spartan322]

@HACKhalo2 .NET handles most of the platform specific stuff, but you need to install a .NET implementation for Unix (which I believe MS did make) or you can try Mono. It be better though if we didn't need to rely on the .NET framework. I'd be all for this game working in C/C++ instead (too) but the developers seem to really love .NET and don't seem to plan to move from it soon (seems they also don't plan to go to OpenGL anytime soon either)

[Sekuraz]

.Net implementation for Unix is mono. And we will NEVER be able to get rid of .Net, simply because ALL code is written in it... When it comes to OpenGL, well, let`s hope, eighter for the developers or for a OpenGL genius, because I am really happy if I can draw a triangle in OpenGL.

Am 27.05.2015 um 22:29 schrieb Geroge:

@HACKhalo2 https://github.com/HACKhalo2 .NET handles most of the platform specific stuff, but you need to install a .NET implementation for Unix (which I believe MS did make) or you can try Mono. It be better though if we didn't need to rely on the .NET framework. I'd be all for this game working in C/C++ instead (too) but the developers seem to really love .NET and don't seem to plan to move from it soon (seems they also don't plan to go to OpenGL anytime soon either)

— Reply to this email directly or view it on GitHub https://github.com/KeenSoftwareHouse/SpaceEngineers/issues/9#issuecomment-106063030.

[jlitewski]

@Spartan322 C/C++ doesn't offer that many benefits compared to .NET/C#, other then being closer to the metal and a finer grain control over how the memory is used. It's not worth the time or the stress for a minimal gain like that.

@Sekuraz OpenGL isn't that awful once you get the hang of it. I'm no god in it, but I can get around. Maybe porting to Mono would be a better option, but there's nothing like bare OpenGL

[Spartan322]

@HACKhalo2 .NET isn't natively cross-platform, is actually pretty slow compared to C (or C++), and while it abstracts a lot it is actually very problematic when you plan to use platform native functions. While C# is a pretty decent language, it wasn't originally built for Unix support. All in all, it seems like C# was a poor choice for a newly built engine, it would have been better to build the engine in C/C++ for speed and control, and use C# like a scripting language instead.

[cryptiklemur]

May be more productive if you guys move to a realtime chat, instead of this...

[Spartan322]

@aequasi How about gitter? It probably be the best for this type of chat.

[cryptiklemur]

That'd work. Theres also the slack chat thats already set up. (which you are already a member of)

https://space-engineers-slack.herokuapp.com/

[croxis]

The other part of this is being able to use other sdks like monodevelop to compile in Linux as well On May 27, 2015 20:29, "Aaron Scherer" notifications@github.com wrote:

That'd work. Theres also the slack chat thats already set up. (which you are already a member of)

https://space-engineers-slack.herokuapp.com/

— Reply to this email directly or view it on GitHub https://github.com/KeenSoftwareHouse/SpaceEngineers/issues/9#issuecomment-106159280 .