Use Go for networking code

[thefallentree]

1. Use go as main() , and implement multi-core networking stuff .
2. Gradually moving Driver code into embed-able LPC VM library.

[silenus-dionysus]

Will more of the code in the driver be migrating to go or will it be staying mostly c?

[Shea690901]

I think we should still use C/C++ and use OpenMP (homepage and on wikipedia) and the features provided by the STL (boost?)... My shared storage template is thread aware (when compiled with the correct define set)... And likewise we can secure other variables/classes/functions...

Btw.: If we´re doing this at all (I hope we do), regardless of the choosen way, we need mutexes on mudlib level ;) But we also gain the possibility to add an efun::sleep(int)

[silenus-dionysus]

I am not sure the idea of multi-threading inside the vm is a good idea for lpmuds in general. Perhaps the next best thing which has been implemented in dgd is some form of software transactional memory.

[thefallentree]

The Driver and the VM are 2 vastly different things.

The driver's main job is to handle network and stuff, driving the VM by providing its input and processing its output.

We can code the driver in C++ (most likely using boost) , doing things the hardway, or, simply, we can use go, it's very pleasant to work with and have mature libraies, especially networking and strings . (do you really want to code web-sockets support by hand?) (or write a /status http page in obscure libevent? I know I don't , so I choose go.

I am working towards a first version right now, and this cuts down a lot of complexity from the driver code itself. So, all should be good now.

On Tue, Jan 31, 2017 at 7:16 PM, cartercheng notifications@github.com wrote:

I am not sure the idea of multi-threading inside the vm is a good idea for lpmuds in general. Perhaps the next best thing which has been implemented in dgd is some form of software transactional memory.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fluffos/fluffos/issues/350#issuecomment-276562270, or mute the thread https://github.com/notifications/unsubscribe-auth/ABMsEIn75DqOYaCSLIrj5jH0pdoOdB0gks5rX_kTgaJpZM4Lx8qV .

[Shea690901]

Websockets? Try websocketpp, seems OK to me (at least according it´s readme)...

And why not supporting multiple threads within the mudlib? OK, no currently existing mudlib supports this, but I´m sure the only reason for this is the missing support from the drivers. If a driver supports a multithreaded vm then mudlibs will follow to exploit this feature...

[thefallentree]

Instead of supporting threads, I'm more willing to explore supporting forks. aka "multiprocess" for python, ruby, nodejs etc.

Threading in dynamic lanugage is usually just for I/O stuff,which we don't really need.

On Wed, Feb 1, 2017 at 12:02 PM, Shea690901 notifications@github.com wrote:

Websockets? Try websocketpp https://github.com/zaphoyd/websocketpp, seems OK to me (at least according it´s readme)...

And why not supporting multiple threads within the mudlib? OK, no currently existing mudlib supports this, but I´m sure the only reason for this is the missing support from the drivers. If a driver supports a multithreaded vm then mudlibs will follow to exploit this feature...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fluffos/fluffos/issues/350#issuecomment-276765681, or mute the thread https://github.com/notifications/unsubscribe-auth/ABMsEK1q_MTVM96IwH9ypnpo9g4RiCX4ks5rYOTzgaJpZM4Lx8qV .

[silenus-dionysus]

I think you overestimate the technical ability of the average domain coder. I think general threads and mutex's would be quite tricky for most coders to handle. It has been discussed many times before by various people in the mud community and the general consensus has always been negative.

Software transactional memory or some form of it has also been discussed and there is one implementation though I have never tested it aka Hydra. Hamlet was at one point attempting to implement this for fluffos but never finished the project since given how the data is handled including the number of global variables it is a bit tricky to implement transaction/commits needed.

Maybe threads could be reargued if someone came up with a good design for it within the lpc language.

[Shea690901]

For most parts only some basic functions would need to use mutexes (e.g. heartbeat and fight related code when accessing the health). All other coders just use those secured functions and don´t notice a real difference from a coders point of view...

But the mud would most likely feal more fluid: Current libs contain some servers, suddenly all those servers truly run in parallel... And livings no longer act one after the other but simultaneously (at least as long as we have enough cores ;) )

[silenus-dionysus]

I think the main issue is that there would need to be some sort of escape hatch for the mutex i.e. a timeout so that it doesn't block indefinitely. i.e. the system should error out after a certain number of cycles and still maintain a reasonably consistent state.

[Shea690901]

That shouldn´t be too difficult to implement...

Btw. concerning a multithreaded driver using C++: When using OpenMP you won´t have to do much low level coding for the threads... Just take a look at the wikipedia article.

[silenus-dionysus]

Well if you can come up with a plan for it go ahead.

[silenus-dionysus]

Also isn't openmp mostly for scientific computing?

[thefallentree]

openMP is a compiler tech , you basically add some annotations on your source and compiler will try to parallel the computation for you. which doesn't work in our VM model, because there is only 1 VM right now, no way to do any sort of parallel evaluations (beside pure number crunching, which I don't think anybody is doing with LPC anyway) ,

All in all, first we need to be able to launch multiple LPC vms, then we can talk about how to use those VMs in LPC

[Shea690901]

Shouldn´t be any problem if we manage to wrap the vm into a single class...

[silenus-dionysus]

I am actually not sure how much effort this would be. You could look into it. I guess the key would be to turn all the global variables required by the vm into non static fields and convert the current api into methods. The vm including the compiler though not huge like other projects is not that small. It might be feasible to do.

[Shea690901]

If no other wan´t to do it I will... But first the conversion of C-strings to STL-strings and alongside the new shared strings... Although I fear that it´s another two to three weeks till I can truly start :(

[silenus-dionysus]

You might run into the problem or work I am running into with figuring out how to implement a garbage collector. The problem is types like object,array,mapping* etc are used both inside and outside of the vm making it hard to determine which ones should be garbage collected. Did fallentree add some sort of a memory checker? I might take a look at that to get an idea of where to "mark" from.

[silenus-dionysus]

I still suggest two "interesting" ideas- STM and distributed muds. i.e. to somehow devise the ground work for a system where the computations on a mud can be not only distributed over a single machine with multi-core processors but across multiple machines as well.

[Shea690901]

Urgh... MPI (message passing interface)... First multithreaded, the other might come later...

[thefallentree]

Guys, if you want to do some big project, please create a new focused issue. The Go train however is leaving town, and first version will be ready within this week.

[silenus-dionysus]

Good to hear fallentree. I was playing around with go a little bit myself recently. Seems quite nice for many of the things we are doing in the driver. I wish that current versions supported dynamic loading though- if this were the case we could easily create a new compiler to go and and dynamically load lpc->go converted programs. Unfortunately my understanding is that this feature still is unimplemented.

[thefallentree]

been there, tried that, currently the networking code is simply too tightly coupled with VM code, it can not be done.