Native MULTITHREADING support

[W1llth0t]

Is your feature request related to a problem?

Your program needs multithreading (multicore) support. I'm not gonna go over the reason why multithreading is needed because I'm sure the person reading this is well aware of what it is (probably better than me). I have a big game that creates a randomally procedural generated world kind of like terraria, but due to everything being crammed onto the one core world creation and load speeds are extremely slow as I need to deactivate everything in the first frame that isn't in the the screen view. deactivating thousands of objects is very intensive and would be much faster if I could just have i run on different cores for each chunk.

Describe the solution you'd like

There is many ways you could do this but maybe you could just do it with script_execute() function and just have an optional parameter to have it run on a specific thread. For example: script_execute(worldGen,1). This example has script_execute() call the script worldGen on the first thread.

Describe alternatives you've considered

You could also have a seperate event like the async events but they just run after every step event for that object or possible just when something is called in the step event it will execute that multithread event such as alarms work.

Additional context

I really want this desperately. like come on pleaseeeeeeee. It will definitely take game maker to the next step and will be much more respected than it already is and will make it much harder for the haters (or whatever you call them) to actually have something against it as a viable commercial game making software.

[tabularelf]

We are already getting this in new runtime… Current runtime cannot support anything close to multithreading for it, as it was not designed for it.

If you need something like that, I’d suggest at the very least using a green thread or promise-like system. I can suggest you these libraries instead:

https://github.com/tabularelf/SimThreads/

https://github.com/JujuAdams/Coroutines

https://github.com/katsaii/future

[tabularelf]

Also on that note, if you need to deactivate thousands of instances without actually destroying and saving their data for instances that are too far away, for a procedural generation game, then you are taking the wrong approach and entirely doing it wrong. Multithreading won't save you there!

[W1llth0t]

So is destroying and recreating objects more performance efficient than deactivating and reactivating? If that's so I already have the framework in place for that but just did the deactivating and reactivating because I assumed it was quicker and thought it wasn't worth testing.

[W1llth0t]

We are already getting this in new runtime… Current runtime cannot support anything close to multithreading for it, as it was not designed for it.

If you need something like that, I’d suggest at the very least using a green thread or promise-like system. I can suggest you these libraries instead:

https://github.com/tabularelf/SimThreads/

https://github.com/JujuAdams/Coroutines

https://github.com/katsaii/future

oh wow these are super helpful. Thanks so much for your help

[W1llth0t]

So is destroying and recreating objects more performance efficient than deactivating and reactivating? If that's so I already have the framework in place for that but just did the deactivating and reactivating because I assumed it was quicker and thought it wasn't worth testing.

wait scratch this . I think I know what you mean now

[tabularelf]

So is destroying and recreating objects more performance efficient than deactivating and reactivating? If that's so I already have the framework in place for that but just did the deactivating and reactivating because I assumed it was quicker and thought it wasn't worth testing.

Just in the basic sense, when you activate/deactivate instances, GameMaker has to crawl through a list of active/inactive instances and handle them. Multithreading wouldn't necessarily resolve the problem, especially if you're calling it every frame. If there's instances that you do no longer need due to being too far away, it is a lot cheaper to remove them. Otherwise you end up having to crawl that list regardless. It is o(1), so the bigger the list, the more time needed to process. Even with multithreading, you'd still eventually hit that same problem. Only now that you've pinned one of the users CPU cores at 100% utilization. Which isn't actually a good thing 😅

The other aspect too is that, inactive instances still take up memory. So if you have over 100,000 of them deactivated, then you'll have 100,000 worth of instances stuck in memory. It's one of those things that works fine for quite a fair amount of games, but not so much for something like procedural generation games.

[tabularelf]

In short: By destroying instances that you no longer need, not only do you avoid the issue with having a huge list of inactive instances... But you also save on memory too!

[jeromepoizat]

Hello, I want to add more support for the Multithread request, as it may seem that many devs requesting it don't always need it an see multithreading only as a cheap strategy of making inefficient code run faster. But I believe that for the more advanced devs and games it can be very usefull. When you are working on a game with a lot of computation it is frustrating to not be able to use the power of all the cores a computer has, and computers all have many cores nowadays. Also just for Game Maker reputation as a game engine I think it's a feature that can give good PR.

In my specific case I am working on a game server for my MMO, and it has happened that I would see my game server capacity maxed out while my total CPU usage would be at less than 10%. So far to enjoy the power of all my cores I am using the method of running multiple server builds and having them communicate between each other to spread the workload between cores. But this method has its limits. It would have made my life easier to be able to use multithreading.

[AtomicUs5000]

I would love this. I assume Gamemaker would have to do something like how JavaFX does things to try to eliminate the countless number of problems that can happen. Basically threads cannot make direct changes to the display... or in this case, the application surface. Instead, if you need display changes, you can either wait for the thread to end and apply results, or within the thread they have a runLater() wrapper which is basically a queue of display updates. The updates are dequeued the next time the main thread updates the display, and it dequeues as many as it can within a frame.

[your-mom-lol]

multithread is pog

[mistletoe]

In short: By destroying instances that you no longer need, not only do you avoid the issue with having a huge list of inactive instances... But you also save on memory too!

Whoa... uh, no.

1. GMS already has a bin-lattice system for activating / deactivating Instances in rectangular areas. The only o(n) cost is what sectors the deactivated Instance is in, vs. the sectors to check against, and that's implemented in C++, so it's not going to get cheaper if managed via GML. I wish we had direct control over the square sizes of the bins (for very large procedurally-generated worlds, bigger bins would be helpful, because we could activate large areas around the player infrequently and cheaply, trading off number-of-instances-alive vs. having to update the world state and activate / deactivate Instances as frequently to keep up with player movement), but that's my only real complaint; speed is fine.
2. If you don't worry about that but presume it'll all be fine because you're using a procedure to generate whatever area you need right now, you cannot really guarantee any persistence without workarounds that are tantamount to saving the entire state. For example, you've entered a procedural area and killed a Boss Monster. Now you've left the area, everything despawns. When you get close enough to the area for the procedural generator to build this area again, the Boss Monster will be right back, along with everything else. Obviously, for a lot of game designs, this is unworkable. So you end up having to save the state anyhow.
3. Anything else about the Instance you've destroyed is lost, unless you write very careful code to save whatever part of the state you want restored later. This can create all sorts of edge cases...

All this... to save a little memory? Oh, wait, if you have to store state information in your GML system for reference later, it's using far more memory than GMS's native system for deactivated Instances, where the Instance's state is already being stored, complete, and really doesn't use much memory (I mean, we're talking about just a handful of Instance variables, in most cases- it's not a big deal).

Basically, what you're saying only makes sense for a procedurally-generated world where you don't care about saving state at all. Then it's fine, but it's a one-trick pony. In all other situations, instance_deactivate is perfectly reasonable and in my experience, works quite well enough for very large procedural worlds.

[tabularelf]

In short: By destroying instances that you no longer need, not only do you avoid the issue with having a huge list of inactive instances... But you also save on memory too!

Whoa... uh, no.

1. GMS already has a bin-lattice system for activating / deactivating Instances in rectangular areas. The only o(n) cost is what sectors the deactivated Instance is in, vs. the sectors to check against, and that's implemented in C++, so it's not going to get cheaper if managed via GML. I wish we had direct control over the square sizes of the bins (for very large procedurally-generated worlds, bigger bins would be helpful, because we could activate large areas around the player infrequently and cheaply, trading off number-of-instances-alive vs. having to update the world state and activate / deactivate Instances as frequently to keep up with player movement), but that's my only real complaint; speed is fine.
2. If you don't worry about that but presume it'll all be fine because you're using a procedure to generate whatever area you need right now, you cannot really guarantee any persistence without workarounds that are tantamount to saving the entire state. For example, you've entered a procedural area and killed a Boss Monster. Now you've left the area, everything despawns. When you get close enough to the area for the procedural generator to build this area again, the Boss Monster will be right back, along with everything else. Obviously, for a lot of game designs, this is unworkable. So you end up having to save the state anyhow.
3. Anything else about the Instance you've destroyed is lost, unless you write very careful code to save whatever part of the state you want restored later. This can create all sorts of edge cases...

All this... to save a little memory? Oh, wait, if you have to store state information in your GML system for reference later, it's using far more memory than GMS's native system for deactivated Instances, where the Instance's state is already being stored, complete, and really doesn't use much memory (I mean, we're talking about just a handful of Instance variables, in most cases- it's not a big deal).

Basically, what you're saying only makes sense for a procedurally-generated world where you don't care about saving state at all. Then it's fine, but it's a one-trick pony. In all other situations, instance_deactivate is perfectly reasonable and in my experience, works quite well enough for very large procedural worlds.

1. In fairness, this has had some issues in the past where it did chug GameMaker a fair bit. The last time I've looked at it, it hasn't really been an issue. However, I do still stand by the fact that having instances with a mix of built in variables and custom variables still can take up significally more than say, a struct that contains the very bare basic information that you need. Which if you were destroying an instance, you could offload that to a struct easily.
2. You do realise with procedural generation, you can save what's been procedurally generated, and load that right back in, all the while excluding the boss. Because the boss is dead. You can make checks for that.
3. Well I mean, that's the point. When you remove an instance that you still need some of it's data, you save that data. And writing a system for that isn't rocket science.

I'm not speaking about this just for "procedural generation". No. This is something that I've seen abused in a lot more ways than one, and I've made several iterations of procedural generation. Yes, I've done the whole activate/deactivate rodeo myself in the past, but I don't do that anymore for procedural generation games. There's some good benefits for activating/deactivating instances, especially outside of infinite procedural generated worlds. But you don't need to do that for everything. Not everything needs to be handled via activate/deactivate. There's more than one way to manage data in GameMaker. As well as more than one game out there that may or may not benefit from activate/deactivate.

There's no "one size fits all solution" here. It can be either super useful or a hinderance. And that's what I was pointing it out to OP for here. There was data that they didn't really need as much, so they shouldn't even have it as an instance until they need it.

[thennothinghappened]

We are already getting this in new runtime…

I'd really like to know more about this and what form it's in