Bring back multi-core rendering

[ghost]

A feature which gmod has been missing/explicitly disabled over a year. It would be great to play gmod with high fps with high player counts >32.

[handsomematt]

It breaks because of Lua multi threading. Iirc Kilburn is going to look into this at some point. On 27 Feb 2014 12:58, "BoXiousNet" notifications@github.com wrote:

A feature which gmod has been missing/explicitly disabled over a year. It would be great to play gmod with high fps with high player counts >32.

Reply to this email directly or view it on GitHubhttps://github.com/Facepunch/garrysmod-requests/issues/152 .

[UltimateByte]

As i was going to say here : https://github.com/Facepunch/garrysmod-issues/issues/1388#issuecomment-106925064

Nope i'm not on linux, I'm running windows 7 x64. The issue is well known and of course, framerate and stuttering (i mean micro lags when doing some actions) could be wayyyy better with multithreading... Maybe i should mention that i'm using 3 screens, run game in windowed noborder and with aero disabled (so i can do my admin life) and that other source games runs perfectly smooth that way. I will maybe check CPU/GPU usage while running HL2 ep 2 for example, as i didn't yet because it was smooth so i didn't worry about that yet.

And by the way, the micro lag on spawning new objects, is for spawning an object the first time in the game, if you spawn several time that object you won't get any lag next times. But maybe it doesn't depend on multithreading and engine just freezes until it's loaded, which is not really good neither btw.

[mcd1992]

That's disk io latency, precache your models/sounds/materials to avoid that (that should be the addon developers job). Multicore Rendering is not the be all end all solution you think it is. You're currently getting ~180 FPS then gmod is running fine. I'm willing to bet the FPS drop is due to poorly written addons.

And even if it was implemented; the ~60 FPS increase it would give is not worth the months of work it would take to implement. Not to mention all the issues it would cause in the future.

[UltimateByte]

I put Gmod onto an SSD (Samsung 840 Pro 256GB), so it couldn't be better conditions. (Cause i play gmod all the time and got 8 servers to administrate...)... It's not addon thing, it happens when you spawn any HL2 item, the game freezes a few milliseconds until it's loaded (of course more if you have a shitty PC and of course it doesn't freeze again if you spawn the same item again).

I'm currently getting +180fps when i do nothing in gmod. You see the difference ? Gmod isn't meant to stay on an empty map. We don't need to go from 180 to 350 fps, we need to go from 20 to 60 in enthusiasts players cases... That'd be great already. And that includes even the most simple build server with only parenting and precision tools... When a guy makes a beautiful boat or train he built with parenting, with 600+ props and you run at 30fps when looking at it, it's quite frustrating. Of course, that includes even more framerate in rp_rockford_v1b that is definitely one of the most used maps currently (see the DarkRP servers...), when you start spawning things in there, framerate just sucks.

If there is no point to multicore, why valve did support it ? Why every single game that is released does support it ? Because it's needed to achieve a smooth game and a less or non CPU limited game... CPUs are not powerful in a single core, we're not at the Pentium 4's century anymore, they are when you use several or all the cores. BTW, when you're online, the server renders physics for you, so you should expect FPS improvements. Well you don't have any improvement, because the simple fact of displaying entities uses a lot of CPU as CPU seems to calculate polygons.

Source engine is CPU intensive, so mutlithread is just essential. As i said, i use 30% of my GPU at peak, and there's no affordable slightly better CPU than mine at stock frequency in single thread (2600K @ 4,6Ghz).

The point is... Usages of gmod evolved a lot, people are asking for more, and will do more anyway, while gmod went back in terms of what users could do when multithread was disabled because of lua instability.

Please... BRING BACK OUR MULTITHREAD

[robotboy655]

The freeze when spawning items is nothing to do with multicore rendering.

As already said, it precaches the model once. Normally, in games such as Half-Life 2 and other games, all models that could possibly appear on screen are precached during loading screens ( That's why they exist ), but this is not an option for GMod.

[robotboy655]

This is obviously not restricted to models.

[alexrsagen]

How is it not an option? Let me suggest one thing... Precache all local models on game start (facepunch loading screen), then when you join a server, have the server send the client a list of server models to precache (assuming they got downloaded beforehand).

[robotboy655]

Because there are way too many models to precahce, it will eat all your RAM and crash.

[UltimateByte]

Yeah, for obvious reason (it's 32 bits) the game cannot take more than a certain amount of ram, and precaching everything would make the game crash as memory limit would be exceed. BUT a modification to the engine to make the spawn of a new object on one thread and not pausing the rest of the engine could make it smooth.

[willox]

Code that spawns entities - especially Lua - expects the model to have been loaded synchronously. What you're suggesting could cause addons to break if they do anything on the same frame as the entity being spawned that'd expect the model to already be loaded (physics, getting bone positions, rendering to RTs for the spawn-menu, god knows what else).

[neico]

That's what a mutex is about, there's no way around those when you enter the multithreading world, because there simply are things that don't like things to be async and expect something to happen before they can do their thing.

[willox]

So the solution is to wait on a mutex every time a model is precached? What's the point in even threading it?

[ar1a]

im just here to say no. no multi-crap. ty

[meepen]

pls make mutexes for every cluainterface function it will fix multicore rendering and speed up the game i swear

[notcake]

we should remove lua and replace it with javascript because garry says it's better. http://garry.tv/2014/08/16/i-fell-out-of-love-with-lua/

[mcd1992]

When a guy makes a beautiful boat or train he built with parenting, with 600+ props and you run at 30fps when looking at it, it's quite frustrating.

Even with multicore rendering, the source engine isn't designed to render that many prop_dynamics at once.

If there is no point to multicore, why valve did support it ? Why every single game that is released does support it ?

Because they can easily, no vanilla source game (except DOTA2?) has Lua bindings to engine functions so they don't have to worry about addons using direct render calls or keeping track of what render hooks get called to make sure they stay in order. Garrysmod is just not like the other source games...

The point is... Usages of gmod evolved a lot, people are asking for more, and will do more anyway, while gmod went back in terms of what users could do when multithread was disabled because of lua instability.

Lua isn't unstable. Quit focusing on Lua being the 'bad guy' here.... You do realize garrysmod wouldn't exist if it didn't use Lua right? More than half of the game is written in Lua.

Multicore rendering would increase FPS slightly, yes. But it would cause headaches for the developers, the increase in FPS is not even close to being worth the trouble. The 400+ prop_dynamics lag is not fixable even with multicore rendering. And your idea that multicore rendering will fix all issues is completely wrong. I'm willing to bet that most of the lag you're experiencing is from addons making too many direct render calls or just not doing things in an efficient manner. I'm having 0 issues with my servers on my shitty Linux laptop using a embedded Intel GPU and my out of date gaming desktop only has issues with shitty servers that have way too many addons. Any server worth it's weight I have no issues on either device. Invest your time in optimizing your addons instead of blaming it on multcore rendering.

[UltimateByte]

"Use Java instead of lua" => If there is a correct interpreter of luaj (if i understand well it would be lua interpreted by java), and it can be multithreaded, and the transition is easy to make, and finally that it doesn't f*ckup every single lua addon, why not ?

"tgp1994 August 16, 2014 at 11:46 am Does this mean you might consider adding JavaScript as a second scripting language to Garry’s Mod?

Reply Garry Newman August 16, 2014 at 1:21 pm No"

Well... Too bad if it's a definitive answer.

"Quit focusing on Lua being the 'bad guy' here.... You do realize garrysmod wouldn't exist if it didn't use Lua right? More than half of the game is written in Lua." => Lua is the reason why multicore isn't supported and gmod exits. We all know that, right. But gmod is unstable while multithreaded because of lua...

"0 issues" with your shitty laptop => You mean 30 fps peak isn't an issue for ya right ? I know openGL can give better performance but anyway, i suppose it doesn't go to 60fps even when looking at the sky. I personally cry when my framerate goes under 100 fps, so imagining thousands of players running at 15 fps in a game that i love makes me sad.

"The 400+ prop_dynamics lag is not fixable even with multicore rendering." => If rendering of those were bound to one thread (if source engine allows it of course), it would be already better, or by the way, it could also make it easier to render by making the "main" thread litgher... Speaking about rendering : I get better framerate without anti aliasing even if my GPU usage doesn't go over 30% WITH AA enabled. I guess it's because AA is consuming a bit of CPU and is still into the same thread... Also,lights (used for.. well, lightning of cars in vcmod or even simple lights) are just using a bit of GPU but a lot of CPU, so whenever you put 3 lights and get your framerate divided by about 2... This is an engine limitation but of course multithreading would help a lot here again. This is quite frustrating, and before talking about lua addon's optimization, you should even consider engine optimization.

Of course there are many addons that f*ckup client's framerate, or server CPU usage, and i already spend so much time to test their performance, or to find and remove them when i notice an anomaly after a while... For example, when optimizing our last server (my first DarkRP), I was testing every single addon, and noticed that the TCB Premium HUD consumed up to 25% of framerate (still don't know why, transparency + size maybe, or a constant check of health or something like that) in my case, so i removed it instantly. Then I noticed that every single HUD was using CPU, and it seems to depend on the size of it. Once again, if a thread was assigned to HUD and/or transparency stuff, it'd help...

I just can't count how many things could be smoother with multithread... And i'm convinced it worth it to find a good and smart solution.

[thegrb93]

Just throwing out an idea here. Keep all the lua state single threaded, but in the lua rendering functions, store rendering calls done via lua. Then dispatch all the call data to their respective threads when it's time to render.

That way lua can stay synchronous and stuff gets rendered when it is time.

[thegrb93]

Another idea, what if you set the lua environment of rendering hooks to only include functions for rendering and other certain safe functions, and pass only a table which contains data defined by the user from the main thread used to share data to the rendering thread.

Would break every lua script, but probably easier and safer than the previous idea.

[notcake]

@thegrb93 I have a strong suspicion that the Lua garbage collection system isn't designed to be multithreaded. String operations, table creation and table insertion would all interact with garbage collection.

A failsafe way to multithread would be to spawn completely seperate Lua environments (no shared functions or data) for each thread, but then you would need to arrange for some kind of message passing system between environments. 90% of addons would need to be rearchitected to support this ("why is muh darkrp hud borken i hate u garry u break eveyrhting")

EDIT: disregard if this is what you just said.

EDIT: also message passing is pretty useless if you're trying to offload CPU-heavy stuff like parsing XML documents or decoding model files or computing Fourier transforms since you'll waste a lot of CPU time encoding and decoding the messages anyway since tables can't be shared between Lua instances.

[thegrb93]

lua_newthread can be used to safely share lua data between threads. http://www.lua.org/manual/5.2/manual.html#lua_newthread

[notcake]

@thegrb93 "threads" in the Lua source refer to coroutines (concurrency not supported) which aren't real threads. lua_newthread creates a new coroutine.

[thegrb93]

Well that's one hell of a misleading function name.

[notcake]

@UltimateByte

1. ""Use Java instead of lua"" whoosh.
2. "tgp1994 August 16, 2014 at 11:46 am [...]" whoosh
3. "Lua is the reason why multicore isn't supported" OK.
4. Many console games are locked to 30 fps. 30 fps is playable. 30 fps is not unattainable in Garry's Mod: I joined a random TTT server: Looking at sky: 70 fps Spectating whole map: 30 fps Average fps hovered around 40 to 50. Before you tell me that my fps is "low", I am playing on a laptop. Convince us that your argument isn't purely "I want more frames per second for the sake of it"
5. "If rendering of those [prop_dynamics] were bound to one thread (if source engine allows it of course), it would be already better" Lua has the capacity to override the rendering of every single entity and we've already told you why it's extremely difficult to multithread Lua.
6. "I get better framerate without anti aliasing even if my GPU usage doesn't go over 30% WITH AA enabled. I guess it's because AA is consuming a bit of CPU and is still into the same thread..." Do you observe higher GPU usage with AA enabled? GPU usage not hitting 100% does not mean that AA is CPU-bound and not GPU-bound.
7. "Also,lights (used for.. well, lightning of cars in vcmod or even simple lights) are just using a bit of GPU but a lot of CPU" Please stop looking at your CPU and GPU usage meters. If the game is using 100% of a CPU core and 20% GPU and then it starts doing an additional GPU-bound task, frames will take longer to render, but your CPU will still be at 100% of one core, regardless of how much extra CPU time it uses. You can't compare CPU and GPU usage percentages like that, it's meaningless and misleading.

7a. "TCB Premium HUD consumed up to 25% of framerate" Meaningless measurement, 25% of framerate could mean anything from half a second to 100 microseconds of render time per frame. The first case is a serious performance issue and the second is a huge waste of developer time if you report it.

7b. "still don't know why, transparency + size maybe" No. Transparency increases the GPU workload very very very slightly. Size on screen is all to do with GPU load as well. CPU load would scale with the number of HUD elements, but all the HUDs I've seen are pretty simple visually anyway.

7c. "Then I noticed that every single HUD was using CPU" Lua addons need CPU time to run? My god.

7d. "if a thread was assigned to HUD and/or transparency stuff, it'd help" No one disagrees that it'd increase performance. But the increase in performance you'd see from shifting HUD rendering to a different thread would be tiny and implementing this is not worth the headache.

I'm sorry, but the reason we're not taking you seriously is because your posts show a huge lack of understanding about the difficulties of implementing multithreading, the problems that it introduces and the expected performance gains from it. You're proposing we make changes to the game's rendering system when you don't seem to have a clue about how it works.

[UltimateByte]

At least now the discussion is (oh noes, was) about how to make it, and not "making it is boring and useless, forget about it".

notcake, you're quite degrading... I'm not a programmer, i'm neither an English native speaker, so i'm doing my best. I'm just an admin and power user... And more i'm into linux servers, web admin, hardware, overclocking, and since about 10 years, into performance analysis, and finally i'm an enthusiastic gamer. I don't know how everything works, i know as a user that gaming experience is degraded because of framerate in many cases in gmod when you try to do cool things. Improving it no matter how, would benefit the whole gmod community, as long as it keeps support for existing addons. And i hope you aggree with that. Nothing is impossible, there must be a good solution, devs (you ? i guess you are one) just need to use their smartness one more time. It's worth the headache to make a source engine game able to use the already old source engine at its maximum capacity and benefit the whole community.

1. See my comment upper
2. Same
3. ok...
4. I never said 30 fps was unplayable (even if i personally think it's shit as i aim 100 fps minimum with my 120hz monitor i won't enter that debate again), but 8 fps is not, and i can tell you for sure that really many players are experiencing this kind of framerate and play anyway because they love GMod and what it brings. BTW around 40 fps isn't amazing... But it's not about how much FPS we personally got but more about how low FPS can go even with kickass computers and so for mid range or a bit old (but multicore) computers.
5. "Lua has the capacity to override the rendering of every single entity and we've already told you why it's extremely difficult to multithread Lua." => Is this lua rendering polygons ? What are you trying to say ? When i say "render" i don't even mean an entity, just a simple prop... Is that rendered by LUA ? Really ? Well if it is, it explains a lot. Then if it's the only issue, is that possible to make a kind of bridge, for example lua calls entities => multithreaded rendering process (java?) => rendering
6. With AA gpu usage is a bit higher and framerate is lower. But the max gpu usage i saw on rp_rockford (made my tests on it) was about 33%. I never said AA was CPU bound, don't make me look like an idiot please. I said AA may involve a bit of CPU according to my results.

7a. You want my full measurments ? Well https://www.dropbox.com/s/4ww4b6wlop6fgke/DarkRP%20HUDs%20Benchs%20-V2.csv?dl=0 (sorry for the form, it wasn't meant to be shared to a lot of people, that's why it's a crappy looking csv)

7b. I didn't push the test to measure GPU usage, i just needed a better framerate for my players. But BTW, the only thing i know is my GPU won't even get close to 35%, so as it wasn't GPU limited, it was obviously CPU limited... So unless you have a good explanation to what i measured, i can't believe what you just said.

7c. Can you really think that lua code for an HUD is CPU intensive enough to drain significant framerate ? My guess until i got a good explanation is that it's rendering that uses CPU.

[UltimateByte]

Well if lua is participating the process of rendering and isn't purely "logical" code, that will explain a lot. If it is the case well i apologize for my ignorance but it's like.. Purely insane for me to think it works this way. And it doesn't remove the fact that multithread is a way to see a big jump in terms of performance.

[neico]

Let's do a quick detour and look at the Dolphin Emulator (GameCube and Wii Emulator), both consoles don't really have multi-threading let alone multi-core support.

Yet they managed it to add that to the emulator with significant performance boosts. Multiplayer is still a bit of a issue but not because of multi-threading (there was a time where it was but they got stuff synced up eventually, you'd need to dive into their code to find the details because I don't have them in my head...) The also had lua up to some point, not too sure of why it was removed in the end though...

It's all a matter of splitting up the work correctly and then syncing it where needed, this however would probably require a significant change on how things are done currently and imo is a bit much for this old engine and the team.

I'd say postpone it to after source2 is released and see what happens afterwards. (shouldn't be that far off)

Oh another pointer I could give of would be the Oxide Mod for Rust, it uses lua as well (together with many more languages), but pretty much event based, not sure if that involves multi-threading though.

Having the choice of multiple languages could be another plus point for gmod in the distant future~

[UltimateByte]

If you're so smart, find a solution.

[robotboy655]

No need for this discussion anyway. We know you want it, but bitching about its absence will not make it come back faster in any way.