What do "game developers" want from the Rust runtime?

[toddaaro]

Something that is pretty undefined right now is what the end goal of the Rust runtime's concurrency/parallelism capabilities are. We have Servo as a motivating application, and there is a solid academic background and some web programming expertise around, which is great. But we do not have game development expertise around, and that seems to be a demographic very interested in Rust.

This article was mentioned by "nawfel" in #rust this morning and I think is a great piece to help us figure this sort of thing out. I haven't read the full article, but it describes a concurrency library in use today and gives motivating examples and discusses design decisions. There are also links to more relevant articles.

http://gameangst.com/?p=526

The "actual issue" here in the short term is to start finding sources like this and getting more concrete wants out of the demographic so we can start thinking about them in the context of the Rust runtime.

This is tagged as a metabug, so we can start collecting issues here. Currently there are no issues to link though.

[emberian]

cc @dobkeratops @Jeaye

[dobkeratops]

IMO - [1]the first useful thing that springs to mind is the zero.rs idea of a subset running without the runtime - i gather this might mean splitting the stdlibs into 2 layers. [1.1] this would facilitate ports.
There is a niche in gamedev for being able to get onto new platforms first - my own career relied on that... launch titles for new consoles, in effect - doing games before the tools were widely available.(after doing a hit driving game on the PS1, we were approached to do one for dreamcast whilst that platform was in development, and then ported this for launch to the xbox - this would have been impossible if we had dependancies on external libraries and were waiting for those to be ported)

[1.2] Its much less common now , but some game-developpers still like to minimize use of dynamic memory allocators, and we certainly did "back in the day" (PS1/DC) -we ran without an allocator, just had dedicated pools for allocating actors and graphics resources. The logic of the engine itself involved measuring available memory in a custom pool and throttling the scene (eg prioritizing objects to display). intead of trying to allocate and handling OOM. This approach carried through to our streaming system on the xbox360.

I recall an obscure bugs from with virtual memory interacting with runtime libraries creating graphics resources whilst decompression threads were consuming data after asynchronous loads. i think Gl implementations are usually general enough that you wouldn't run into this,but this was with the console customized apis.

[2] The rust ideaof lightweight tasks does seem rather useful; Exposing some control over how these tasks map to physical cores - perhaps being able to spawn tasks 1:1 with cores, manually (bypassing the scheduler) might be nice- gamedeveloppers do like to feel in control.

[3] Another idea that might be worth mentioning is control over the growable stacks; ithink there's been some discussion on that and I'm not entirely familiar with the details here;

• currently there is a little overhead with function calls for checking if stacks overflow? if you could disable this check/grow behaviour on a task by task basis that might put peoples' mind at rest on that front - also provide a 100% 1:1 mapping with what you get from a C compiler

[3.1] Would it be possible to do something like the "leaf-function" idea inspired by how register saving wored on MIPS. Lets say the stack was always reallocated with some fixed over-estimate, any leaf function that is known to use less than this can omit any checks. I would hope this would allow the best of both worlds.. minimzing the overhead of the growable stack and being safe- .

[4] dont add exception handling :) rust already gets that right i think. Gamedevs usually use C++ with exceptions disabled. This might be worth mentioning r.e. [3],[3.1] - the reason we didnt use exception handling is we didn't like the idea of extra overhead on the stack associated with function calls- its possible hearing that "stacks check to reallocate" might put some gamedeveloppers off.

(now, Apologies if i'm rambling or diverging from the original question, the Runtime) Overall from my experience in game development the things that attracted me to rust generally are:-

[1] the open-world replacement for classes - avoiding the problems of headers&classes in c++. better ability to decouple code whilst using the friendly a.foo(b) syntax with autocomplete and easy chaining. We had terrible problems with this in C++ over the years especially as the company grew, merging work between source-bases. This is where i get the desire for an option to impl for types in other crate when making exe's instead of libs.. with a warning instead of an error :)

[2] the promise of a language better at (compile-time) reflection (serialization in particular) Game development is very data driven . Getting between creation tools/export pipeline & runtime is a big deal.

remembering all the hacks with C x-macros or manually keeping serializes/deserializers/runtime representations in sync, or generating debuug UI's etc. game engines frequently had their own hacks for getting around that and I always looked at c# with envy. multi-item macros would be a great feature here,

[3] the functional 'flavour' (const by default etc) - I became interested in this after experience parallelizing on the xbox360(ILP&SIMD, not just threads); it seemed the whole 'map/filter' mentality was highly applicable to the challenges there. I think the ideal would be to write code in a 'functional style' with iterators that expect pure functions/predicates which can then compile (if need be) what you wrote to patterns without branches, or turning decisions into SIMD masks or whatever... whilst not having to completely butcher your source like we did on those platforms . Now there's 8way simd, and gather instructions on intel.. C++11 got lambdas about 5+ years too late for us. i've been encouraged by the way the chainable lazy iterators work in rust.

[3.1]Even if you have "enough processing power", parallelizing can reduce latency.

[4] the concurrency oriented memory model, seperate heaps per task, seems perfect. As a gamedevelopper i had never liked the idea of multiple threads poking into the same heap.

[dobkeratops]

is there an expplicit "yield" and/or ability to make a task sleep until woken by a message , that would be nice aswell

[emberian]

@dobkeratops very valuable feedback, thank you!

As for yield, there's actually no implicit yield. It's always explicit, it's just that sometimes the runtime will call the yield function for you: std::task::deschedule (because yield is a keyword now, for coroutines post-1.0)

[dobkeratops]

Something else r.e. @ptrs: ... consider the use of C++ in conjunction with an embedded language for scripting or application code ; Unity is gaining popularity and that's C++ low-level engine & GC'd C# for the app; .. I get the impression the team is in favour of removing @ptrs, but it seems to me that rust could have an interesting niche being able to closely integrate both styles of code. Thats got to be better than having to deal with interfacing 2 languages.

(my mythical "perfect langauge" would also have an interpreted subset for rapid iteration.. ) i liked the ideas about making @'s behaviour replaceable program-wide, but i understand you can hardly chop and change between rc & gc with the same libraries.

Whenever i see a GC system i always ask for a once-per-frame callable full collect (or call it at other convinient points.. after level load etc..). I'm usually told its sub-optimal compared to fully developped GC systems, but maybe something like that would help early on, and its a hint that could be ignored if you get a superior implementation..

[jeaye]

@dobkeratops mentioned that Rust should never get exceptions. While I'm not against this idea, I will say that game developers (especially in C++ land) often rely on some way of unwinding when things start going wrong. Of course, for a proper game, a fail!() is simply not going to cut it. Where I work, we use exceptions for this, since it's guaranteed that everything in scope will be destroyed (server connections, In-App-Purchase managerment, file handles, etc). I don't know of such a construct in Rust.

[thestinger]

Catchable exceptions won't be implemented due to the memory safety guarantee. It's just not going to happen.

Why isn't fail!() enough when you have an error you don't want to handle (via conditions or sum types)?

[jeaye]

Why isn't fail!() enough when you have an error you don't want to handle (via conditions or sum types)?

In the current system I work with, we throw a runtime exception if the game receives bad config, can't connect to the game server, or something else fatal occurs. There's quite a bit of state to clean up between where we get that error and where we want to prompt the user that something bad just happened. Furthermore, we cross a couple app boundries between native C++ and, for example, Java code (where there's not a very clear route 'back').

If this were Rust, I don't see a good way to guarantee everything is unwound properly. Do you?

[dobkeratops]

i suppose a rust- friendly method could involve spawning tasks and message passing? .. failure shutting down a task.. but that sounds like a completely different architecture to what you describe

[thestinger]

Task failure is just an exception you can only catch at a task boundary. I'm not sure where the problem would be.

[thestinger]

There are no criteria for when this could be closed, so I think it belongs on the mailing list instead.