search

rust-gamedev / wg

Coordination repository of the Game Development Working Group
514 stars 10 forks source link

Planning a survey for Rust game developers #21

Open erlend-sh opened 5 years ago

erlend-sh commented 5 years ago

I mentioned several weeks back that I wanted to do a survey. Will try get that drafted here now. I plan to do something similar to what I’ve done before with Discourse. Main difference will be that I plan to source the survey answers through issue/forum replies, so survey answers are public to all (except for private answers sourced through a Google Form).

The survey will consist of a few simple questions, with mostly (or completely) freeform answers. If you have some pressing questions for the Rust gamedev community, please bring them up here for consideration. If we find ourselves with an abundance of good questions we’ll just have to do multiple surveys over time.

Questions I’d like to ask the Rust game dev community:

bitshifter commented 5 years ago

I know of many people working in game development, both AAA and indie who are Rust enthusiasts but aren't using Rust in their professional work. Would this survey cover them too?

erlend-sh commented 5 years ago

Would this survey cover them too?

As long as we're able to reach out to them and get them to answer the survey, absolutely!

I think we should add one more question then, conditional on checking "professional":

msiglreith commented 5 years ago

Awesome! Sounds like one of the most important tasks to me. Getting a better understanding of the different subareas (audio, graphics, threading, math, physics, etc.) would be ideal as basis for further discussions. Would be interesting to see which libraries are used in the specific areas and if people are rolling their own ones, why so?

Lokathor commented 5 years ago

No one has more NIH in their blood than a game dev does.

erlend-sh commented 5 years ago

Survey has been published:

https://users.rust-lang.org/t/survey-from-the-rust-game-development-working-group/31270

msiglreith commented 5 years ago

I would be interested what's the state of this topic?

ozkriff commented 5 years ago

https://twitter.com/erlend_sh/status/1167343250258223105

Yep! The survey is considered soft-closed now and I’ve begun summarizing the answers. We got 360 responses so it’s gonna take me a few days though.

erlend-sh commented 5 years ago

Yup, still on this! Been quite occupied lately but it’s 80% done now.

Wodann commented 4 years ago

@erlend-sh Were you able to wrap up the results?

erlend-sh commented 4 years ago

Survey results were at long last finished up today. I’ll make a PR tomorrow.

ozkriff commented 4 years ago

This can be closed now, right?

AlexEne commented 4 years ago

Yup. Thanks for all the hard work everyone!

erlend-sh commented 4 years ago

I'm preparing a new survey for 2020. You can test it here:

https://docs.google.com/forms/d/e/1FAIpQLSfu1gAi_Gbp8ODJY3tYkXkYS6nQI0G_qaHvwkIfGNNljuETdg/viewform?usp=sf_link

I changed last year's free-form entries into a ranked-choice vote.

repi commented 4 years ago

@erlend-sh nice! any specific feedback you want on it? When do you plan to send it out?

erlend-sh commented 4 years ago

Sending it out in a week or two.

I’d love to know if there is something more specific we could try dig into in addition to the “most important?” query.

Maybe another ranked choice for “What has been taking up more of your development time than you’d like?”, and we could list all major areas of development like Graphics, Physics, Networking, Input, Audio etc.

What do you think about the free-form question?

What has been the GOOD (worked well), the BAD (difficult) and the UGLY (works, but...) of your Rust gamedev journey so far?

I want to solicit brief stories from people.

17cupsofcoffee commented 4 years ago

The ranked choices seem like a good idea (nice to have a mix of hard data and free-form feedback) - wonder if it might be worth having an optional field where people can list anything they feel is important that was left off the ranked list? I can't think of anything off the top of my head, but it might be handy for refining the list next time round.

kvark commented 4 years ago

I feel like the "What is most important for you right now as a Rust game developer?" question is a bomb there. All the other ones are small and quick, but this one is heavy and overloaded. Could it be split into several questions, touching different areas? For example, one question could be about platform support, such as Console/Web/Mobile, integrating with toolchains. Another - about dev experience, including IDE, compile speed, and documentation. Splitting would also help to solicit separate pieces of feedback on those areas in custom "type what you want" fields.

erlend-sh commented 4 years ago

Yes! I was thinking exactly the same thing. Respondents shouldn’t need to rank more than 3-5 items at a time.

erlend-sh commented 4 years ago

Here's what I've come up with:

https://www.surveymonkey.com/r/F2JYRFF

@17cupsofcoffee None of the survey systems I've tried have a "custom entry" option to go with a ranking question. I think the workaround will just have to be the last free-form question.

kvark commented 4 years ago

Most important for ecosystem maturity

It's weird that one can both set the priority numbers and re-order the entries. Just re-ordering would be nicer.

We could also add "polish existing crates better", since this need doesn't get covered by the listed entries.

Most important for a "full-featured" Rust game engine or framework

This could have an additional "Performance" choice.

Most important ecosystem-segment to contribute to right now

This could have an additional "Organization" kind of choice for ECS and things

Good,Bad,Ugly

Can we have those multi-line? If somebody wants to provide more info, we don't want to penalize this.

17cupsofcoffee commented 4 years ago

The ranked choice questions look way better now than they did with the big 1-10 grid 👍

erlend-sh commented 4 years ago

We could also add "polish existing crates better", since this need doesn't get covered by the listed entries.

Is it meaningfully different from "More v1.0 crates, less churn"? More v1.0 crates is what "more polish" looks like to me in the less abstract. How do you measure "polish"?

This could have an additional "Performance" choice.

✅

This could have an additional "Organization" kind of choice for ECS and things

✅ Added as "ECS and similar"

Can we have those multi-line? If somebody wants to provide more info, we don't want to penalize this.

Unfortunately we cannot. As a workaround I added a notice about it:

"Note: This would be a multi-line field if we could make it so! Feel free to write multiple paragraphs."

I'm gonna go public with this in 24hrs if there aren't any objections.

kvark commented 4 years ago

I just cringe every time I see "more v1.0 crates", since versioning is very loosely related to the actual level of completeness and polish. Some authors kick off with a 1.0 release (e.g. https://crates.io/crates/density-mesh-core). Some crates go nuts and just keep releasing major versions ("nom", "intl_pluralrules", "triple_buffer", "raw-cpuid"). Extreme examples reach double-digit major versions without any trouble (looking at you, "core-text", "wast", "glean-core"). Is that what the ecosystem needs, really? I'd rather have more polished v0.x crates than just more v1.0 crates :)

erlend-sh commented 4 years ago

Some crates may diverge from the SemVer standard, but afaik it's still the best available measure we've got for API stability. What I'm trying to get at here is that in a year from now, we'll want to measure somehow if the overall stability/polish of crates went up in a meaningful way (and if so, then dig into the cause behind the effect). We definitely don't need to constrain ourselves to any single data point; the more the merrier.

I've changed the text to "More stable crates, less churn".

aclysma commented 4 years ago

Feedback for https://www.surveymonkey.com/r/F2JYRFF:

Do we have multiple people with access to the results of this survey?

Thanks for putting this together!

erlend-sh commented 4 years ago

Do we have multiple people with access to the results of this survey?

I'll share it with anyone in the WG org who requests it. My email is public on my profile.

Survey is out! 📬

https://twitter.com/rust_gamedev/status/1324349001706229760 https://users.rust-lang.org/t/rust-gamedev-ecosystem-survey-2020/51094 https://www.reddit.com/r/rust/comments/joj5e0/rust_gamedev_ecosystem_survey/

neachdainn commented 4 years ago

It's probably too late for this but I would like to point out one thing in regards to the survey questions:

There is a non-zero percentage of the Rust game-dev community that aren't making games but are making simulations. For the most part, the two overlap in that they utilize much of the same techniques and technologies. However, the end goals are different - my programs are likely to only be used by me and aren't intended to be entertainment. The survey is currently written in a way that makes it difficult for me to answer in a useful way. Am I using Rust for game development? Well, no. Am I using Rust professionally (academia) for simulation? Yes.

It might be the case that I'm talking about a demographic so small as to not be interesting, or maybe I'm mistaken in simulation also falling in this working group, but providing a way to indicate between the two might provide useful information.