[docs] Write guide for "Building User Interfaces in VR"

[ngokevin]

Description:

For A-Frame, this is uncharted territory for a lot of us. But let's get a start on it, and build from there. People with experience include:

• @feiss - A-Painter Interface
• @kfarr - City Builder Interface
• @mflux - dat.GUI VR

Throwing up a layout because I don't have much experience, but we can look at the implementations above and explain how they were made.

1. Building Interfaces with HTML
2. Building an Interface Using an Image
3. Building an Interface with 3D Objects

[mflux]

Is this about building an interface-builder in VR? Or assembling an interface in HTML, then have it show up as a VR interface?

This is definitely hard because there are no user guidelines and best practices for what a user interface should be like in VR. A lot of UI is trending towards laser-pointer based because we're so used to 2d surfaces and mouse.

Is the focus on data-display? Or about button pushing (like implementing levers, knobs, dials, buttons, etc)?

What's the goal here?

[ngokevin]

I'm thinking more 2D interfaces that can be interacted with controllers. True, there's no one way to do VR interfaces, but it's better to have a guide showing a few implementations than nothing at all. That way, at least the community can build off of it. As we converge to more best practices, these guides can always be updated.

For levels, knobs, dials, buttons, I have an issue for "Adding Interactions in VR" guide https://github.com/aframevr/aframe/issues/2274

[machenmusik]

It was very difficult to find one set of movement/interaction code to accommodate mobile, typical desktop/laptop web, phone VR, PC VR, and roomscale, controllers / gamepad / remote. Even just the VR subset of that was challenging.

[mflux]

Boris' Ray-Input will help tremendously https://github.com/borismus/ray-input as it adapts to Daydream, Cardboard, Vive/Oculus, as well as mouse input types.

[machenmusik]

... touchscreen for non-VR mobile? touchpad on laptops as well as mouse? what if you want movement not just gaze selection?

[machenmusik]

feels almost like a topic with blog posts from various folks on what worked well and didn't -- here, with an eye on implementing with A-Frame...

[ngokevin]

The guide will be targeted towards VR where you have positional tracking and 6DoF controllers.

[machenmusik]

ok great that makes it much more straightforward, although perhaps less widely applicable (thinking of Daydream 3DOF, if not GearVR)

[ngokevin]

Yeah. The future of VR's lowest common denominator at some point will be positional tracking + 6DOF controllers, so that's what we're aiming for.

[feiss]

With all my respect (and at least regarding A-Painter), I wouldn't call our projects as examples of 'best practices', but 'first practices'.. We had a problem or goal, and we solved it in the best way we could using the time we had. But that does not mean that was the best way of doing it, or that other UIs should be done that way..

I'm talking about the technical implementation of a certain UI. When talking about the conceptual design of VR UIs, I think nobody knows the best practices yet, this is a barely explored territory..

What I think @ngokevin wants is to write about the different ways of implementing and drawing UI elements using A-Frame. Something like what he has in the last bullet list of the first post: how to make ui elements using html tags only, using images, 3d objects, even procedural objects...

Teaching people different ways of making a button in A-Frame, not telling about how a button in VR should behave or look like, or how responsive VR UIs should be.

[ngokevin]

Yup, pretty much wanted a "first practices" guide on some existing implementations to get a start, not a Bible. We can speed up experimentation if we provide a foundation using what we know so far rather than leaving it at "no one knows".

[Utopiah]

I confirm this is a recurrent need for newcomers.

[kfarr]

Hey all, great conversation sorry for being late to jump in. I like the idea of breaking this up into a series of blog posts about "VR UI First Practices" that includes both the UI challenge scenario and then the code-based solution with a link to a repo or component at the end. I'm happy to take one on with regards to city builder. My topic could be something like - "creating interface for previewing and selecting objects across Oculus and Vive".

Maybe if a few more folks can commit to topics and a proposed date for their writeups? I could commit to a first draft by Jan 30 for mine. And then someone from Mozilla can make a nice intro / context post.

On Fri, Jan 13, 2017 at 6:02 AM, Fabien Benetou notifications@github.com wrote:

I confirm this is a recurrent need for newcomers.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aframevr/aframe/issues/2281#issuecomment-272449499, or mute the thread https://github.com/notifications/unsubscribe-auth/AActzVjompS8r3GrH7nOcER2_9IFydyRks5rR4P0gaJpZM4LidW9 .

[ngokevin]

That'd be great! Yeah, if anyone wants to volunteer a case study, that'd be useful for people. Else at some point, someone or me can read through some of the implementations and regurgitate it.

[rapoulson]

Are we definitely looking for all original content here or would a page with curated links with a blurb about what they're good for also be helpful?

[ngokevin]

I think both. Some original content with describing a couple implementations that we have so far, with links out for further reading. As we evolve our knowledge and prototype more in VR UI, we add and update content.

[rapoulson]

Cool, I haven't built anything with 6DoF (yet!) but I'd be happy to get a further reading list started.

[ngokevin]

@imgntn is now working on this.

@rapoulson Would love to see a further reading list if it's not too much work. Not sure what would be useful, they'd mostly be for inspiration? Not much content right now describing building room scale interactions for WebVR.

[ngokevin]

Sorry, wrong issue. The comment above was for the Interaction Guide. No one is working on this currently.

[kfarr]

Okay did a first draft here, comments enabled and welcomed: https://docs.google.com/document/d/1RvmQ-ExpLIQFr8kK4d67jndI070dCtz3AI2BeJYYJjY/edit

[ngokevin]

@kfarr Thanks, will take a look right after this A-Frame release.

[kfarr]

I've revised (edited down) and prepared a Medium post. It's ready to publish, but I'm in no hurry as I'm hoping that we'll have a stable Chromium build soon so people can actually use this: https://medium.com/@kfarr/38b0c86ed7b7

[jjhesk]

we need a builder interface like unity3d