Closed ngokevin closed 8 years ago
ngokevin
commented
8 years ago 
An issue to consider: many titles bind "body" orientation to head orientation. A game where you maneuver a flying vehicle by looking around, for example. Many mobile apps work this way. There is much prior art here that we can learn from.
ngokevin
commented
8 years ago 
I would name controls components based on the input device: hmd-controls, keyboard-controls, gamepad-controls, mouse-controls. They all do the same: change position and orientation of the vr-object they apply to. The logic that adds/removes/enables/disables/configure them depending on the environment (mobile, desktop, hmd available, fullscreen...) should live outside the controls.
Types like body/head don't convey well what they do and are tailored to a very specific use case. What if I want to use WASD to control a vehicle from a 3rd person view? Or to control a crane? Which one should I use? On the vr-components side of things we can provide things like <vr-fps-controls> or <vr-fly-controls> that you can apply to any object: <vr-fly-controls><vr-camera></vr-camera></vr-fly-controls>. They are different configurations of the more basic controls components: hmd-controls, keyboard-controls....
ngokevin
commented
8 years ago
I can see the logic in separation by input. I'm less clear on how a dev would use it to set up a basic setup. For example, how would I implement the following:
The logic that adds/removes/enables/disables/configure them depending on the environment (mobile, desktop, hmd available, fullscreen) should live outside the controls.
Same question as above. What would it look like in practice?
Types like body/head don't convey well what they do and are tailored to a very specific use case.
Agree that devs need a way to design experiences where body orientation follows head orientation. In a body / head / cursor architecture, perhaps this could be achieved with a setting that binds one component's orientation to another's. For example:
We should also consider Martin's strong advice: that if we believe social is important to the future of webvr, that we incorporate avatar primitives into our system from the start.
ngokevin
commented
8 years ago
Setting aside the implementation, the desired experience for 1.0 looks like:
https://docs.google.com/spreadsheets/d/1flEJ_81DfcgkwRN740wT_-X_27Zq1CBzxI0zxKGCDiM/edit#gid=0
That's a breakdown of responsiveness for entering VR and interacting with content across desktop and mobile.
ngokevin
commented
8 years ago 
Naming is different, but the scheme is essentially the same as what @jcarpenter posted.
This is what it might look like as markup implementation:
<!--
Rotations and positions controls are composed onto the entities object3d.
Seperation of concern is by:
- Entity component property (orientation, position)
- Input type and actions.
-->
<!-- ORIENTATION -->
<!-- hmd drives state.orientation drives entities rotation on axis.-->
<vr-object orientation-controls="input: hmdOrientation">
<!-- mouse drives screen x, y drives entities rotation on axis. -->
<vr-object orientation-controls="input: mouse">
<!-- Q, E and arrow keys drive entity rotation on world Y axis. -->
<vr-object orientation-controls="input: keys">
<!-- DeviceOrientation (mobile) -->
<vr-object orientation-controls="input: deviceorientation">
<!-- Gamepad analog stick -->
<vr-object orientation-controls="input: gameAnalogStick">
<!-- On-Screen analog stick (mobile) -->
<vr-object orientation-controls="input: screenAnalogStick">
<!-- when specified, displays screen control overlays -->
<!-- POSITION -->
<!-- drives entity position using A, S, D, W keys. -->
<vr-object position-controls="input: keys">
<!-- Forward is world Z axis with entity rotation applied. -->
<!-- Assumes world Y up and movement on world X, Z planes. -->
<!-- drives entity position on hmd positional camera -->
<vr-object position-controls="input: hmdPosition">
<!-- position is with entity rotation applied. -->
<!-- gamepad dpad -->
<vr-object position-controls="input: gameDPad">
<!-- on-screen DPad (mobile) -->
<vr-object position-controls="input: screenDPad">
<!-- when specified, responsible for painting controls onto screen. -->
<!-- CURSOR -->
<!-- provides object intersection or raycast testing and emits appropriate events on those entities. -->
<vr-object
cursor="type: raycast; input: click, gameButton, fuse">
<!--
type: raycast
- raycasts from entity orientation and position into scene (or assumed to be viewport center if camera)
- dispatches events on intersected scene objects when input is triggered.
type: screen
- works only when attatched to camera/view transform.
- raycasts from OS mouse x, y position on screen into scene.
type: area
- AABB bounding box intersection with scene objects.
- dispatches events on intersected scene objects when input is triggered.
input: click, gameButton
input: fuse
- fuse timer begins when intersected object is detected to have event handler/attribute.
- dispatches event when timer expires.
-->
<!-- DEFAULT RESPONSIVE MODES -->
<!-- if just the component attribute is used, the component will default into "responsive" mode. In this mode, the following component attributes are assumed -->
<!-- desktop Mono mode: -->
<vr-object
orientation-controls="input: hmdOrientation, mouse, keys, gameAnalogStick"
position-controls="input: hmdPosition, keys, gameDpad"
cursor="type: screen; input: click, gameButton">
<!-- desktop VR stereo mode: -->
<vr-object
orientation-controls="input: hmdOrientation, mouse, keys, gameAnalogStick"
position-controls="use: hmdPosition, mouse, keys, gameDpad"
cursor="type: raycast; input: click, gameButton">
<vr-object id="cursor" position="0 0 -100"><!-- mesh cursor -->
<!-- mobile Mono mode: -->
<vr-object
orientation-controls="input: deviceorientation, screenAnalogStick"
position-controls="input: screenDpad"
cursor="type: screen; input: touch">
<!-- mobile VR stereo mode: -->
<vr-object
orientation-controls="input: deviceorientation"
position-controls="input: [none]"
cursor="type: raycast; input: fuse, gameButton">
<vr-object id="cursor" position="0 0 -100"><!-- mesh cursor -->
<!-- Desktop with motion controllers: -->
<vr-object
id="avatarHead"
orientation-controls="input: hmdOrientation"
position-controls="use: hmdPosition">
<vr-object
id="leftHand"
geometry="primitive: sphere"
material="color: blue"
position-controls="input: motionLeftPosition"
orientation-controls="input: motionLeftOrientation"
cursor="type: area; input: motionLeftButton">
<vr-object
id="rightHand"
geometry="primitive: sphere"
material="color: red"
position-controls="input: motionRightPosition"
orientation-controls="input: motionRightOrientation"
cursor="type: area; input: motionLeftButton">
A spaceship that flies in the direction you look. Orientation on parent object, camera on child object?
<a-object id="spaceshipVector" orientation-controls="input: hmdOrientation">
<a-object id="spaceshipModel" geometry="url: model.dae" camera="fov: 45"></a-object>
</a-object>
@jcarpenter If you added it as a child, the spaceShipModel would follow the rotation of the HMD continuously. Which probably isn't the effect that you want.
To do what you want, it would involve some javascript:
The rotation would be retrieved and applied to spaceshipVector at the time of a event eg. click.
ngokevin
commented
8 years ago Don is solving this with general controls="position: ...; rotation: ..." API which is similar to what Casey described. Being tracked at https://github.com/aframevr/aframe/issues/975
Discussed control scheme with @caseyyee, per action item from weekly sync. Tried to come up with a proposal that captures immediate and future needs.
Proposal for A-Frame controls
Priorities
5 Add look-based cursor
129 Add fuse-based cursor trigger for mobile devices w/o inputs
Proposal
Naming convention proposed in #241 is:
keyboard-controlsmouse-controlsWe should instead group controls into "types", within which multiple input devices are possible. For example:
body-controlscursor-controlsFor additional separation of concerns, we should also move head orientation logic into it's own
head-controlscomponent.Components
body-controlsContains logic for user position in space.
Support for:
cursor-controlsContains logic for cursor.
Support for:
head-controlsContains logic for head orientation.
Support for:
Responsiveness
For example:
v1:
body-controldefault is keyboard (option to disabled)cursor-controldefault is mouse clickhead-controldefault is mouse dragbody-controldefault is no support (devs can always implement own solutions with inline JS).cursor-controldefault is fuse-cursorhead-controldefault is device orientationv2:
Additional devices are supported, and devs can now indicate a preference.
Way of doing so could by ordering, with the system going through the user specified options and falling back to the next if support is unavailable. For example, a user who visited the following site with OS X Safari would get pointer-lock controls.
<a-object head-controls="input: hmd, pointer-lock, mobile">Or a developer could decide to create a non-VR experience by only providing mouse-drag head control:
<a-object head-controls="input: mouse-drag">