search

wonderunit / storyboarder

✏️ Storyboarder makes it easy to visualize a story as fast you can draw stick figures.
https://wonderunit.com/storyboarder
3.17k stars 293 forks source link

Shot Generator 2.0: mobile control via phone doesn't work #1663

Open audionerd opened 5 years ago

audionerd commented 5 years ago

For security reasons, deviceorientation events are no longer available to mobile browsers from HTTP sites. This breaks our mobile controller.

For now, we can serve via HTTPS with a self-signed cert, and users who want to use this feature will have to either bypass the private connection warning or install the cert on their device.

But if we do that, Web Sockets won't work, because they won't accept a self-signed certificate!

TASKS

audionerd commented 5 years ago

Deferring this for now. devicemotion and WebSockets require HTTPS, but with self-signed certs that won't work in modern mobile browsers. We'll need to find another way to do phone control.

DominikNovosel commented 2 years ago

Now that Storyboarder uses P2P to communicate with other devices (#2129), will this functionality be added back in? It looks pretty useful and it's a shame I never got the chance to try it out before it became obsolete.

audionerd commented 2 years ago

It might be possible to add that feature back in the future. Something like:

@charles @SergiiSharpov Is a feature like that part of the plan for AR? https://github.com/wonderunit/storyboarder/tree/ar

Maybe could even use RTCDataChannel to avoid remote server roundtrip latency when forwarding the devicemotion events?

Relatedly, we had gamepad control at one point, which we also had to drop for technical reasons. Could be revisited.

DominikNovosel commented 2 years ago

Sounds great! Having more accessible and easy to use input methods that don't require exotic or expensive hardware is definitely welcome. This would be great for people who want more control over the scene but aren't willing to invest in a VR headset. I saw what I believe to be the mobile interface at #1533, and had a minor quibble about it, you might want to check it out.

DominikNovosel commented 2 years ago

Thought: basic VR functionality might not require a full-blown headset and can be hella cheap Perhaps a subsystem could be devised where a single user has two phones, with one being slotted into a Google Cardboard device acting as the viewer and the other acting as the controller. If we approximate the position of the controller relative to the headset, we don't need to track its position (we couldn't even if we tried, something else is needed, see below), only its orientation (this is a pretty big compromise and requires a separate UI system just for this configuration 😟).

Alternatively, if we were hell-bent on tracking the position as well, a cardboard fiducial may be mounted atop the controller phone and tracked by the viewer's camera using the gyroscope for assistance if it loses sight of the fiducial. This, however, is ludicrously overcomplicated to create just for Storyboarder and would probably be better off in a separate library or WebXR itself. Historically, there were at least two implementations of something this, but without the phone. I tried out two of them and they worked quite well.

  1. Realiteer RealControl (https://www.google.com/search?channel=fs&client=ubuntu&q=realiteer+vr)

    • RealControl v2 has a single trigger that can be used for additional input
    • last time I checked (about 3 months ago), both of the compatible games I could find didn't work due to failure to initialize Vuforia

  2. PaperStick (http://paperstick.net/)

There are a few ready-made solutions such as TrinusVR, but they are geared towards tethered desktop VR and are kind of cumbersome to set up.

Also, this repo proposes several ways to do tracking using PS3 Eye cameras, but my goal for this proposal is to minimize the need for hardware.

Still, I think I might be onto something, even if not for Storyboarder in particular.