Input stops working

[Scrantch]

Hi, I just downloaded and started evaluating your ar-sim tool. Seemed great out of the gate as everything was just working but I quickly found that input clicks just stop working where in the build they don't so it's not the code.

I am using Unity 2019.3.15f1, AR Foundation 3.1.3, and XR Interaction Toolkit 0.9.4 on MacOS 10.15.5 with AR-Sim 1.0.2.

I am placing several objects via my code and then manipulating them with XRIT. As I said, works in a build but in AR-Sim it works for a bit and then just.... stops. No more input is accepted for interacting with objects in AR via XRIT after a short time though the interface still works and I can still place them.

Thanks much...

[marwie]

Hello, thanks for your feedback!

Just to be sure: Are you testing your app with the Device Simulator or in Game View? If Game View: Do you have the Device Simulator window open while testing? If Device Simulator: Which version are you using?

[hybridherbst]

I think I've seen a similar issue in the past but thought we'd fixed it. In addition to the questions above, would you be able to provide a repro project? (You can also share it via email if privacy is an issue).

[Scrantch]

Hi, sorry for the delay.

No, not using the Device Simulator nor was it open. Been using the Game View window. I went through and removed a bunch of packages thinking there may have been a conflict but to no avail. It works for about 30 seconds and then just stops. I am using AR Foundation and XR Interaction Toolkit for input gestures.

Can’t really send you a repo due to security unfortunately. What else can we do? Would love to get this working…

Thanks.

On Jul 30, 2020, at 9:45 AM, Marcel Wiessler notifications@github.com wrote:

Hello, thanks for opening an issue and your feedback!

Just to be sure: Are you testing your app with the Device Simulator or in Game View? If Game View: Do you have the Device Simulator window open while testing?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/needle-tools/ar-simulation/issues/31#issuecomment-666517109, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOLH7OJHABZPZGUHX6OOQITR6GPRPANCNFSM4PLZUSJA.

[marwie]

No worries, thanks for getting back at us. That's really weird - the issues described above only happened in a device-simulator context on iOS I think. I'll setup a project with your setup and try to reproduce it tomorrow!

[Scrantch]

Also, I just noticed that after it freezes when I stop the Game View play an error pops up:

InvalidOperationException: Can't call "GetChanges" without "Start"ing the reference-point subsystem! UnityEngine.XR.ARSubsystems.XRAnchorSubsystem.GetChanges (Unity.Collections.Allocator allocator) (at Library/PackageCache/com.unity.xr.arsubsystems@3.1.3/Runtime/AnchorSubsystem/XRAnchorSubsystem.cs:50) UnityEngine.XR.ARFoundation.ARTrackableManager`4[TSubsystem,TSubsystemDescriptor,TSessionRelativeData,TTrackable].Update () (at Library/PackageCache/com.unity.xr.arfoundation@3.1.3/Runtime/AR/ARTrackableManager.cs:107)

Not right when it freezes but only after stopping Play mode so not sure it’s related but thought I’d send it along. Never see this when running on device or elsewhere.

On Aug 4, 2020, at 9:44 AM, Grant Anderson gma@icloud.com wrote:

Hi, sorry for the delay.

No, not using the Device Simulator nor was it open. Been using the Game View window. I went through and removed a bunch of packages thinking there may have been a conflict but to no avail. It works for about 30 seconds and then just stops. I am using AR Foundation and XR Interaction Toolkit for input gestures.

Can’t really send you a repo due to security unfortunately. What else can we do? Would love to get this working…

Thanks.

On Jul 30, 2020, at 9:45 AM, Marcel Wiessler <notifications@github.com mailto:notifications@github.com> wrote:

Hello, thanks for opening an issue and your feedback!

Just to be sure: Are you testing your app with the Device Simulator or in Game View? If Game View: Do you have the Device Simulator window open while testing?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/needle-tools/ar-simulation/issues/31#issuecomment-666517109, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOLH7OJHABZPZGUHX6OOQITR6GPRPANCNFSM4PLZUSJA.

[marwie]

Also, I just noticed that after it freezes when I stop the Game View play an error pops up: InvalidOperationException: Can't call "GetChanges" without "Start"ing the reference-point subsystem! UnityEngine.XR.ARSubsystems.XRAnchorSubsystem.GetChanges (Unity.Collections.Allocator allocator) (at @./Runtime/AnchorSubsystem/XRAnchorSubsystem.cs:50) UnityEngine.XR.ARFoundation.ARTrackableManager`4[TSubsystem,TSubsystemDescriptor,TSessionRelativeData,TTrackable].Update () (at @./Runtime/AR/ARTrackableManager.cs:107) Not right when it freezes but only after stopping Play mode so not sure it’s related but thought I’d send it along. Never see this when running on device or elsewhere. … On Aug 4, 2020, at 9:44 AM, Grant Anderson @.> wrote: Hi, sorry for the delay. No, not using the Device Simulator nor was it open. Been using the Game View window. I went through and removed a bunch of packages thinking there may have been a conflict but to no avail. It works for about 30 seconds and then just stops. I am using AR Foundation and XR Interaction Toolkit for input gestures. Can’t really send you a repo due to security unfortunately. What else can we do? Would love to get this working… Thanks. > On Jul 30, 2020, at 9:45 AM, Marcel Wiessler @. @.***>> wrote: > > > Hello, thanks for opening an issue and your feedback! > > Just to be sure: Are you testing your app with the Device Simulator or in Game View? > If Game View: Do you have the Device Simulator window open while testing? > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub <#31 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOLH7OJHABZPZGUHX6OOQITR6GPRPANCNFSM4PLZUSJA. >

Hello, thanks for the report - that's a known issue with AR Foundation, which we can't fix ourselves unfortunately (https://github.com/needle-tools/ar-simulation/issues/7) but a bug has been reportet (1268386)

I just setup a project for repro and found that I won't get any Keyboard events after having clicked from GameView to SceneView and attempt to use GameView again afterwards - left clicking still works and after having left clicked in GameView once again Input starts working again. Do you think that could be related to your issue (changing window focus?)

When Input is not working anymore I assume neither left clicking nor right-presseing and using WASD work anymore? Do you think you could send a screenshot of the "Input Helper" component in debug mode once Input has stopped working? (That component will be added to the camera when entering playmode)

[marwie]

Hello, I think I found a fix for the issue - please try the new preview version 1.0.3-preview I just released

[hybridherbst]

@Scrantch as marwie said above just with a Ping :) Please try 1.0.3-preview, you can update to the new version directly from the Package Manager Window inside Unity. Let us know how it goes!

[Scrantch]

Holy cow! It works! Amazing. Doesn’t drop out now and can switch views and come back and it all seems to work. I’ll keep trying it out and see if I run into any more problems. Can you tell me what you changed to make it work?

Also, any chance of adding some additional navigation controls? Would love to be able to zoom straight in to the scene similar to how you can in Scene view.

Thanks!

On Aug 5, 2020, at 10:55 AM, hybridherbst notifications@github.com wrote:

@Scrantch https://github.com/Scrantch as marwie said above just with a Ping :) Please try 1.0.3-preview, you can update to the new version directly from the Package Manager Window inside Unity. Let us know how it goes!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/needle-tools/ar-simulation/issues/31#issuecomment-669341020, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOLH7OKMWRWA77AL2WWYGIDR7GMH5ANCNFSM4PLZUSJA.

[hybridherbst]

We worked around a bug in Unity's code, as usual 🥇 There's some internal Unity stuff that can be used to inject touches. Turns out that breaks when used properly (e.g. injecting touches with incrementing IDs). At a random point, even with all our data being correctly sent to Unity, Unity just stops processing the events it seems.

When only reusing touch 0 all the time (which is not how a touchscreen works) it seems to not break.

Great that it's solved for you! I opened #32 to keep track of future work in this area. Could you open a Feature Request for navigation controls, ideally with some descriptions of what you'd like to see / how you'd like to navigate? EDIT: @marwie added some relevant questions below - @Scrantch would be great if you answer them in that new feature request you'll hopefully create. Thanks!

[marwie]

I'm glad to hear that! Let me know if you run into any more problems or something does not work as expected.

Previously we incremented the touch ID when calling the private touch method that's also used by Device Simulator - but that seems to cause issues at some point, so we changed the code to always use ID 0 (similar to how Device Simulator does it as of now). We will have a look at this again when we tackle multitouch

By zooming straight into the scene do you mean 1) zooming in by scrolling the mouse wheel 2) or "flying forward" just like in scene view using WASD 3) or pressing F to focus/zoom closer on some object in the center of the screen?

There is an option for 2 already ("Fly Mode"). You can change that in preferences to fly mode already

[Scrantch]

Thanks all! I’m glad to hear you are working on multi-touch. Really really need that as well obviously. Any idea when that might be in?

Oh! Fly mode much better. Thanks for pointing that out. Yeah would be great to get a Focus (F) mode as well.

Thanks! Will let you know how it all proceeds.

Cheers... On Aug 6, 2020, 1:29 PM -0700, Marcel Wiessler notifications@github.com, wrote:

I'm glad to hear that! Let me know if you run into any more problems or something does not work as expected. Previously we incremented the touch ID when calling the private touch method that's also used by Device Simulator - but that seems to cause issues at some point, so I changed the code to always use ID 0 (similar to how Device Simulator does it as of now). We will have a look at this again when we tackle multitouch By zooming straight into the scene do you mean

1. zooming in using the mouse wheel
2. or "flying forward" just like in scene view
3. or pressing F to focus/zoom closer on some object in the center of the screen?

There is an option for 2 already ("Fly Mode"). You can change that in preferences to fly mode already — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[marwie]

For multitouch Id suggest to use LeanTouch for now until we have a better solution (ideally in collaboration with the Unity team). We have been using LeanTouch very successfully in production using ARSimulation!

[Scrantch]

Oh cool. Thanks! Will check it out.

On Aug 7, 2020, at 4:01 AM, Marcel Wiessler notifications@github.com wrote:

For multitouch Id suggest to use LeanTouch for now until we have a better solution (ideally in collaboration with the Unity team). We have been using LeanTouch very successfully in production using ARSimulation!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/needle-tools/ar-simulation/issues/31#issuecomment-670461352, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOLH7OO3TZLGCLBJ44D4UCLR7PNIZANCNFSM4PLZUSJA.

[marwie]

Hi @Scrantch just wanting to let you know that I started working on having a focus feature. I created a new issue to track progress. Please reply to the new issue https://github.com/needle-tools/ar-simulation/issues/33 if you have any requests or comments. It would be nice to get some feedback from you if that is roughly what you imagined :)

[Scrantch]

This looks great! How do I go about getting it? Did you release a version with it yet?

On Aug 8, 2020, at 7:13 AM, Marcel Wiessler notifications@github.com wrote:

Hi @Scrantch https://github.com/Scrantch just wanting to let you know that I started working on having a focus feature. I created a new issue to track progress. Please reply to the new issue if you have any requests. It would be nice to get some feedback from you if that is roughly what you imagined :)

33 https://github.com/needle-tools/ar-simulation/issues/33

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/needle-tools/ar-simulation/issues/31#issuecomment-670933738, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOLH7OL5HEY7IJBSLO2E6L3R7VMRDANCNFSM4PLZUSJA.

[marwie]

I havent published a new version with it yet, it will be in the next update tho

[hybridherbst]

Please continue discussions in #33 if there's additional questions :) Thanks!