Motion Updates Sluggish - tvOS 9.1 beta 2

[arvinkx]

I'm experiencing sluggish motion updates, I have a feeling it's something in beta 2 as before using your library I was using UDP and my motion updates were ok but after going to beta 2, my own updates were sluggish. I found this library when looking to solve my problem but after getting this to work in my project I am now experiencing the issues with this as well. I am receiving alot invalid message responses from my central when trying to use the motion updates, do you experience the same on beta 2? Even when it tries to lower the number of updates I am seeing the red flash in the view controller.

[robreuss]

Just tested it and I can confirm that beta 2 is resulting in a slowdown. With the release version, my faster devices would run without any errors, and it was only my 4s that would struggle. Now, with beta 2, even my 6s+ and iPad Pro are generating some errors. As you probably see in the console, I'm backing off the update rate dynamically when malformed messages come arrive. Not that this really matters, but if I disable the motion handler that updates the UI, things improve - but of course that is not a solution.

What do you think the update interval needs to be to deliver a good user experience in a typical game?

I've run into major performance problems with motion on the Watch, which leads me to believe that it is completely impractical as a motion-based game controller (although some game developers have said they are coming out with games that leverage the Watch motion). I think the Watch issues are principally related to throughput limitations with the Watch Connectivity framework. I tried implementing it over HTTP (putting a lightweight server in the Central) but it didn't offer much better performance.

[arvinkx]

I have an inclination that the slowdown may be related to this note in the Release Notes for beta 2 under Networking: • Explicit Congestion Notification (ECN) is now enabled by default on Ethernet and Wi-Fi. This is designed to reduce network delays and reduce packet loss.

I'm not sure what else to attribute it to as it seems there is a major slowdown in networking in beta 2 that wasn't there in beta 1 or the release. The other part that puzzles me is that the Siri remote doesn't exhibit the issues - my game is attempting to use motion from both the Siri remote and another iOS device and the Siri remote is fine.

In regards to an update interval, 1/60 would be ideal, I would think it depends on the frame rate of the game - but I'm by no means an expert in this area. I tried playing with the functionality you have that alters the update rate dynamically in response to invalid messages but I wasn't able to get anything to work reliably. I did back off the time restraint on updating the UI from the motion updates from 0.05 to 0.02 as it was a little jittery otherwise for my game.

I haven't had an opportunity to try motion on the watch yet but that's disappointing to hear. Watch Connectivity framework in my experience has throughput limitations, I can't imagine being able to get any real time info across for a use like this (I think Watch OS 1 was better in regards to that).

[robreuss]

Hopefully this is a bug in beta 2 that will be fixed in the next release.

I think the remote is transmitting over IR, so who knows if that's going through the network layer - IR may account for it's not being impacted by this networking issue.

Apart from the issues with beta 2, do you get the sense that the framework is sufficiently performant? 1/60 was my target interval, and I see good performance at that rate with everything except for my iPhone 4s, which struggles.

[arvinkx]

Hopefully it is a bug because other than that the 1/60 seems to work fine the only times I've run into issues is if the network is weaker (some cases where the router is on the other side of the house there is some noticeable lag at times) but that is expected. I hadn't got as far as falling back to bluetooth yet in my efforts but my experience with bluetooth was limited and I didn't get the results I was hoping for when I looked into it initially. Does this fallback to bluetooth?

Completely forgot about IR - that explains the remote's performance.

[robreuss]

Yes, it is based NSNetService so it will fall back to Bluetooth. I have not been impressed by the performance.

[robreuss]

I'm getting better results with beta 3. Let me know what you see.

I'm curious to know if you need all of the motion inputs (acceleration, attitude, rotation and gravity) or if your game only requires a subset. My goal has been to support all four at 1/60th, but I'm not sure real-world requirements would typically call for all four. Only supporting two inputs also makes things easier for the Siri remote.

Syntax for selectively disabling motion inputs:

 VgcManager.enableMotionAttitude = false

[arvinkx]

It definitely has improved in beta 3 for me, although its still a bit sluggish for what I am attempting. It's very close though. I went further than disabling the motion input groups I wasn't using, I disabled the individual values as well (i.e. stopped sending motionGravityX and motionGravityZ and only Y) and it helped too.

I changed the update interval to 1/70 and I think that is where I would I ideally need it at as it becomes as smooth as the remote but there are some noticeable "jumps" where I think the network can't keep up. I also removed the throttling of the update interval and also the check to ensure the UI is not updated too often (I'm using SceneKit and updating the screen during the update function).

If I could only find a way to smooth out those "jumps" I'd be set...

[robreuss]

Another factor that you could play with is netServiceBufferSize, which is defined in VgcManager. Perhaps increasing it will smooth things out.

[arvinkx]

I adjusted the buffer size without much difference, to be honest even if I lower the game's frame rate to 30 fps and only update motion 1/30s the jumps are still there too...

[robreuss]

By jumps you mean stuttering movement of your on-screen elements? What's the frequency/interval of the jumps?

Might be helpful to make sure that your UI updates are being done on the main thread. I'm using...

    dispatch_async(dispatch_get_main_queue()) {
        // UI Updates
    }

...in the Central sample, but the motion handling code puts it on the main queue at a lower level, so this probably won't make a difference.

[robreuss]

I'll be pushing out two changes today that might help.

1. Found a bug in VgcController where the motion handler was being called twice. It should be around line 1644 if you have the latest, or look for it in the motion profile section:

    func callHandler() {
        if let handler = vgcValueChangedHandler {
            handler(self)
            dispatch_async((vgcController?.handlerQueue)!) {
                handler(self)
            }
        }
    }

1. I'm changing the motion updates to be off the main thread. Since no UI updates are done on the Peripheral side based on motion data, may as well have it on a background thread.

Neither of these have been a panacea in my experience, but perhaps will incrementally help things.

[robreuss]

I'm making good progress on this issue. Adapted the SceneKit starting code (with the 3d space ship) to function as a Central, and that's given me a good testbed for evaluating performance in real-world terms. I'm now get highly responsive and smooth results.

The most significant three optimizations:

1) I've added (but not yet pushed to Github) a low pass filter based on Apple's sample project AccelerometerGraph. Makes a big difference in smoothing things out.

2) I've disabled includesPeerToPeer on NSNetservice.

3) I've disabled browsing for services once a connection is made to the Central.

Those last two changes reflect advice from Apple here: https://forums.developer.apple.com/thread/16069

I'll expose the includesPeerToPeer parameter for developers who want to enable it because they need bluetooth.

[arvinkx]

Sorry haven't had a chance to respond until now but I'm excited to try these changes out as I was struggling to get it smooth enough for a SceneKit game - it was responsive but the movement would stick at times and I would see the red flash indicating an invalid message, hopefully this helps with that.

[robreuss]

I've pushed the version that includes the low-pass/adaptive filter - this has smoothed things out for me. I also increased the buffer value from 512 to 2048, and that also seemed to improve network reliability (fewer invalid messages). The end result is that my iPhone 4s, which had been the worst offender when it came to stuttering movement and invalid messages, is now responsive, smooth and has greater network reliability (fewer invalid messages - near none).

Note that I refactored the location of the properties to modify which motion inputs are active - I moved it to the motion class, which makes more sense:

VgcManager.peripheral.motion.enableUserAcceleration = true

Let me know if it improves things for you. I'm happy to keep working on this - I think it is essential to the utility of my framework that the motion capability is performant and therefore usable.

[robreuss]

I've added a SceneKit demo to the distribution, which is what I used to evaluate the performance impact of my modifications.

[arvinkx]

I can definitely report great results - the motion seems to work very well for my needs at 1/60. I am only using a small portion of the motion inputs but it works smoothly. I think we can close this issue, if you're good with it too.

I've also made some modifications to send a larger amount of data as bytes using custom elements as I was having trouble sending data across (the checksum didn't end up correct on the other side) I can open another issue for that or I can just email you the two files I changed as I'm not positive if it's an issue with the way the dataType .Data is sent or not (I'm not as experienced as you are in this area and would love your feedback on it).

[robreuss]

Glad the motion is working better for you! I'd be happy to help with the data sending issues. I've not experimented with sending larger amounts of data that way, only smaller archived objects.

Send the files along to virtualgamecontroller@gmail.com.