microsoftarchive / iot-journey

a guidance project for implementing an IoT solution on Azure
Other
89 stars 34 forks source link

Larry/fix faultytiming #224

Closed lbrader closed 9 years ago

lbrader commented 9 years ago

connects to #177

In fixing this issue, it helped us define bit more to understand and validate our accuracy in our sending per seconds. To help to quantify our timing accuracy added & fix unit tests.

In going through our timing implementation. It's worth going a bit deeper to understand the moving parts that this PR should fix. Since it's a major control in our simulation coordination for sending events from the devices.

And always looking for improvements.

Device.cs contains our device model and has a message loop in RunSimulationAsync method. The loop is emitting message(s) at certain frequency to the event hub. In this case we are only sending 1 message type at time. And is running the code for our Task.Delay(loopfrequency). Our loopFrequency set's the frequency of when we are going to check our elapsed time. It's set to 300 ms.

The issue was improving our accuracy because the imprecision that occurred in measuring our elapse time with the our desired frequency. Via testing found we could lose up to 200 to 299 ms. Per loop.

The solution was to calculate the remainder time, which is the we were off from the frequency we wanted. The next loop would start at this remainder. This was fixed in the ResetElaspedTime in Entry.cs

Entry.cs which handles the timing for its message for a device with the following 3 APIs:

Adding the remainder meant we had to handle the initial condition where we get a high negative remainder it helped us go from an estimated 30 % accuracy to a measurable 85 to 95%. Depends on your sampling frequency. One of the tradeoff's is CPU.

So this also starts a way for us to measure and tune the simulator throughput performance of our system.

The Unit test are in the ScenarioSimulator.Tests in WhenTrackingEventsToSend:

The JitterShouldOscillateAroundSendingFrequency() is the method that works on quantifying the accuracy.

bennage commented 9 years ago

@lbrader I separated the unit test for the time remainder from the test for jitter. I also did a little bit of house keeping. Please make sure that I didn't remove anything that I shouldn't have.

lbrader commented 9 years ago

@bennage Looks good Christopher. The unit test ended up with a lot of instrumentation to validate assumptions, but the boundary test itself is all you need for the unit test.

bennage commented 9 years ago

Let's rebase this before merging.

mattjohnsonpint commented 9 years ago

I did a little cleanup, but the logic is sound. :+1:

mattjohnsonpint commented 9 years ago

Updated that one comment and rebased (squashed).