Add a NUS througput test as a new shell command

[microbuilder]

Using the nordic UART service, a throughput test should be implemented with the Bluefruit LE Connect app in UART mode on the other end.

Mynewt should be several times faster than the nRF51822 using the Nordic SD since we have a huge packet size of 512 bytes per transactions compared to 20 bytes with Nordic. I'd like to have some concrete numbers here, though, with the shortest connection interval possible.

I think we should be able to send a lot of data over BLE with Mynewt, probably even basic audio like a single channel mic with a low sampling rate ... but concrete numbers will help narrow down what we can and can't do without fancy tricks.

Shell command name: nustest [KBytes] <-- default = 100KB

[hathach]

to be honest, I think we should make an separated project for throughput. That's way it could stay away from the Application image. Although it is small and does not occupy much RAM nor ROM. I don't feel adding it as an feature is necessary. Just a simple project like Blinky should work, right !!??

[microbuilder]

I'm OK making this a separate target but I'd still like to have it as a shell command since it's a good reference and we will use this often with changing data lengths.

[microbuilder]

Everything required to add shell support should be here, though please update if anything has change with the latest Mynewt code: https://github.com/adafruit/Adafruit_Mynewt/blob/master/docs/06_Shell.md

[hathach]

nustest <count> <size>

will send count packets of size to test throughput. For notification, newt core will truncate the data to MTU (20bytes), so size is effective capped at 20.

[microbuilder]

From my understanding of the core spec it should be possible to send more than 20, but lets confirm this with the mynewt dev list.

[hathach]

for reference http://mail-archives.apache.org/mod_mbox/incubator-mynewt-dev/201609.mbox/%3C00de01d215b3%249ceb6ed0%24d6c24c70%24%40tinyusb.org%3E

[microbuilder]

I think this is the link you want :) http://mail-archives.apache.org/mod_mbox/incubator-mynewt-dev/201609.mbox/%3C20160923170232.GE21154%40pseudoephedrine.nightmare-heaven.no-ip.biz%3E

[hathach]

OK here is number with several combination of . To my surprise, the packet of 20 has a better speed than larger MTU. Maybe you want to check it out our your ip6+

I test with our Bluefruit on my Android LG G5. I used a stopwatch to stop when Bluefruit receive enough number of chars. Please note for MTU size > 20, there is 0.5 second delay for MTU exchange procedure. I.e you need to substract 0.5 second for (size > 20).

count size time speed
100 20 1.50 1.33 KB/s 20 100 4.50 0.44 KB/s 10 200 4.40 0.45 KB/s

100 50 4.70 1.06 KB/s 50 100 10.20 0.49 KB/s 25 200 10.10 0.50 KB/s

100 100 19.38 0.52 KB/s

[microbuilder]

BTW ... it's easy to time this on the Central side with our apps ... just switch to 'timestamp' mode in UART and you'll see things on a packet level complete with a ms timestamp, so you just need to compare the time on the first and last entry! Much easier than manually timing.

I just did a test with the OS X version Antonio just updated with some new features, for example, available here: http://openroad.es/projects/bluefruit/install/

running 'nustest 100 20' I see 33,2170 to 33,7730 so 556ms for 100 packets of 20 bytes each or 2KB data, meaning 3.512KB/s, which is still better than I got with Nordic even if we're still stuck with 20 bytes per packet.

[microbuilder]

Here is an example just as a reference (using the OS X version):

[microbuilder]

• nustest 100 20 = 2,000 bytes in 510ms = 3921 bytes per second (3.83KB/s)
• nustest 100 100 = 10,000 bytes in 2.011s = 4972 bytes per second (4.856KB/s)
• nustest 100 200 = 20,000 bytes in 2.101s = 9519 bytes per second (9.296KB/s)

That's fairly impressive to me! Maybe OS X is just way faster than Android, though ... I'll do some more comprehensive testing but this is a good sign so far.

[microbuilder]

PS: Ignore the last test with nusttest 100 200 ... there seems to be a bug somewhere that the command never sends more than 100 bytes regardless of the second param, so 100 100 and 100 200 behave the same.

[hathach]

Thanks, I will leave this statistics work for you heehee. If you test with (100,20), please also (20,100) to see if larger MTU did slow thing down. The large MTU is not just about the speed, it is a nice feature that we could implement large sensor data or long SDEP command over BLE without too many headache. It is nice feature to have

PS: It is not fair to compare nrf51 vs nrf52 . You need to run the test with nrf52 latest SDK + SD. But I think we could ask Nordic directly for this !!!!!!!!!

[microbuilder]

Can you look into why only 100 bytes is going out with 100 200? It isn't urgent .... I have the basic information I need ... but I can't do thorough performance testing as is. Thanks!

[microbuilder]

nustest 1 100 causes the system to crash:

nustest 1 100
Sent 100 bytes (1 packets of 100 size) in 0 milliseconds
687681:Unhandled interrupt (3), exception sp 0x2000a948
687681: r0:0x00000039  r1:0x00000000  r2:0x2000adc8  r3:0x00008fc9
687681: r4:0x00000064  r5:0x2000ce94  r6:0x00000001  r7:0x00000064
687681: r8:0x00000000  r9:0x000a7e3e r10:0x20009050 r11:0x55555556
687681:r12:0x2000a8c8  lr:0x00008275  pc:0x00008274 psr:0x61000000
687681:ICSR:0x00411803 HFSR:0x40000000 CFSR:0x02000000
687681:BFAR:0xe000ed38 MMFAR:0xe000ed34
nustest <count> <packetsize>

[hathach]

That is due to my bad code Divide by Zero exception. The queuing time for 1 packet is < 1 ms --> 0 ms causing the hardfault :( https://github.com/adafruit/Adafruit_Mynewt/blob/master/apps/throughput/src/main.c#L225

We will remove the speed calculation anyway since it does not make any sense now. Just a queuing time. The message is simply as follows

nustest 1 100 Submit 100 bytes (1 packets of 100 size) in 0 milliseconds