Printf executes too long for displaying data

[yzliao]

We setup the sampling frequency to be 100Hz but the log file shows the sampling frequency is 74Hz. Some data are missing in this case.

[bufordsharkley]

Reading the data as serial input, translating to ASCII, printing to screen-- that all takes time.

...I've faced a similar problem in the past, and instead saved all log files as pure bytes (turning into human-readable form later). Want that script?

On Thu, May 29, 2014 at 4:02 PM, yzliao notifications@github.com wrote:

We setup the sampling frequency to be 100Hz but the log file shows the sampling frequency is 74Hz. Some data are missing in this case.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64.

[yzliao]

that may be helpful. but we need to send to server side as well. So probably we need to have the server side to convert hex into int/float....

On Thu, May 29, 2014 at 4:04 PM, Mark Mollineaux notifications@github.com wrote:

Reading the data as serial input, translating to ASCII, printing to screen-- that all takes time.

...I've faced a similar problem in the past, and instead saved all log files as pure bytes (turning into human-readable form later). Want that script?

On Thu, May 29, 2014 at 4:02 PM, yzliao notifications@github.com wrote:

We setup the sampling frequency to be 100Hz but the log file shows the sampling frequency is 74Hz. Some data are missing in this case.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64#issuecomment-44596874.

[yzliao]

Also, I would firstly try to see if the mote really prints the data in 100Hz.

On Thu, May 29, 2014 at 4:06 PM, Yizheng Liao liaoyizheng@gmail.com wrote:

that may be helpful. but we need to send to server side as well. So probably we need to have the server side to convert hex into int/float....

On Thu, May 29, 2014 at 4:04 PM, Mark Mollineaux <notifications@github.com

wrote:

Reading the data as serial input, translating to ASCII, printing to screen-- that all takes time.

...I've faced a similar problem in the past, and instead saved all log files as pure bytes (turning into human-readable form later). Want that script?

On Thu, May 29, 2014 at 4:02 PM, yzliao notifications@github.com wrote:

We setup the sampling frequency to be 100Hz but the log file shows the sampling frequency is 74Hz. Some data are missing in this case.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64#issuecomment-44596874.

[bufordsharkley]

Yes, that's an important test.

On Thu, May 29, 2014 at 4:07 PM, yzliao notifications@github.com wrote:

Also, I would firstly try to see if the mote really prints the data in 100Hz.

On Thu, May 29, 2014 at 4:06 PM, Yizheng Liao liaoyizheng@gmail.com wrote:

that may be helpful. but we need to send to server side as well. So probably we need to have the server side to convert hex into int/float....

On Thu, May 29, 2014 at 4:04 PM, Mark Mollineaux < notifications@github.com

wrote:

Reading the data as serial input, translating to ASCII, printing to screen-- that all takes time.

...I've faced a similar problem in the past, and instead saved all log files as pure bytes (turning into human-readable form later). Want that script?

On Thu, May 29, 2014 at 4:02 PM, yzliao notifications@github.com wrote:

We setup the sampling frequency to be 100Hz but the log file shows the sampling frequency is 74Hz. Some data are missing in this case.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64#issuecomment-44596874.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64#issuecomment-44597060.

[yzliao]

After some experiments, this bug is more likely due to the slow execution of printf

I try to use Sky mote to read the data from I2C sensor with 100Hz sampling frequency and write the data to serial port (USB). I tried a couple of tests and realized that the total output data have a sampling frequency of 78Hz. I used rtimer to profile my sampling and printing code and found the print function slows down the entire process. Here are some profiling outputs:

start 50628
15490,f074,20,3b8c,ffab,49,ffcf,fb70
end 51189
start 51293
15491,f0a8,fff4,3ba4,ffc6,24,ffd8,fb90
end 51880
start 51984
15492,f094,20,3b30,ffa7,5b,fff3,fb70
end 52544
start 52647
15493,f118,bc,3ce0,ffab,70,fffc,fb90
end 53207
start 53311
15494,f030,1b0,3b44,ffa9,1f7,1f,fb80
end 53871

The rtimer has 4098*8 ticks for one second. Here we can clearly see that the printing takes about 560 ticks (17ms). If the sampling frequency is 100Hz, the printing function should be completed within 10ms (327 ticks).

So we need to figure out how to print the data in a nicer way

[bufordsharkley]

Printing in ASCII obviously incurs a heavy overhead... printing in pure bytes would reduce the throughput considerably.

On Fri, Jun 13, 2014 at 1:09 AM, yzliao notifications@github.com wrote:

After some experiments, this bug is more likely due to the slow execution of printf

I try to use Sky mote to read the data from I2C sensor with 100Hz sampling frequency and write the data to serial port (USB). I tried a couple of tests and realized that the total output data have a sampling frequency of 78Hz. I used rtimer to profile my sampling and printing code and found the print function slows down the entire process. Here are some profiling outputs:

start 50628 15490,f074,20,3b8c,ffab,49,ffcf,fb70 end 51189 start 51293 15491,f0a8,fff4,3ba4,ffc6,24,ffd8,fb90 end 51880 start 51984 15492,f094,20,3b30,ffa7,5b,fff3,fb70 end 52544 start 52647 15493,f118,bc,3ce0,ffab,70,fffc,fb90 end 53207 start 53311 15494,f030,1b0,3b44,ffa9,1f7,1f,fb80 end 53871

The rtimer has 4098*8 ticks for one second. Here we can clearly see that the printing takes about 560 ticks (17ms). If the sampling frequency is 100Hz, the printing function should be completed within 10ms (327 ticks).

So we need to figure out how to print the data in a nicer way

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64#issuecomment-45985935.

[yzliao]

tried... no improvement

On Fri, Jun 13, 2014 at 1:17 AM, Mark Mollineaux notifications@github.com wrote:

Printing in ASCII obviously incurs a heavy overhead... printing in pure bytes would reduce the throughput considerably.

On Fri, Jun 13, 2014 at 1:09 AM, yzliao notifications@github.com wrote:

After some experiments, this bug is more likely due to the slow execution of printf

I try to use Sky mote to read the data from I2C sensor with 100Hz sampling frequency and write the data to serial port (USB). I tried a couple of tests and realized that the total output data have a sampling frequency of 78Hz. I used rtimer to profile my sampling and printing code and found the print function slows down the entire process. Here are some profiling outputs:

start 50628 15490,f074,20,3b8c,ffab,49,ffcf,fb70 end 51189 start 51293 15491,f0a8,fff4,3ba4,ffc6,24,ffd8,fb90 end 51880 start 51984 15492,f094,20,3b30,ffa7,5b,fff3,fb70 end 52544 start 52647 15493,f118,bc,3ce0,ffab,70,fffc,fb90 end 53207 start 53311 15494,f030,1b0,3b44,ffa9,1f7,1f,fb80 end 53871

The rtimer has 4098*8 ticks for one second. Here we can clearly see that the printing takes about 560 ticks (17ms). If the sampling frequency is 100Hz, the printing function should be completed within 10ms (327 ticks).

So we need to figure out how to print the data in a nicer way

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64#issuecomment-45985935.

— Reply to this email directly or view it on GitHub https://github.com/s3lsensor/snowflake/issues/64#issuecomment-45986548.

[yzliao]

A quick fix:

Call the UART write function directly. This approach displays the ascii character on the screen. The Python script on BS needs to retrieve the solution and convert from ascii character to decimal representation.

#include "dev/uart1.h"
.....
            uart1_writeb((unsigned char)SN_ID);
            uart1_writeb(0); // avoid SN_ID is combined with others
            uart1_writeb((unsigned char)samples.reg.x_accel_h);
            uart1_writeb((unsigned char)samples.reg.x_accel_l);
            uart1_writeb((unsigned char)samples.reg.y_accel_h);
            uart1_writeb((unsigned char)samples.reg.y_accel_l);
            uart1_writeb((unsigned char)samples.reg.z_accel_h);
            uart1_writeb((unsigned char)samples.reg.z_accel_l);
            uart1_writeb((unsigned char)samples.reg.x_gyro_h);
            uart1_writeb((unsigned char)samples.reg.x_gyro_l);
            uart1_writeb((unsigned char)samples.reg.y_gyro_h);
            uart1_writeb((unsigned char)samples.reg.y_gyro_l);
            uart1_writeb((unsigned char)samples.reg.z_gyro_h);
            uart1_writeb((unsigned char)samples.reg.z_gyro_l);
            uart1_writeb((unsigned char)samples.reg.t_h);
            uart1_writeb((unsigned char)samples.reg.t_l);
            uart1_writeb((unsigned char)'\n');

This has been verified for MPU 6050 up to 128Hz (sampling frequency). For higher sampling frequency, #67 needs to be fixed at first

[yzliao]

see merge #68 #69