Handling capture timestamps in HDF5 conversion script

[sbrodeur]

It seems that timestamps are not guaranteed to be sorted in time. This should be fixed. Also, the capture timestamps from the topic /irobot_create/cmd_raw seems to be uninitialized.

[sbrodeur]

Sorting issue was solved with commit d46a598.

Now it seems only that the audio clock is incoherent. The audio length in seconds is different whether it is calculated from the timestamps of the audio chunks or from the cumulative length of the audio samples at the sampling frequency.

[sbrodeur]

Here are possible solutions:

• Increase the publisher queue size to 10 or more for audio. This is because messages could be sent in burst because of internal audio buffering, if the CPU is really busy. See https://github.com/sbrodeur/ros-icreate-bbb/blob/master/src/audio/scripts/capture.py#L76
• Try capturing audio with pyaudio in callback mode, instead of blocking mode.
• Try capturing audio with alsaaudio, which seems more efficient.

In all cases, it should be tested while recording speech or music from the robot, so we can listen (and visualize with Audacity) if the audio really sounds choppy.

[sbrodeur]

Note that I believe that if we would truly miss some audio samples during recording, a buffer overrun error would occur and that would be seen in the log outputs. I experienced this problem at the beginning when using a too short buffer size (e.g. 128 instead of 2048) during recording.

[sgcarrier]

Here is the histogram for S2 of room C1-4016 (using msg header timestamps)

This shows that some messages are late but they are (almost) always sent in waves with a fixed period. Perhaps increasing the queue size as suggested above, will help the CPU and avoid to have to sample in <12.5 to meet the "dead line" and thus eliminate the second peak.

[sbrodeur]

See commit 4942bf0 to output histogram figures as below using script: https://github.com/sbrodeur/ros-icreate-bbb/blob/master/src/action/scripts/record/data/stats_rosbag.py

Use flag "--save-histograms" when specifying an output stats file.

Note that the red bars correspond to the drop threshold, and blue bar is mean delay.

[sgcarrier]

Reducing the frame buffer size from 2048 to 512 reduces the number of 'dropped' messages. Also, increasing the queue size of the publisher from 1 to 20 helps as well, since now there are more packets in quick succession. Changes in commit : b8b671fcef8d232f8d66ea27d2cae691b6eaa464

[sgcarrier]

For Left audio in a small test run, dropped packet were at 1.72 %. Before, we were seeing in the ~5%.

Concerning, the right mic, it always has around half the number of dropped messages compared to the right. I'm not really sure what causes this. My best explanation is the USB timing favors the right one.

[sbrodeur]

See commit c563dfd for added support for recording with alsaaudio using ALSA, which is lower-level in the Linux audio system: http://tuxradar.com/content/how-it-works-linux-audio-explained

We should check if using ALSA solves the latency issue. I've also included a script which uses pyaudio in callback mode, but I was not able to make it work on my laptop.

To install alsaaudio on Debian:

sudo apt-get install libasound2-dev
sudo pip install pyalsaaudio

[sgcarrier]

Using the capture_alsa.py node proved to be more efficient. As seen above, all "late" packets have dissapeared (0% dropped packets for both microphones). The system as a whole seems to perform better.

Also, those pesky alsa warnings in the service logs that didn't really cause issues have disappeared as well.

Good Job!

[sbrodeur]

This is good news! I think we could get back to slightly larger buffers, like 1024. This would make audio stream at a message rate of about 15 Hz (rather than 30 Hz).

[sbrodeur]

See commit 92ffa7d for a C++ version of the audio capture node (based on ALSA), which should be even more efficient.