PiSleepTalk

The idea behind this project was to provide a "should work out of the box" solution to make a Raspberry Pi record what you are talking while sleeping. See our issue tracker to see whats going on. You can find some screeshots at the end of this file.

Feel free to support me if you like this project.

Feature list

Automatically record audio while you sleep
Rating of audio and automated deletion of non-sleeptalk-recordings
Web interface to add subtitles to recordings and download the videos
Automatic video generation
Possibility to import audio recorded from third party devices
Noise filtering
Multiple recording triggers like a hardware switch, the time, last.fm or Philips Hue
Audio manipulation (cropping, volume)

Videos

Here is a list of videos people generated using PiSleepTalk. Feel free to create an issue if you want me to list your video, too.

Installation

To install PiSleepTalk and all its dependencies, just copy and run this shell command. Beware, the installation may (it depends on the version of your Raspberry Pi) up to one day! On the latest Raspberry Pi the installation will take about 2-4 hours.

curl -s -L https://goo.gl/AksVZR | bash

Or if you dont trust google:

curl -s -L https://raw.githubusercontent.com/blaues0cke/PiSleepTalk/master/install.sh | bash

Warning: The install script will overwrite the hostname on your machine and assumes an interface called wlan0 exists. It installs various packages (nginx, samba etc.) but is otherwise limited to /usr/sleeptalk

Microphone configuration

You may configure the gain of your microphone to match your requirements. To do so, just type

sudo alsamixer

and select your soundcard. The you can configure the whole stuff. Once you are done, just hit the ESC key and type

sudo alsactl store

to save the new configuration. Make sure you reboot your Raspberry Pi or at least restart the recording to apply your changes.

File system

/usr/sleeptalk/cache Some generated data is stored in here, clear it if you want, the content will be re-generated
/usr/sleeptalk/debug A folder that contains debug output, not required for production mode
/usr/sleeptalk/frontend Contains the whole node.js frontend
/usr/sleeptalk/noise-data Contains noise that is removed from new recordings
/usr/sleeptalk/scripts Contains all bash scripts
/usr/sleeptalk/records-amplitude Contains files that have a pending amplitude check
/usr/sleeptalk/records-crop Contains files that have to be cropped
/usr/sleeptalk/records-decrease-volume Contains files that are too loud
/usr/sleeptalk/records-import Contains files you want to import in the system
/usr/sleeptalk/records-import-instant Contains files you want to import in the system as full recordings (no postprocessing)
/usr/sleeptalk/records-increase-volume Contains files that are too quiet
/usr/sleeptalk/records-final Contains video scenes generated from your texts
/usr/sleeptalk/records-raw Contains the raw recordings
/usr/sleeptalk/records-rendered Contains fully rendered movies (multiple scenes)
/usr/sleeptalk/records-timestamp Contains files that have a pending timestamp addition
/usr/sleeptalk/records-to-render Contains files that have to be rendered to scenes
/usr/sleeptalk/test-data Just contains test data I need to implemenet the whole stuff

All folders that contain record data have to start with record- since the status page only watches folders starting with this prefix.

Workflow

Files are processed in the following folders:

/usr/sleeptalk/records-import put audio files to want to import in the system in this folde using samba. It will processed like it is a recording made from your Raspberry Pi.
/usr/sleeptalk/records-import-instant put audio files to want to import in the system in this folde using samba. It will processed like a recording generated by your Rasberry Pi.
/usr/sleeptalk/records-raw contains all raw audio chunks recorded by record-chunks.sh.
/usr/sleeptalk/records-timestamp contains all audio chunks added with a timestamp by add-timestamp-to-filename.sh.
/usr/sleeptalk/records-amplitude contains all audio chunks added with its amplitude data by add-amplitude-to-filename.sh.
/usr/sleeptalk/records-to-render. contains all data that is required to render a video.
/usr/sleeptalk/records-decrease-volume the volume of all files in this folder is decreased by 50% by process-volume.sh. All processed files are moved back to /usr/sleeptalk/records-to-render.
/usr/sleeptalk/records-increase-volume the volume of all files in this folder is increased by 50% by process-volume.sh. All processed files are moved back to /usr/sleeptalk/records-to-render.
/usr/sleeptalk/records-rendered contains finally rendered videos including their subtitles with reduced noise.
/usr/sleeptalk/records-final contains upload ready videos build by concating multiple rendered records.

Audio detection and rating logic

There are some rules that control the way a recording is generated, basically:

The whole recording is dropped when it contains more than 50 chunks
A valid recording is enclosed by two records that were too quiet to make sure we got everything you spoke on tape
A chunk is always limited to 5 seconds

.sleeptalk format

I needed to implement a simple format to transmit the text data from node.js to the bash part of PiSleepTalk. For this reason the following format exists. It just contains the frame count and the corresponding text, like this:

0000|First text at second 0
0015|Follows after 1 second
0060|Appears after 4 seconds from start

Hardware

I am using the following hardware, but I think the most Raspberry Pi models, WiFi sticks, usb-microphones should work. Feel free to click on this link before you buy the stuff, so our non-profit hackspace Toolbox Bodensee e.V. will get a commission.

Raspberry Pi 2
Power supply (2000mA)
WiFi stick
USB microphone
- Cheap, low quality: Kinobo - USB 2.0 Mikrofo
- Expensive, good quality: Audio Technica AT2020USB+e
MicroSD card (16GB)

Actulally, PiSleepTalk generates very nice recording if you use a microphone that is in the professional segment, I switched from a cheap microphone to a more professional one and dont regret it.

Hardware switch and led

PiSleeptalk also supports a hardware switch to enable and disable the recording feature. Make sure you enabled this feature by chaning the button_enabled setting in config.cfg. Wire the buttons cables to the following wiring (Between pin 6 and pin 11):

    ────────────────────────────┐  <= Top right corner of your Raspberry Pi
                                |
                                |
                |────||────|    |
                | 01 || 02 |    |
                |────||────|    |
                |────||────|    |
                | 03 || 04 |    |
                |────||────|    |
                |────||────|    |
                | 05 || 06 | <= Switch -
                |────||────|    |
                |────||────|    |
                | 07 || 08 |    |
                |────||────|    |
                |────||────|    |
                | 09 || 10 |    |
                |────||────|    |
                |────||────|    |
    Switch + => | 11 || 12 | <= Force switch +
                |────||────|    |
                |────||────|    |
   Rec led + => | 13 || 14 | <= Rec led -
                |────||────|    |
                |────||────|    |
    On led + => | 15 || 16 | <= Led switch +
                |────||────|    |
                |────||────|    |
                | 17 || 18 |    |
                |────||────|    |
                |────||────|    |
                | 19 || 20 | <= Force switch -
                |────||────|    |
                |────||────|    |
                | 21 || 22 |    |
                |────||────|    |
                |────||────|    |
                | 23 || 24 |    |
                |────||────|    |
                |────||────|    |
    On led - => | 25 || 26 |    |
                |────||────|    |
                |────||────|    |
                | 27 || 28 |    |
                |────||────|    |
                |────||────|    |
                | 29 || 30 | <= Led switch -
                |────||────|    |
                                |
                                |
                                |

So at the end, your switch should sit between pin 6 and pin 11. To enable the led, just place it between pin 13 and 14. Don't forget to enable the led_enabled settings.

Importing audio

You may want to import audio you recored with third party deviced like an iPhone. To do so, just put your files in the /usr/sleeptalk/records-import or /usr/sleeptalk/records-import-instant folder and make sure, you file name contains something like a number to make sure the first file (if you have multiple chunks) is processed first. The importer script (import-audio.sh) will also add a seed at the end of the file name to make definitely sure you won't overwrite any existing file.

The importer will try to convert any audio file format to wav. If it fails, your file will be just deleted. After the import, the file is treated like a file that was recorded by the Raspberry Pi itself. Read the "Workflow" section to get an idea what is done with your file.

Beware: The system will wait a configured amount of time (default: {{500 seconds}}) until the import folder is checked after it was changed. You can force the import using the "Force import" button in the frontend.

Videos

Videos are rendered with 15 FPS. So one second is equivalent to 15 frames.

Samba

The configuration for the samba server is stored here: /etc/samba/smb.conf. Type sudo nano /etc/samba/smb.conf to edit the file. By default all folders documented in the "File system" area are a separate share. You may not touch them, but they are there for debug reasons.

To restart the samba server, type: sudo /etc/init.d/samba restart

Thanks to

Jonas Otto (https://github.com/ottojo) for the hardware support
Michael Malura (https://github.com/maluramichael) for the node.js support

Experience

I used this for my own a few days now in kinda production environment. Things I learned:

Do not set the gain of the microphone to 100%, it will record disorders that will annoy you

License

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/.

References

Bootstrap

http://v4-alpha.getbootstrap.com/

node.js

Wavesurfer

http://wavesurfer-js.org/

Screenshots

On the main page, you can add texts to the recording.
Or crop unwanted parts of the recording.
You can use a lot of shortcuts to process your data as fast as possible.
The overview page gives you an overview about all audio files that require interaction by the user.
The video overview shows you all generated scenes and movies.
You can watch generated videos directly in your browser.
You can concat scenes to a full movie with intro and outro.
The status page, here you can get an overview what is going on behind the scenes.
The input screen, here you can add audio files you recorded with external devices.