Capture Audio with Video Recording

[danrasmuson]

I'm interested in saving the audio played during a recording.

I've tried removing the -an argument when ffmpeg is invoked, but didn't have any success. https://github.com/elgalu/docker-selenium/blob/master/video-rec/bin/start-video-rec.sh#L46

Any suggestions on how to accomplish this?

[elgalu]

Good point @danielrasmuson , I haven't considered this use case.

I'm at SeleniumConf and after that on vacations so might not take a look at this within the next 2 weeks.

If you find how to add audio support and make it optional through config params, like an environment variable passed to docker run -e VIDEO_CAPTURE_ALSO_AUDIO=true I would be very glad to accept a pull request:)

[danrasmuson]

Okay did some more investigation this evening. When I run ffmpeg with a capture audio flag it returns a audio driver error.

[alsa @ 0x201d320] cannot open audio device hw:0 (No such file or directory)

When I list the audio drivers it reports no soundcards.

seluser@b6c4636e661e:~$ aplay -l
aplay: device_list:268: no soundcards found...

Any ideas on how to add a soundcard to this image?

[elgalu]

Thanks for investigating, can you check http://stackoverflow.com/questions/28985714/run-apps-using-audio-in-a-docker-container

[danrasmuson]

Hi @elgalu, I wanted to let you know I started a $50 bounty for this issue. https://www.bountysource.com/issues/43724935-capture-audio-with-video-recording

[diemol]

@danielrasmuson I am interested to see how the tests for audio look like :)

[danrasmuson]

Hi @diemol,

I'm interested in recording audio so I can record my google hangout meetings.

Here is how I have my project setup

package.json

{
  "name": "somename",
  "version": "1.0.0",
  "main": "index.js",
  "repository": {},
  "license": "MIT",
  "dependencies": {
    "selenium-webdriver": "^3.3.0"
  }
}

run.sh

# Startup Selenium server
docker run --rm -d --name=grid -p 4444:24444 -p 5920:25900 \
  --shm-size=1g -e VNC_PASSWORD=hola \
  -e VIDEO=true elgalu/selenium
docker exec grid wait_all_done 30s

# Run selenium to capture video
node record-youtube-example.js

# Save the captured video
docker exec grid stop-video
docker cp grid:/videos/. videos
docker stop grid

record-youtube-example.js

var webdriver = require('selenium-webdriver'),
    By = webdriver.By,
    until = webdriver.until;

var driver = new webdriver.Builder()
    .forBrowser('chrome')
    .usingServer('http://localhost:4444/wd/hub')
    .build();

driver.get('https://www.youtube.com/watch?v=n8lpLgWrHGQ');
driver.sleep(1000 * 5);
driver.quit();

Running

docker pull elgalu/selenium
npm install
bash run.sh

Wait for run.sh to finish and open the completed .mp4. Notice it did record the youtube video for 5 seconds but does not include any sound.

[elgalu]

I think @diemol was referring to how do you automate the audio testing, or you check it manually by actually listening to it?

[elgalu]

@danielrasmuson do you have Linux or Mac? Ubuntu?

[danrasmuson]

I have a Mac, but I will be running Linux in production. I can run on a virtual machine if solution is OS dependent.

[danrasmuson]

FYI I was able to get sound working on a fresh ubuntu image.

docker pull ubuntu
docker run -it --rm ubuntu

# Inside the image
apt-get update
apt-get install pulseaudio socat
apt-get install alsa-utils
apt-get install ffmpeg

# Start the pulseaudio server
pulseaudio -D --exit-idle-time=-1

# Load the virtual sink and set it as default
pacmd load-module module-virtual-sink sink_name=v1
pacmd set-default-sink v1

# set the monitor of v1 sink to be the default source
pacmd set-default-source v1.monitor

# Start the ffmpeg capture to an audio file
ffmpeg -f pulse -i default out.mp3

Then in a separate session

paplay /usr/share/sounds/alsa/Front_Center.wav

And out.mp3 successfully recorded the sound that was played.

[elgalu]

@danielrasmuson managed to get this working, for anyone that needs this please follow his code.

I'll close this here as, for now, we don't intend to support this feature.

[bichotll]

Hi, I know this is closed, but I tried to follow those steps with selenium-recorder and I didn't get any luck. *I'm barely new with docker.

Things that I have noticed:

• alsa does not create any configuration file.
• paplay just returns Failed to open audio file.
• I tried to find the documentation of the module module-virtual-sink, but that module does not exist in the documentation? https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/
• I checked any possible log with docker logs --follow. I could not find anything related to audio, alsa or pulse.

I was wondering if the reason is the environment I'm working on? I'm working on mac with docker (virtualbox).

Finally, @elgalu , I actually think it would be good to have this as a feature of this docker image 👍

Cheers

[elgalu]

Hi! thanks for your interest. This feature in particular is difficult to support, For instance, I wouldn't know how to add tests for it. A PR is welcome to add this, it should contain some test that works in TravisCI.

[bichotll]

I could try to create a PR if I find out how to make it work first 😅

Re tests, I understand that we could play a sound using selenium on Chrome and Firefox, record it with Ffmpeg and compare its fingerprint with the expected one? https://stackoverflow.com/questions/6979352/how-can-i-compare-2-audio-files-programmatically

[v0idp]

I know this is old and closed. I recently hit this issue myself and for everyone ever finding this place. I made a fork with video + audio recording support: https://github.com/v0idp/docker-selenium

If you record a test. It will also record the audio now.

[elgalu]

thanks @v0idp great work!

Could you send a PR ? just make it optional to activate, through an environment variable like the other existing options

[mohammadsaleh]

Hi @v0idp , "docker exec grid stop-video" got error when I set AUDIO=true

this is error message:

video-rec: stopped
Waiting for video to stop recording...
video-rec                        STOPPED   Apr 15 02:45 AM
Done waiting for video recording to stop.
Video recording stopped
--LOG 02:45:23:982030558 -- DEBUG: video-rec-stdout.log ----
--LOG 02:45:03:598400358 Waiting for file /home/seluser/videos/"test"* to be created...
Waiting for ffmpeg video recording to start...
.ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
[x11grab @ 0x1326660] Cannot open display default, error 5.
default: Input/output error
............................

Thanks to any help

[v0idp]

I'll take a look. Thanks

[mohammadsaleh]

@v0idp Thank you, I'm waiting for your answer ...

[olberger]

The latest merge of my proposed PR fixes the "Cannot open display default, error 5" reported by @mohammadsaleh in https://github.com/elgalu/docker-selenium/issues/147#issuecomment-613765353 But there's still an issue if the default (mp4) extension is selected, where conversion between the captured mkv and the expected mp4 fails.

Hopefully, a workaround is to get the mkv video and perform the conversion outside of the docker container. Avoiding the conversion can be done using the -e VIDEO_FILE_EXTENSION="mkv" arguments to the docker run.

Will probably file another issue for that specific problem.

Hth,

[v0idp]

@olberger if you set the vcodec to libx265 it will fix the conversation process inside the docker. I also recommend changing the conversation process in "fix_video.sh" or whatever the name was to use FFMPEG_CODEC_VA_ARGS. that should also convert the audio properly to mp4. lemme know if it worked.

[olberger]

@v0idp I'm afraid I don't see where exactly to change to use libx265.... and could you elaborate on how precisely that would help ? I'm almost clueless wrt video codecs. Thx in advance.

[v0idp]

@olberger I'm pretty new to ffmpeg myself and it's black magic to me. specially all the different codecs with all these different results. I was trying around different ones and libx265 seemed to work best for this case.

[olberger]

Ok. Isee, @v0idp. All I could see is a size factor (4x) gain, but appart from that, I'm not sure that makes a significant change. Thx anyway.

[v0idp]

@olberger for me it fixed the conversion from mkv to mp4. but forwhatever reason it doesnt work anymore. It seems like I also have issues recording audio at all. anyone can confirm audio is working fine for you ? if yes what video and audio codecs are you using ?

[nishu]

Can captured audio + video be transcoded to HLS?