How to record both microphone input and computer output?

Benjamin-Loison commented 8 months ago

Your idea

Previously used OBS with empty screen but does not look clean and potentially involves a heavier workload.

Alternatively to reduce workload can use phone but might not be synchronized and will only capture microphone input, if use a headset for the computer output.

If I remember correctly this is not very easy but doable.

https://support.audacityteam.org/basics/recording-desktop-audio

While usually easy I do not achieve to record my microphone this time.

https://support.audacityteam.org/basics/recording-your-voice-and-microphone

How to detect pure black video to convert them to audios? If I remember correctly I checked once but have not quickly found something interesting.

May be interesting to use OBS anyway to record screen suddenly, however some softwares such as RustDesk enable just recording it this also allows (even if OBS allows but need) to precise the window to record.

+18

Problem to be solved

Not read.

Prior art

No response

Additional context

No response

Benjamin-Loison commented 3 months ago

Filter by nemo Size, then consider the biggest.

ffmpeg -i a.avi -vf mpdecimate,setpts=N/FRAME_RATE/TB a_.avi

Source: the Video Stack Exchange answer 20959

mediainfo --fullscan a_.avi | grep "Frame count"

Source: the Super User answer 1237621

ffmpeg -i a.avi a.flac

Benjamin-Loison commented 2 months ago

Can first filter videos with a first frame not being pure black.

ffmpeg -i video.mkv -vframes 1 first_frame.png

Source: the Super User answer 1682482

Python script to determine if an image is pure black it seems:

```python import PIL with PIL.Image.open('first_frame.png') as image: print(not image.getbbox()) ```

Source: the Stack Overflow answer 14041871

python3 -c 'from PIL import Image; exit(int(not Image.open("first_frame.png").getbbox()))'; echo $?

Benjamin-Loison commented 2 months ago

Pay attention to Python code injection.

Bash script to list black videos:

```bash TEMPORARY_FIRST_FRAME_FILE_NAME=first_frame.png for file in *.mkv do echo -n $file ffmpeg -i "$file" -vframes 1 $TEMPORARY_FIRST_FRAME_FILE_NAME -y 2> /dev/null python3 -c "from PIL import Image; exit(int(not Image.open('$TEMPORARY_FIRST_FRAME_FILE_NAME').getbbox()))" || echo -n " black" echo done rm $TEMPORARY_FIRST_FRAME_FILE_NAME ```

Benjamin-Loison commented 2 months ago

I have 46 black videos.

Benjamin-Loison commented 2 months ago

Bash script to remove black videos:

```bash for video in `cat black_videos.txt` do duplicatedLessVideo=$video.tmp ffmpeg -i $video -vf mpdecimate,setpts=N/FRAME_RATE/TB $duplicatedLessVideo # If `2`, then convert to `.flac`. mediainfo --fullscan $duplicatedLessVideo | grep 'Frame count' rm $duplicatedLessVideo #ffmpeg -i $video ${video%.*}.flac done ```

Benjamin-Loison commented 2 months ago

Bash script to remove duplicated videos:

```bash while read video; do duplicatedLessVideo="${video%.*}_tmp.mkv" ffmpeg -i "$video" -vf mpdecimate,setpts=N/FRAME_RATE/TB "$duplicatedLessVideo" mediainfo --fullscan "$duplicatedLessVideo" | grep -E '^Frame count.*: 1$' && rm "$duplicatedLessVideo" break done < black_videos.txt ```

Source: the Stack Overflow answer 1521470

Benjamin-Loison commented 2 months ago

Output:

``` ffmpeg version 6.1.1-3ubuntu5 Copyright (c) 2000-2023 the FFmpeg developers built with gcc 13 (Ubuntu 13.2.0-23ubuntu3) configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared WARNING: library configuration mismatch avcodec configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared --enable-version3 --disable-doc --disable-programs --disable-static --enable-libaribb24 --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc --enable-libsmbclient libavutil 58. 29.100 / 58. 29.100 libavcodec 60. 31.102 / 60. 31.102 libavformat 60. 16.100 / 60. 16.100 libavdevice 60. 3.100 / 60. 3.100 libavfilter 9. 12.100 / 9. 12.100 libswscale 7. 5.100 / 7. 5.100 libswresample 4. 12.100 / 4. 12.100 libpostproc 57. 3.100 / 57. 3.100 Input #0, matroska,webm, from '2023-03-22 16-14-44.mkv': Metadata: ENCODER : Lavf58.76.100 Duration: 00:32:39.97, start: 0.000000, bitrate: 213 kb/s Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn (default) Metadata: DURATION : 00:32:39.966000000 Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp (default) Metadata: title : simple_aac_recording DURATION : 00:32:39.850000000 Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264)) Stream #0:1 -> #0:1 (aac (native) -> vorbis (libvorbis)) Press [q] to stop, [?] for help [libx264 @ 0x59a74ed8c2c0] using SAR=1/1 [libx264 @ 0x59a74ed8c2c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x59a74ed8c2c0] profile High, level 4.0, 4:2:0, 8-bit [libx264 @ 0x59a74ed8c2c0] 264 - core 164 r3108 31e19f9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=24 lookahead_threads=4 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, matroska, to '2023-03-22 16-14-44_tmp.mkv': Metadata: encoder : Lavf60.16.100 Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 1k tbn (default) Metadata: DURATION : 00:32:39.966000000 encoder : Lavc60.31.102 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A Stream #0:1: Audio: vorbis (oV[0][0] / 0x566F), 48000 Hz, stereo, fltp (default) Metadata: title : simple_aac_recording DURATION : 00:32:39.850000000 encoder : Lavc60.31.102 libvorbis Frame count : 1 Frame count : 1 ffmpeg version 6.1.1-3ubuntu5 Copyright (c) 2000-2023 the FFmpeg developers built with gcc 13 (Ubuntu 13.2.0-23ubuntu3) configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared WARNING: library configuration mismatch avcodec configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared --enable-version3 --disable-doc --disable-programs --disable-static --enable-libaribb24 --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc --enable-libsmbclient libavutil 58. 29.100 / 58. 29.100 libavcodec 60. 31.102 / 60. 31.102 libavformat 60. 16.100 / 60. 16.100 libavdevice 60. 3.100 / 60. 3.100 libavfilter 9. 12.100 / 9. 12.100 libswscale 7. 5.100 / 7. 5.100 libswresample 4. 12.100 / 4. 12.100 libpostproc 57. 3.100 / 57. 3.100 Input #0, matroska,webm, from '2023-03-22 16-14-44.mkv': Metadata: ENCODER : Lavf58.76.100 Duration: 00:32:39.97, start: 0.000000, bitrate: 213 kb/s Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn (default) Metadata: DURATION : 00:32:39.966000000 Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp (default) Metadata: title : simple_aac_recording DURATION : 00:32:39.850000000 Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> png (native)) Stream #0:1 -> #0:1 (aac (native) -> flac (native)) Press [q] to stop, [?] for help [flac @ 0x5c4fadcb8180] encoding as 24 bits-per-sample, more is considered experimental. Add -strict experimental if you want to encode more than 24 bits-per-sample [flac @ 0x5c4face9f100] Video stream #0 is not an attached picture. Ignoring Output #0, flac, to '2023-03-22 16-14-44.flac': Metadata: encoder : Lavf60.16.100 Stream #0:0: Video: png, rgb24(pc, gbr/bt709/bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default) Metadata: DURATION : 00:32:39.966000000 encoder : Lavc60.31.102 png Stream #0:1: Audio: flac, 48000 Hz, stereo, s32 (24 bit), 128 kb/s (default) Metadata: title : simple_aac_recording DURATION : 00:32:39.850000000 encoder : Lavc60.31.102 flac ^C[out#0/flac @ 0x5c4face9f780] video:128366kB audio:124615kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown frame=21099 fps=147 q=-0.0 Lsize= 124623kB time=00:11:43.78 bitrate=1450.6kbits/s speed=4.89x Exiting normally, received signal 2. ```

So before flac conversion:

ls -lh '2023-03-22 16-14-44.mkv'

-rw-rw-r-- 1 benjamin benjamin 50M Mar 22  2023 '2023-03-22 16-14-44.mkv'

ls -lh '2023-03-22 16-14-44_tmp.mkv'

-rw-rw-r-- 1 benjamin benjamin 22M Aug 28 21:16 '2023-03-22 16-14-44_tmp.mkv'

ls -lh '2023-03-22 16-14-44.flac'

-rw-rw-r-- 1 benjamin benjamin 119M Aug 28 21:19 '2023-03-22 16-14-44.flac'

So does not help here...

Maybe could sort by biggest file size per video duration and stop until it is no more worth it.

Benjamin-Loison commented 2 months ago

List video size per second:

```bash while read video; do VIDEO_LENGTH=`ffprobe -i "$video" -show_entries format=duration -v quiet -of csv='p=0'` VIDEO_SIZE=`ls -l "$video" | cut -d ' ' -f 5` echo `bc -l <<< $VIDEO_SIZE/$VIDEO_LENGTH` done < black_videos.txt ```

Source: the Stack Overflow answer 12723330

Benjamin-Loison commented 2 months ago

Maybe related to Benjamin_Loison/ffmpeg/issues/3.

Benjamin-Loison commented 1 month ago

To reduce OBS workload maybe can set Settings > Video > General:

Base (Canvas) Resolution to 1280x720 being lowest proposed value but maybe can try a lower resolution anyway
Same comment for Output (Scaled) Resolution
Integer FPS Value to 1 (Settings > Output > Recording default quality is the lowest one).

Benjamin-Loison commented 1 week ago

Benjamin-Loison / audacity