Expose Camera and Microphone

[ErSoul]

Feature description

Termux has a very complete collection of packages that can help to create a streaming server from the phone (eg. rtmpdump, icecast, nginx, etc), so I think it would be very useful to get the real-time data from those devices, instead of saving them to a file.

So to clarify the idea, I'm thinking of accesing the micro and camera as I would in a GNU/Linux system (through /dev/videoX or /dev/snd), but in this case would use the termux-api lib. So let's say I want to record from termux (with ffmpeg) my camera, it would be something like:

ffmpeg -i video=${termux-api Camera} -i ${termux-api Microphone} -o $OUTPUT

So ffmpeg is so versatile that $OUTPUT can be anything: a file, a streaming server, an URL, etc.

Reference implementation Mobile WebCam - Though I couldn't manage to stream the camera through MJPEG (I believe because is outdated), it is supposed to do it. Termux, SoX, PulseAudio - Here is an example of how to record the microphone input with pulseaudio, instead of termux-microphone-record

[xalexalex]

Definitely not a bad idea. Have you checked the other things of the "Feature request" template?

**Reference implementation**
Have you checked if the feature is accessible through the Android API?
Do you know of other open-source apps that has a similar feature as the one you want? (Provide links)

[ghost]

@xalexalex I guess everything that is needed is to create possibility to send camera or microphone data to stdout.

Note that microphone output can be obtained through PulseAudio. Although this can be complicated to users which are not familiar with PA, real-time microphone output issue is technically already solved.

[ErSoul]

@xalexalex Updated.

[jokester]

If we are to expose a/v stream to software like ffmpeg, what are some appropriate protocols?

ffmpeg supported protocols: https://ffmpeg.org/ffmpeg-protocols.html#Description

Also this may require termux-api to work as a streaming server or alike. Does termux or termux-api already have similarr infrastructure for such use?

[ErSoul]

There's a way to stream the audio, but the problem still is the camera.

For the sound you could do:

ssh $USER@$HOST -p $PORT "pulseaudio -k; pulseaudio -L 'module-sles-source' -D ; rec -t wav -" | mpv -

[v6ak]

Well, ffmpeg seems to support android_camera input as long as it is compiled with NDK: https://ffmpeg.org/ffmpeg-devices.html#android_005fcamera

However, it does not seem like being the case:

~ $ ffmpeg -y -f android_camera -i 0:0 -r 30 output.mkv
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with Android (6454773 based on r365631c2) clang version 9.0.8 (https://android.googlesource.com/toolchain/llvm-project 98c855489587874b2a325e7a516b99d838599c6f) (based on LLVM 9.0.8svn)
  configuration: --arch=aarch64 --as=aarch64-linux-android-clang --cc=aarch64-linux-android-clang --cxx=aarch64-linux-android-clang++ --cross-prefix=aarch64-linux-android- --disable-indevs --disable-outdevs --enable-indev=lavfi --disable-static --disable-symver --enable-cross-compile --enable-gnutls --enable-gpl --enable-libass --enable-libdav1d --enable-libmp3lame --enable-libfreetype --enable-libvorbis --enable-libopus --enable-libx264 --enable-libx265 --enable-libxvid --enable-libvpx --enable-shared --enable-libsoxr --enable-libvidstab --enable-libwebp --prefix=/data/data/com.termux/files/usr --target-os=android --extra-libs=-landroid-glob --enable-neon
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Unknown input format: 'android_camera'

So, it should be theoretically just matter of compilation environment.

[SDRausty]

stream the audio

termux take photo

How do you record a video in Termux?

[SDRausty]

ffmpeg -i video=${termux-api Camera} -i ${termux-api Microphone} -o $OUTPUT

file size comparison should give motion detector

#!/usr/bin/env bash
set -eu
CAMID=0
THRESHOLDSET=2048 # 64 128 256 512 1024 2048 4096 8192
FRAMERATE=4
TOTFRAMECOUNT=512
_CAMS0_ () {
        FRAMECOUNT=0
        while [ "$FRAMECOUNT" -le "$TOTFRAMECOUNT" ]
        do
                FRAMENAME="$(printf '%s.%04d.jpg' "${PWD##*/}$CAMID" "$FRAMECOUNT")"
                sleep 0.0"$(shuf -i 101-420 -n 1)"
                termux-camera-photo -c "$CAMID" "$FRAMENAME"
                _ISZERO_
        done
}
_CHECKMOTIONDIFF_() {
        if [ "$FRAMECOUNT" -ne 0 ]
        then
                THRESHOLD="$((LASTZERO - ISZERO))"
                if [ "${THRESHOLD#-}" -le "$THRESHOLDSET" ]
                then
                        printf 'Deleting file %s; %s threshold; Redoing frame %s one more time; Continuing...%s\n' "'$FRAMENAME'" "${THRESHOLD#-}" "'$FRAMENAME'"
                        rm -f "$FRAMENAME"
                else
                        printf 'Saving frame %s; threshold %s; Continuing...%s\n' "'$FRAMENAME'" "${THRESHOLD#-}"
                        FRAMECOUNT="$((FRAMECOUNT + 1))"
                fi
        else
                FRAMECOUNT="$((FRAMECOUNT + 1))"
        fi
}
_ISZERO_ () {
        if [ -n "${ISZERO:-}" ]
        then
                LASTZERO="$ISZERO"
        fi
        ISZERO="$(find . -type f -name "$FRAMENAME" -printf "%s\n" )"
        if [ "$ISZERO" -eq 0 ]
        then
                if [ "$FRAMECOUNT" -eq 0 ]
                then
                        printf 'Could not begin shoot; Please check for issues and run %s again; Exiting...\n' "'${0##*/}'"
                        rm -f "$FRAMENAME"
                        exit 1
                else
                        printf 'Deleting zero file size file %s; Redoing frame %s due to zero file size result; Continuing...\n' "'$FRAMENAME'" "'$FRAMENAME'"
                        rm -f "$FRAMENAME"
                fi
        else
                _CHECKMOTIONDIFF_
        fi
}
termux-wake-lock
_CAMS0_
FRAMEPREFIX="${FRAMENAME%%.*}"
nice -n 20 ffmpeg -y -framerate "$FRAMERATE" -i "$FRAMEPREFIX".%04d.jpg "$FRAMEPREFIX".mp4
termux-open "$FRAMEPREFIX".mp4
# https://get.videolan.org/vlc-android/3.3.4/
# cams0n.bash EOF

Originally posted by @SDRausty in https://github.com/termux/termux-app/issues/981#issuecomment-953429237

Hopefully, this is a step closer in resolving this enhancement. Please let me know if it is.

[aleclearmind]

Enabling android_camera support in ffmpeg is easy and there was a PR, however when trying to use it the acquisition seems to be stuck (and others have experienced this too).