Open tejado opened 1 year ago
Commit https://github.com/tejado/android-usb-gadget/commit/5aaf88d114a0267befe5361fb0a348571e5ac7e9 fixes a few things in the UVC function. Now android-usb-gadget is able to create a UVC gadget which works with "uvc-gadget" (https://github.com/wlhe/uvc-gadget). Tested with Pixel 5 and self-build Kernel.
sudo apt install android-tools-adb android-tools-fastboot git
curl -O http://dl.google.com/android/repository/android-ndk-r12b-linux-x86_64.zip
unzip android-ndk-r12b-linux-x86_64.zip
./android-ndk-r12b/build/tools/make_standalone_toolchain.py --arch arm64 --install-dir ~/arm
git clone https://github.com/wlhe/uvc-gadget
cd uvc-gadget
~/arm/bin/clang -pie uvc-gadget.c
Transfer the "a.out" bin to your target device.
I was not able to get uvc-gadget to work with any of the camera video device files. It seems that v4l2 on Android has a different specification than on generic Linux. But to test just the UVC functionaility, you can use the dummy image mode:
chmod +x a.out
./a.out -i /storage/emulated/0/test.mjpeg -u /dev/video5 -f 1 -r 1
With this, the Android device gets detected as a UVC Webcam under Windows 10 and plays the mjpeg video in a loop. Example mjpeg file: https://filesamples.com/samples/video/mjpeg/sample_1280x720.mjpeg
I attached a signed interims version of android-usb-gadget with the fixed UVC function. Don't add UVC as a function to the existing default gadget - create a new UVC gadget. The "adb" function in the default gadget does not work stable together with other functions and you also can't disable it, as Android will immediately enable it again.
After successfully enabling the UVC function using USB Gadget App, I am not able to write to /dev/video3
with the main error being:
java.io.IOException: write failed: EINVAL (Invalid argument)
at libcore.io.IoBridge.write(IoBridge.java:654)
at java.io.FileOutputStream.write(FileOutputStream.java:401)
at java.io.FileOutputStream.write(FileOutputStream.java:379)
Steps I have taken:
1. Enable wifi debugging so we can use the USB for UVC only
In terminal On the Rooted Pixel 4A with custom ROM/Kernel (thanks @tejado 🙏 )
su
setprop service.adb.tcp.port 5555
stop adbd
start adbd
On my computer computer:
adb connect 192.168.1.XXX
adb -s 192.168.1.XXX:5555 shell
2. Configure UVC function in Android USB Gadget
Select +, UVC, enable it Connect the Android device to PC
3. Set the permissions for newly created /dev/video3
stream so we can write to it from an Android app without the need for root access:
chmod 666 /dev/video3
chcon u:object_r:zero_device:s0 /dev/video3
Thanks again @tejado for the chcon
command, saved me from having to deal with root access on the Android app.
4. On another Android app, grab the camera frames as following
I get 24 frames per second using imageAnalyzer from CameraX:
imageAnalyzer = ImageAnalysis.Builder()
.setTargetResolution(Size(640, 480))
.setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_YUV_420_888)
convert the 3 YUV planes to ByteArray
imageAnalyzer.setAnalyzer(cameraExecutor) { image ->
// getting image in ImageProxy format
val imageByteArray = image.toByteArray()
...
}
Then based on UVC Specifications I build the header of the frame:
// UVC Video Payload Header has a payload size of 26 bytes, and the UVC MJPEG Payload Header has a payload size of 12 bytes.
val header = ByteBuffer.allocate(26)
val frameSize = image.width * image.height * ImageFormat.getBitsPerPixel(image.format) / 8
// End of Header: This field indicates the end of the UVC Video Payload Header and the start of the video data for the frame. It is a single byte that is set to the value 0x01.
val EOH = 0x01
// Error Code: This field indicates the error code for the frame. It is a single byte that is set to the value 0x00.
val ERR = 0x00
// Still Image: This field indicates whether the video frame is a still image or a video frame.
// It is a single byte that is set to the value 0x00 if the frame is a video frame, or to the value 0x01 if the frame is a still image.
val STI = 0x00
// Reserved: This field is reserved for future use. It is a single byte that is set to the value 0x00.
val REST = 0x00
// Source ID: This field specifies the source of the video frame, such as a camera or a video file. It is a single byte that is assigned a unique value for each source.
val SRC = 0x00
// // Presentation Time Stamp: This field specifies the time at which the video frame was captured or generated.
// It is a 32-bit value that represents the number of 100-nanosecond intervals since a reference time, such as the start of the video stream or the beginning of a recording.
val PTS = (System.currentTimeMillis() - referenceTime) * 10000
// // End of Frame: This field indicates the end of the video data for the frame and the start of the next frame. It is a single byte that is set to the value 0x01.
val endOfFrame = 0x01
val FID = (frameId).toByte()
Add all of the above to the header
header.putInt(frameSize)
header.putShort(image.width.toShort())
header.putShort(image.height.toShort())
header.put(image.format.toByte())
header.put(((EOH shl 7) or (ERR shl 6) or (STI shl 5) or (REST shl 4) or SRC).toByte())
header.putLong(PTS)
header.put(endOfFrame.toByte())
header.put(FID)
Open the FileOutputStream and try to write the header and the image:
val uvcFileOutputStream = FileOutputStream("/dev/video3", true)
uvcFileOutputStream.write(header.toByteArray() + data)
uvcFileOutputStream.close()
Getting java.io.IOException: write failed: EINVAL (Invalid argument)
I tried to break down the imageByteArray into smaller chunks and write to the FileOutputStream by leveraging the endOfFrame
or sameFrame
bit in the header. Tried 512bytes, 1024bytes, 256kb, 512kb (image size output is smaller than 1024kb)
To test the UVC stream file and ensure it is writeable, I used the FFmpeg Android Library
Based on their docs, the ffmpeg -f video4linux2 -list_formats all -i /dev/video3
command should list the possible formats I can write to the stream.
Running their sample app with the following command:
String[] command = {"-f", "video4linux2", "-list_formats", "all", "-i", "/dev/video3"};
FFmpeg.getInstance(this).execute(command)
FFmpeg suggests /dev/video3
is not a video capture device:
Compared to /dev/video2
which it recognizes as a video device but fails to open it, probably due to it being used by the system.
Next steps/ideas:
/dev/video3
is writeable with the expected video stream format. Maybe we need to configure the filedead chat xd
@luchfilip As I know you made a lot of progress, could you provide an update on your learnings and how to implement this? Let me know if I can support you in this.
Related:
Use a device as a webcam
This issue documents the work to get UVC up & running with android-usb-gadget.