Open Sailiy opened 2 years ago
mmmm are you using my openCV code ?
what ever scrcpy send video frame, not JPEG buffer.
I also have a video support, but I it currently segfault on large video resolution. it mainly works in 480p....
tell me if you want to debug the video demux.
the demux code project is here: https://github.com/UrielCh/beamcoder
Yes, I used your @u4/ OpencV4Nodejs library and saved a lot of time on Windows, great; Thank you for your service.
My project needs to be integrated in Electron. Since I am not good at C/C++ programming, it is too difficult for me to use C/C++ in Electron, so I don't want to use C/C++ to complete decoding.
I have seen this project, which may also be enlightening to you, it uses both WebCodec and Tinyh264 decoding methods, which may be integrated into this project;
Now I'm working on remote-droid mostly precisely on its Kubernetes deployment.
Once remote-droid is completed I will improve scrcpy integration.
But the scrcpy frames must be demux before any openCV usage. I tried to use it directly with OpenCV but it only supports hardware video input or video file input. so there is no practical way for any real-time processing.
I've seen the scrcpy source code, and it's actually not that complicated to transfer a JPEG stream like minicap, which would eliminate a lot of decoding. I looked at your Github Pages, you used to do Java, I thought you could try it
我大概写了一下代码,修改的文件时scrcpy项目server目录内的文件ScreenEncoder.java,由于android环境比较复杂,还没编译测试,我想可以解决。 有兴趣可以尝试一下
Image mImage=codec.getOutputImage(outputBufferId);
ByteBuffer buffer = mImage.getPlanes()[0].getBuffer();
byte[] bytes = new byte[buffer.capacity()];
buffer.get(bytes);
Bitmap bitmapImage = BitmapFactory.decodeByteArray(bytes, 0, bytes.length, null);
ByteArrayOutputStream memStream = new ByteArrayOutputStream();
bitmapImage.compress(Bitmap.CompressFormat.JPEG,70,memStream);
byte[] buf=memStream.toByteArray();
ByteBuffer jpegBuf=ByteBuffer.wrap(buf);
yep but for that I need to maintain a scrcpy fork. I have too many active projects. I can not hold that one.
I can integrate your fork if you want to deal with this subproject.
But patch it properly, it must be retro compatible with the original script. since scrcpy V 1.20, it contain a real argument parser, add a flag for replacing stream video data with jpeg data, and we are good to go.
All STF features are available in scrcpy with better implementation.
You can check my fork: https://github.com/UrielCh/scrcpy I do not like the memory management of this code, and I do not know if it can be built.
I do not have and java or android dev environment here.
And I did not write any line of java for years.
Hey, do you have a running example on how to use scrcpy? I need to stream the incoming video to multiple clients and save it on my server. For testing purposes I've tried
const adbClient = createClient();
const devices = await adbClient.listDevices();
const deviceClient = devices[0].getClient();
const scrcpy = deviceClient.scrcpy();
const writeStream = fs.createWriteStream('./output.mp4');
await scrcpy.start()
setTimeout(async () => {
scrcpy.stop();
}, 7000);
scrcpy.on('frame', (frame) => {
writeStream.write(frame.data);
})
but the output.mp4
always shows errors/ can't be opened.
Thanks!
check tasks/realTest.ts
file, it contains test for most functions.
I've been testing some more and had a look at tasks/realTest.ts/extractFramesStream
. I do get the 100 files, however I still can't view/open them, no matter which encoder for scrcpy I'm using. I'm currently using @u4/adbkit: 4.1.11
with two Android Devices running Android-Version: 10
and one other device running Android Version: 11
. I also tried changing the scrcpy-jar version. Results are always the same.
Additionally I tried using the WebCodecs API to check if I can render the video on my client but with the same results. There's no errors thrown, but also no images/video.
Is there a good way to debug this?
I only succeeded in extracting images from this stream once using https://github.com/UrielCh/beamcoder I think it's possible to render the stream directly in a browser, but I did not have success yet.
since I was able to identify keyframes and update frames, but I did not retire.
Do you know if I can build scrcpy on a windows host?
My Mac SSD is mostly full.
Did you use beamcoder in a realtime-application or with a file? I'd like to start the stream from my backend, record the incoming scrcpy video there and then send it to the clients. I also had a look at the repos mentioned by @Sailiy, which use the WebCodecsAPI to display the videostream in a canvas. They use the keyframe to define the canvas size (which works in my setup) and then use the remaining data for displaying (which does not work in my setup). I don't get any errors though and it's just an unfilled canvas.
I've downloaded the scrcpy zip with the download-link for windows, unzipped the folder and then defined the path with
const adbClient = createClient({ bin: `${config.SCRCPY_PATH}` });
which works for me.
If you want to integrate video from adbkit, you may be interested in remote-droid
Once fully deployed I get un GUI like:
Each server accepts multiple android devices, and all devices are visible at the same time as thumbnails
The front end is built in deno fresh. and I need some help to complete it.
Most of the configuration is done by Kubernetes.
This sounds similar to what I'm trying to achieve. Just tested your docker image but unfortunately I can't see my devices. Maybe a problem because I'm using windows?
I only need scrcpy with one device and the video-stream should be recorded, before raw-data or a compiled video is sent to (multiple) clients. I need to continuously write the incoming data to a file because there's a high chance connection will be lost and I can't afford to lose all the data. Preferably this should be it's own service, so if there's a crash other services in my setup won't be affected.
My Docker remote-droid is not compatible with windows, It is not possible to share USB to the Docker container.
I have just refactored my scrcpy integration and prepared an @u4/adbkit v5.0.0, this version is an ESM + CJS package.
Is this GUI already available?
no open-sourced version yet. if you are interested in it follow the project remote-droid
Hey, do you have a running example on how to use scrcpy? I need to stream the incoming video to multiple clients and save it on my server. For testing purposes I've tried
const adbClient = createClient(); const devices = await adbClient.listDevices(); const deviceClient = devices[0].getClient(); const scrcpy = deviceClient.scrcpy(); const writeStream = fs.createWriteStream('./output.mp4'); await scrcpy.start() setTimeout(async () => { scrcpy.stop(); }, 7000); scrcpy.on('frame', (frame) => { writeStream.write(frame.data); })
but the
output.mp4
always shows errors/ can't be opened.Thanks!
我也想像你这样设计 后端写入本地文件 但是如何控制文件大小呢 我只想要某个可以控制范围内的大小的文件 让他的流数据更新
hello,我添加了以下代码, minicap可以正常工作
scrcpy卡在 start()地方