Open j-mir-prazak opened 2 years ago
H-M-H
commented
2 years ago Thanks for the detailed report! As you can compile on your own please check if adding AMD RENOIR to this blacklist
https://github.com/H-M-H/Weylus/blob/67500d87769c9b9c1acc87ebd1c15488d0c7a3d2/lib/encode_video.c#L219-L220 prevents crashes.
I suspect this is the same problem as discussed here: https://github.com/H-M-H/Weylus/issues/62#issuecomment-780251858.
j-mir-prazak
commented
2 years ago @H-M-H i already tried to forced it manually and also tried to force AV_PIX_FMT_NV12 and commenting out the "quality" option (as amd does not seem to care about it much). with that the vaapi does not crash, but then the browser spills out: "ERROR: Failed to decode video, try reducing resolution or disabling hardware acceleration and reload the page. Error src: blob:http://127.0.0.1:1701/f73373ea-90e1-4579-86ad-fd1d8cecf8d1"
H-M-H
commented
2 years ago I am afraid but in this case you may be out of luck. Please check if you can create a video with this ffmpeg command: ffmpeg -hwaccel vaapi -f lavfi -i testsrc2=size=1920x1080 -vf 'format=nv12,hwupload' -vcodec h264_vaapi -qp 23 -quality 7 -t 10 -bf 0 -movflags frag_keyframe+empty_moov+default_base_moof test.mp4. If your browser can't play this either this is definitely an issue with ffmpeg/AMD drivers.
j-mir-prazak
commented
2 years ago as is the i cannot even run the file in mpv. however, if i do not use empty_moov, mpv and browser works just fine. so it seems amdgpu vaapi bug + empty_moov brakes the file.
there is line in the bash log: "Empty MOOV enabled; disabling automatic bitstream filtering" - so you have alternative in the code, if you cannot use empty moov?
H-M-H
commented
2 years ago The point about empty_moov is to enable instant streaming without buffering. You can try to build Weylus without it by removing the option from this line:
https://github.com/H-M-H/Weylus/blob/67500d87769c9b9c1acc87ebd1c15488d0c7a3d2/lib/encode_video.c#L411
But yeah, the conclusion is your driver in combination with ffmpeg is faulty and it's not a bug inside Weylus. If you get things to work with above workaround I will consider adding it.
j-mir-prazak
commented
2 years ago @H-M-H thanks for the tips!
i did removed the empty_moov and added faststart for good measure, and it seems to work fine: however it is kinda unstable, if you change resolution in the browser, it tends to crash. compare to x264 it seems to not be willing to push as many frames. i will mess with vaapi arguments tomorrow to see if i can make it behave, but i won't probably make much difference and it won't be much use for general solution.
x264 also seems to be less laggy: even with the same no empty_moov it is noticably more in sync. vaapi seems to be more then round 10 frames behind. localhost testing seems to be quite taxing - chromium + weylus + davinci resolve (also tapping into gpu). however with phone as the display, it can go round 60fps quite okey (also my wifi is probably not the best, might try to use hotspot on the phone for that). but still vaapi do not seem to be as stable.
can i just put here two questions unrelated? there seems to be bug (i am running gnome-shell), where after starting client my cursor disappers - is that known?
and are you planning to add keyboard support? it could be nice to be able to use the client keyboard as well (i was planning on getting chromebook tablet with keyboard, so i could use it with my main machine as secondary display, or as remote access terminal - using the client inputs)
thank you though for the tips and great tool! j
H-M-H
commented
2 years ago Alright, hopefully you can make it behave tomorrow!
Regarding your questions:
there seems to be bug (i am running gnome-shell), where after starting client my cursor disappers - is that known?
This is might not be a bug but intended behavior. Weylus simulates stylus and touch input and a cursor doesn't make much sense in this case. Are you running X11 or Wayland?
keyboard support
Actually there already is! All you need is a physical keyboard (onscreen keyboards typically do not work as they do not generate key codes that I could simulate). I have used a bluetooth keyboard on Android and iPadOS quite a few times.
thank you though for the tips and great tool!
Thanks!
j-mir-prazak
commented
2 years ago @H-M-H okey, so i was messing around with the settings and i was able to get fairly stable vaapi encoding when i changed the time_base to {1, 60} : vaapi might have been chocking on the 1000 framerate. or is it something virtual and you change it on a fly via the browser slider?
also to the silder: what values does it operate in? it is slider from 100 to 999.999 - i wanted to try to specifiy the appropriate value for set framerate, but i cannot figure out what is the constatnt you are using there for the conversion.
This is might not be a bug but intended behavior. Weylus simulates stylus and touch input and a cursor doesn't make much sense in this case. Are you running X11 or Wayland?
yes, it might be result of me running the localhost instance - just to clarify it the cursor disapers on the host, not client, until i do some crazy mouse movements and workspace changes. i will try to find when it happens.
Actually there already is! All you need is a physical keyboard (onscreen keyboards typically do not work as they do not generate key codes that I could simulate). I have used a bluetooth keyboard on Android and iPadOS quite a few times.
i have second machine that i tried running weylus on and control with my laptop. the mouse is fine, however keyboard is no dice. running Xorg on both.
also to the second machine, i am testing nvenc there: it works, i was able to make it less laggy using the same faststart and time_base settings, however it seems to be not so springy as the x264. it have not problem running x264 there, as it is 8c/16t machine, however i like to try hw encoding whenever i can.
would you consider adding text field for fixed framerate to the gui and as command line argument? and maybe some other ffmpeg injection tools? i know you get you are mostly aiming for tablet as tablet/pen control, but the secondary monitor is nice thing to have and might be nice to have some tweaking space without having to edit and recompile.
also, i was thinking about trying to make some nodejs interface for the ws communication, so i can just something like mpv for video only - as browser can get bit prickly with playback. do you have any documentation for the ws communication, or should i just dig into the code?
thanks!
H-M-H
commented
2 years ago i was able to get fairly stable vaapi encoding when i changed the time_base to {1, 60} : vaapi might have been chocking on the 1000 framerate. or is it something virtual and you change it on a fly via the browser slider?
time_base does not describe the framerate. Actually Weylus generates a variable framerate video. The actual framerate is unknown and depends on user input and how fast your computer is at encoding. That is also why the video is not encoded at a fixed frame rate. time_base merely describes at what precision time is being kept track of in the encoded video. That is the time resolution with time_base: {1, 60} is less precise than {1, 1000}. A time_base like {1, 15} can even lead to errors if you encode faster than at 15 fps because two consecutive frames will then be encoded as if they happened simultaneously.
also to the silder: what values does it operate in?
The slider Min Frame Interval can be used to force a delay between two consecutive frames of up to 1 second. Weylus encodes the video with variable framerate and the user can throttle it like that. The values on the slider are not distributed linearly, see: https://github.com/H-M-H/Weylus/blob/67500d87769c9b9c1acc87ebd1c15488d0c7a3d2/ts/lib.ts#L56-L62
This enables being more precise for lower framerates.
keyboard is no dice
That should not be the case please check if keyboard events are actually sent from the browser by looking for them in the debug log you get by running Weylus like this: WEYLUS_LOG_LEVEL="DEBUG" weylus.
i was able to make it less laggy using the same faststart and time_base settings
Have you tried unchecking the Lower Latency checkbox? On Firefox this makes things noticeably smoother.
and maybe some other ffmpeg injection tools
I will have to think about this as this is very much non-trivial. I'd prefer good defaults here.
do you have any documentation for the ws communication, or should i just dig into the code?
The protocol is pretty simple and I have an example python script that feeds into mpv: https://github.com/H-M-H/Weylus/issues/87#issuecomment-850481361
Alright, I hope this helps. Happy hacking!
hammerfunctor
commented
2 years ago As for the original bug, some information can be found at https://gitlab.freedesktop.org/mesa/mesa/-/issues/3524.
hammerfunctor
commented
2 years ago As for the original bug, some information can be found at https://gitlab.freedesktop.org/mesa/mesa/-/issues/3524.
Update: mesa-22.1.0-1 fixes this problem. Now it no longer exits with Segmentation fault
choiman1559
commented
2 years ago Well, I had the same problem, but still having the same problem despite compiling after disabling "empty_moov".
As you can see, updating mesa to 22.1.0 or higher does nothing for me.
Even though my cpu is not that bad stuff (Ryzen 5800u), but the video stutters quite a bit when encoding with cpu only, so i have to use gpu acceleration but these issues bother me.
Is there no way?
Weylus works with CPU, but VAAPI crashes. ffmpeg with VAAPI works just fine (only mkv output fails).
Tried release version and build my own: the same error.
va.log.014607.thd-0x00027707