Open TheHacker66 opened 7 years ago
cgutman
commented
2 years ago If the hardware offer still stands, I'd be happy to take a look.
Is there an FFmpeg build that supports hardware decoding on the Vero's current current media stack? FFmpeg would probably be easiest to integrate because Moonlight Qt supports it already.
Do you know what APIs the Kodi folks are using on this hardware? Is it V4L2+DRM or the old amcodec stuff?
samnazarko
commented
2 years ago Hi - we still use AMCodec on this device. It’s possible to support AMCodec via ffmpeg with some hacks but this hasn’t been tested.
AML’s current focus is a V4L2 GStreamer stack.
Happy to send some hardware.
cgutman
commented
2 years ago Hi - we still use AMCodec on this device. It’s possible to support AMCodec via ffmpeg with some hacks but this hasn’t been tested.
So it sounds like our existing amcodec support should be expected to work then. Hopefully it is something relatively simple.
Happy to send some hardware.
OK, please reach out to cameron@moonlight-stream.org and we can get the shipping details sorted out.
TheHacker66
commented
2 years ago @cgutman I tried syncing and compiling my fork yesterday to see if updating to Debian 11 (which will be released soon on OSMC) changed anything but the same issue occurs.
Here's the fork https://github.com/TheHacker66/moonlight-embedded
Let me know if there's something I can do while you wait to receive the hardware.
Thanks for not abandoning OSMC support!
samnazarko
commented
2 years ago Thanks, I’ll get in touch with you shortly.
samnazarko
commented
2 years ago Hi -- I was travelling so it took a bit longer than expected to email you, but I've done so now and put a unit aside for you. Please let me know if you've received my email
cgutman
commented
2 years ago Yep, I got it and just replied. It got caught in my spam filter, so I didn't see it earlier.
samnazarko
commented
2 years ago This has shipped and I sent you tracking details. There’s currently a postal strike for two days in the UK, so I wouldn’t expect the package to move much until next week
cgutman
commented
1 year ago I worked on this for a few hours today trying various things and couldn't get anything working.
To see if it was Moonlight-specific, I tried the samples from the OSMC Amcodec repo and those exhibited the same symptoms (reporting successful decoding but not displaying anything).
The log messages from the kernel seem to be as expected:
[ 78.416045] vdec_init, dev_name:ammvdec_h265, vdec_type=VDEC_TYPE_STREAM_PARSER id = 0
[ 78.416050] vdec_init set vfm decoder 00000000428cad09
[ 78.416862] HEVC stbuf alloced at 00000000f2624b3e, secure = 0, size = 10485760
[ 78.416977] pts_start, type=2
[ 78.417534] video first pts = 0
[ 78.417542] vdec_request_irq 00000000a9a08f5a, parser
[ 78.418868] decoder_bmmu_box_alloc_box, tvp_flags = 0
[ 78.419082] [0]dynamic_buf_num_margin=7
[ 78.419088] [0]double_write_mode=0
[ 78.419097] [0]h265:pts_unstable=0
[ 78.419102] [0]h265: ver (201602101,0) decinfo: 1280x720 rate=1600
[ 78.420499] [TEE], the fw (hevc_mmu_swap) will be loaded.
[ 78.420602] hevc mux clock is 666666666 Hz
[ 78.420608] hevc video changed to 3840 x 2160 60 fps clk->667MHZ
[ 78.420898] ppmgr local_init
[ 78.420911] di_receiver_event_fun: vframe provider reg ppmgr
[ 78.421990] set run_early_proc_fun_flag to 1
[ 78.422000] vpu_clkb = 0.
[ 78.422001] DI: reg f
[ 78.425832] ppmgr local_init
[ 78.426125] vdec_init, vf_provider_name = decoder, b 0
[ 78.426153] vdec->port_flag=0x102, port_flag=0x10b
[ 78.451041] osmc videoenhancement: session context configured successfully
[ 78.478751] [0]vh265 cached=16384 need_size=16384 speed= 60 ms
[ 78.479573] [0]hevc->performance_profile 0
[ 78.479587] [0]set pic_list_init_flag 1
[ 78.481705] [0]init_buf_spec2 1280 736
[ 78.481712] [0]set pic_list_init_flag to 2
[ 78.481739] [0]set pic_list_init_flag to 3
[ 78.481742] [vdec_kpi][vh265_isr_thread_fn] First I frame coming.
[ 78.484248] [0]VH265: output first frame
[ 78.484269] [vdec_kpi][post_video_frame] First I frame decoded.
[ 78.484337] DI bypass all 1280x720-0x501000.
[ 78.511486] cal_dur_from_pts inner driver dur1600
[ 78.533530] vframe_rate_uevent: sent uevent FRAME_RATE_HINT=1600The [ 78.484248] [0]VH265: output first frame line in particular seems to indicate that it did successfully output a frame to somewhere.
I also added a call to codec_get_vdec_state() before submitting each frame to confirm that the decoder is not reporting errors and it doesn't seem to be.
Received first video packet after 0 ms
Stream info: 0x0x60 | Errors: 0 | Status: 255
Stream info: 0x0x60 | Errors: 0 | Status: 255
Stream info: 1280x736x60 | Errors: 0 | Status: 255The first two have no dimensions because the decoder hasn't processed enough of the bitstream yet, but then it does initialize (further indicate that the codec is indeed parsing the actual bitstream correctly and it's an output-related issue).
@samnazarko Is there something special that needs to be done to get amcodec output working? Since even the amcodec samples fail to play, I think this is probably related to Vero-specific kernel or amcodec changes.
I also tried moonlight-qt to see if maybe it could work more easily, but it looks like the Vero uses a legacy fbdev driver which is not something moonlight-qt supports. Even if it did support fbdev, it would not be able to do the performant zero-copy rendering that we can do with EGL or DRM renderers, so the performance likely wouldn't be good.
samnazarko
commented
1 year ago Hi,
The amcodec in that repository is quite old and we've a much more improved one which is installed in OSMC automatically. You can get the appropriate headers with sudo apt-get install libamcodec-dev-osmc. That's just a newer library itself -- the examples are the same and unchanged.
I assume you stopped Kodi first: sudo systemctl stop mediacenter
samnazarko
commented
1 year ago According to @tanio99
It might be related to the decoder type being used when initializing libamcodec. For HEVC playback one needs to set
codecParam.decoder_type = DECODER_TYPE_FRAME_MODE;
Additionally, the VFM map doesn't seem to be set in the logs.
You should initialise the codec parameter structure fully, i.e:
params.video_pid = m_am_private->video_pid;
params.video_type = m_am_private->video_format;
params.stream_type = STREAM_TYPE_ES_VIDEO;
params.decoder_type = DECODER_TYPE_SINGLE_MODE;
params.video_path = FRAME_BASE_PATH_AMVIDEO;
params.display_mode = DISPLAY_MODE_AMVIDEO;
params.format = m_am_private->video_codec_type;
params.width = m_am_private->video_width;
params.height = m_am_private->video_height;
params.rate = m_am_private->video_rate;
params.ratio = m_am_private->video_ratio;
params.ratio64 = m_am_private->video_ratio64;
params.param = NULL;The amcodec in that repository is quite old and we've a much more improved one which is installed in OSMC automatically. You can get the appropriate headers with sudo apt-get install libamcodec-dev-osmc. That's just a newer library itself -- the examples are the same and unchanged.
Is the source for that available somewhere? I found it useful to look at the old one to try to figure out what the various functions and fields do.
I assume you stopped Kodi first: sudo systemctl stop mediacenter
Yep
It might be related to the decoder type being used when initializing libamcodec. For HEVC playback one needs to set
codecParam.decoder_type = DECODER_TYPE_FRAME_MODE
When I set DECODER_TYPE_FRAME_MODE for HEVC, it seems to break even worse. dmesg fills up with [0]cur lcu idx = 0, (total 920), set error_mark messages and the error count as reported by codec_get_vdec_state() increases with each frame.
[ 386.289232] the demux clock on, ref cnt: 1
[ 386.289469] the parser_top clock on, ref cnt: 1
[ 386.289474] the vdec clock on, ref cnt: 1
[ 386.289498] the clk_hevc_mux clock on, ref cnt: 1
[ 386.289508] hevc mux clock is 500000000 Hz
[ 386.289623] vdec_create instance 000000004e80c985, total 1, PM: legacy
[ 386.289722] no drmmode
[ 386.289786] vdec_init, dev_name:ammvdec_h265, vdec_type=VDEC_TYPE_FRAME_BLOCK id = 0
[ 386.292733] decoder_bmmu_box_alloc_box, tvp_flags = 0
[ 386.293169] [0]dynamic_buf_num_margin=7
[ 386.293175] [0]double_write_mode=0
[ 386.293183] [0]h265:pts_unstable=0
[ 386.293189] [0]h265: ver (201602101,0) decinfo: 1280x720 rate=1600
[ 386.293940] [TEE], the fw (hevc_mmu_swap) will be loaded.
[ 386.294037] hevc mux clock is 666666666 Hz
[ 386.294041] hevc video changed to 3840 x 2160 60 fps clk->667MHZ
[ 386.294287] vfm map created
[ 386.294334] vf_reg_provider Error to notify receiver
[ 386.294338] Error: vf_notify_receiver, fail to get receiver of provider vdec.h265.00
[ 386.294341] Error: vf_notify_receiver, fail to get receiver of provider vdec.h265.00
[ 386.294345] vdec_init, vf_provider_name = vdec.h265.00, b 0
[ 386.320143] vdec->port_flag=0x102, port_flag=0x10b
[ 386.324283] osmc videoenhancement: session context configured successfully
[ 386.338968] [0]vh265 cached=16384 need_size=16384 speed= 40 ms
[ 386.341230] [0]hevc->performance_profile 0
[ 386.341253] [0]set pic_list_init_flag 1
[ 386.343744] [0]init_buf_spec2 1280 736
[ 386.343752] [0]set pic_list_init_flag to 2
[ 386.343793] [0]set pic_list_init_flag to 3
[ 386.343796] [vdec_kpi][vh265_isr_thread_fn] First I frame coming.
[ 386.549268] [0]timeout_process decoder timeout
[ 386.549277] [0]cur lcu idx = 151, (total 920), set error_mark
[ 386.549318] [0]cur lcu idx = 151, (total 920), set error_mark
[ 386.549813] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.549817] [0]VH265: output first frame
[ 386.549837] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.550312] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.550754] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.550758] [0]VH265: output first frame
[ 386.550808] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.551277] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.551283] [0]VH265: output first frame
[ 386.551306] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.551753] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.551756] [0]VH265: output first frame
[ 386.551773] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.552244] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.552665] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.552669] [0]VH265: output first frame
[ 386.552684] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.553146] [0]cur lcu idx = 0, (total 920), set error_mark
[ 386.553567] [0]cur lcu idx = 0, (total 920), set error_markAdditionally, the VFM map doesn't seem to be set in the logs.
Hmm, what's a VFM map?
You should initialise the codec parameter structure fully, i.e:
This is what I'm doing now but no luck so far: https://github.com/moonlight-stream/moonlight-embedded/commit/87cad37958210dbcda2d2f14cccc99ad72449bb3
With H.264 I get these log messages:
Stream info: 1280x720x60 | Errors: 0 | Status: 0x16
Stream info: 1280x720x60 | Errors: 0 | Status: 0x6
Stream info: 1280x720x60 | Errors: 0 | Status: 0x6
Stream info: 1280x720x60 | Errors: 0 | Status: 0x6
Stream info: 1280x720x60 | Errors: 0 | Status: 0x6
Stream info: 1280x720x60 | Errors: 0 | Status: 0x6
...With HEVC:
Stream info: 0x0x60 | Errors: 0 | Status: 0x1
Stream info: 1280x736x60 | Errors: 0 | Status: 0x5
Stream info: 1280x736x60 | Errors: 1 | Status: 0x5
Stream info: 1280x736x60 | Errors: 2 | Status: 0x5
Stream info: 1280x736x60 | Errors: 3 | Status: 0x5
Stream info: 1280x736x60 | Errors: 4 | Status: 0x5
Stream info: 1280x736x60 | Errors: 5 | Status: 0x5
...Do I need to do anything special with fb blanking for the Vero 4K? Right now Moonlight blanks both fb0 and fb1 on Amlogic devices. There are also things like /sys/class/video/disable_video (which defaults to 1) that maybe I need to mess with?
If you've got a simple amcodec sample that works on the Vero 4K, I can probably easily mimic what that does. I just couldn't get any of the ones from the old amcodec repo working at all.
tanio99
commented
1 year ago Hmm, what's a VFM map?
That's the missing link :-) . That "map" tells the kernel how video processing should be done. Without that map you get errors like
[ 386.294334] vf_reg_provider Error to notify receiver
[ 386.294338] Error: vf_notify_receiver, fail to get receiver of provider vdec.h265.00If you've got a simple amcodec sample that works on the Vero 4K, I can probably easily mimic what that does. I just couldn't get any of the ones from the old amcodec repo working at all.
There's no "simple" sample. The only one which I could give you is the Kodi source :-) . But you can have a look at https://github.com/tanio99/xbmc/blob/Leia-OSMC/xbmc/cores/VideoPlayer/DVDCodecs/Video/AMLCodec.cpp . Although that one is a little bit outdated, it should give you an idea how Kodi is talking to the kernel to get things working. In that file you find an OpenDecoder function which initializes the decoder. You will also find an example how the vfm map should be set:
SetVfmMap("default", "decoder ppmgr deinterlace amlvideo amvideo");That command sets the "default" vfm map to the modules decoder, ppmgr, deinterlace, amlvideo and amvideo. The meaning of this "map" is that the decoder will feed each frame to the ppmgr module, then that frame will be sent through the deinterlace module and so on. Important to know is, that the amlvideo module is being used to communicate and sync with the user mode application. In the above mentioned AMLCodec.cpp you will find a GetPicture function which gets a decoded frame from the kernel through that amlvideo module. Well, you don't get the decoded frame, but just some metadata like e.g. the timestamp of that frame. You can then decide (e.g. based on that timestamp) when that frame should be displayed. You do that by calling the ReleaseFrame function, which tells the kernel that the given frame should be passed to the next module on the vfm map. The "amvideo" module is then doing all the magic to finally display that frame.
As you see, without that map you will never ever get any frame out of the Vero.
If you want to look at the current version of AMLCodec.cpp (which no longer exists btw) you will have to look at that patch (https://github.com/osmc/osmc/blob/master/package/mediacenter-osmc/patches/vero3-000-add-vero-support.patch) and extract the AMLInsecureVideoCodec.cpp file which is the successor and e.g. the HEVCCodec.cpp which is the h.265 decoder which inherits from AMLInsecureVideoCodec.
That patch contains also the LibAmcodec.cpp which is the interface to the libamcodec.so shared library.
Finally, you get errors like
[ 386.549268] [0]timeout_process decoder timeout
[ 386.549277] [0]cur lcu idx = 151, (total 920), set error_mark
[ 386.549318] [0]cur lcu idx = 151, (total 920), set error_mark
[ 386.549813] [0]cur lcu idx = 0, (total 920), set error_markwhenever the decoder was unable to decode a given frame. Most often the reason is that the encoded frame had some errors in it.
cgutman
commented
1 year ago As you see, without that map you will never ever get any frame out of the Vero.
On a freshly booted Vero (with Kodi disabled), it looks like vfm is set to default { decoder(0) ppmgr(0) deinterlace(0) amlvideo(0) amvideo} which works for us. Is that default initialization safe to rely on?
You can then decide (e.g. based on that timestamp) when that frame should be displayed. You do that by calling the ReleaseFrame function, which tells the kernel that the given frame should be passed to the next module on the vfm map. The "amvideo" module is then doing all the magic to finally display that frame.
Aha! This was the missing part. Adding a pair of VIDIOC_DQBUF and VIDIOC_QBUF ioctl() calls makes the video display properly.
This code was previously used by other Amlogic decoders that apparently didn't need these ioctls to be invoked to display the output on screen. How do I tell programmatically if I'm dealing with a decoder that needs VIDIOC_DQBUF+VIDIOC_QBUF vs. one where codec_write() alone is enough? I don't want to break existing Amlogic devices with the Vero changes.
Most often the reason is that the encoded frame had some errors in it.
Yep, the reason is obvious in hindsight. DECODER_TYPE_FRAME_MODE is telling the decoder we will give it full frames, but we call codec_write() multiple times with partial frames. DECODER_TYPE_SINGLE_MODE seems to work for me for both H.264 and HEVC without issues.
tanio99
commented
1 year ago How do I tell programmatically if I'm dealing with a decoder that needs VIDIOC_DQBUF+VIDIOC_QBUF vs. one where codec_write() alone is enough?
It's not the decoder that needs the ioctl calls - it's the amlvideo module in the vfm map. If you remove it then you don't need the ioctl calls. But be aware - that's a setup which we've never used and tested. The setup would be:
echo add default decoder ppmgr deinterlace amvideo > /sys/class/vfm/mapIf you want to check programmatically if you're running on a Vero, then you could use the output of
cat /proc/cpuinfoand grep for "OSMC Vero".
cgutman
commented
1 year ago Okay, with https://github.com/moonlight-stream/moonlight-embedded/commit/b2192eda254fc55dff31ca0d6709b0a26661ef72 and my prior ALSA fixes, Moonlight now works pretty well on the Vero at 4K 30 FPS.
However, I am seeing significant video latency issues (independent of resolution) when streaming at 60 FPS. I assume this is a 59.94 Hz display vs 60 FPS stream issue which is causing those extra frames to accumulate in the display output pipeline. The accumulated delay can get up to 5+ seconds when streaming for a while.
I'm not seeing extended delays from either codec_write() or VIDIOC_DQBUF/VIDIOC_QBUF, so it looks like the buffering is happening further along in the display pipeline, perhaps in amvideo.
Is there a way to either limit the number of frames buffered (preferably to 1 or 0), or request that amvideo only display the most recently queued frame? If neither of those are possible, is there a way I can synchronize with VBlank to dispatch frames in lock step with the rate they are being consumed by the output?
tanio99
commented
1 year ago Okay, with https://github.com/moonlight-stream/moonlight-embedded/commit/b2192eda254fc55dff31ca0d6709b0a26661ef72 and my prior ALSA fixes, Moonlight now works pretty well on the Vero at 4K 30 FPS.
Great to hear that!
I assume this is a 59.94 Hz display vs 60 FPS stream issue
Hm, if it's a fractional rate thing, then it could help to set the "fractional rate policy" like it is done in Kodi:
int fractional_rate = (res.fRefreshRate == floor(res.fRefreshRate)) ? 0 : 1;
CLog::Log(LOGDEBUG, "AMLUtils::aml_set_display_resolution setting frac_rate_policy to %d", fractional_rate);
SysfsUtils::SetInt("/sys/class/amhdmitx/amhdmitx0/frac_rate_policy", fractional_rate);The buffering you've mentioned happens directly in amlvideo, right in the ioctl(videoFd, VIDIOC_QBUF, &vbuf) call, where the received frame is put into a queue. There's an interrupt service routine in amvideo running, which is being called 60 or 59.94 times a second depending on the refresh rate. The ISR will look at the queue in amlvideo then take a frame and display it.
If you want to sync with VBlank then you would need to use polling like it was done in AMLCodec.cpp (see CAMLCodec::PollFrame() function). But with kernel 4.9 that method became quite unreliable, since the thread wakeup time varies too much, and that was causing a lot of micro-stutter issues.
TheChoconut
commented
1 year ago It's great to see some progress on the AMLogic backend, I have been using hacks ever since to try and get this to work on CoreELEC... But this new code seems to work there too.
I've also noticed a delay at 60 FPS, which was happening on my end. Feeding pts into the decoder seems to fix this:
codec_checkin_pts(&codecParam, decodeUnit->presentationTimeMs);
I don't know why this works, but it seems to be fix the delay?
UPDATE: Doesn't actually work. It only works if combined with sending full frames to decoder and using STREAM_TYPE_FRAME.
cgutman
commented
1 year ago @TheChoconut Can you submit your changes upstream? It looks like several of them should be acceptable as-is.
For the others (like https://github.com/moonlight-stream/moonlight-embedded/commit/02d9f42761cc9cebec9754a68b363ccc9a8331b0), we can collaborate on a solution that works for all AML devices.
samnazarko
commented
1 year ago As always, I'm happy to provide hardware to participating developers to improvement outcomes for OSMC Vero owners.
Cheers
Sam
cgutman
commented
1 year ago Things are in pretty good shape now on the Vero 4K as of 00c28f53d8e7abb5b8468b10cac055d9414c5b16, however I'm seeing some issues with H.264 where the video freezes suddenly and the dmesg log is full of:
[1210245.164026] 0: frame number gap error
[1210245.165691] 0: frame number gap error
[1210245.167596] 0: frame number gap error
[1210245.169410] 0: frame number gap error
[1210245.170936] 0: frame number gap error
[1210245.172667] 0: frame number gap error
[1210245.174431] 0: frame number gap error
[1210245.176183] 0: frame number gap error
[1210245.177937] 0: frame number gap error
[1210245.179854] 0: frame number gap error
[1210245.181579] 0: frame number gap error
[1210247.575741] 0: frame number gap errorI don't believe a frame was actually dropped, since moonlight-common-c logs in that case and I don't see any frame drop logged. This also seems to only affect H.264, while HEVC seems completely unaffected.
tanio99
commented
1 year ago These errors are new to me, need to investigate. Usually, errors like these are coming from the decoder and indicate issues with the stream.
tanio99
commented
1 year ago Ok, it seems there's something wrong with the stream. According to the H.264 spec:
When gaps_in_frame_num_value_allowed_flag is equal to 0, the bitstream shall not contain data that result in
frame_num not being equal to PrevRefFrameNum or ( PrevRefFrameNum + 1 ) % MaxFrameNum.
NOTE – When gaps_in_frame_num_value_allowed_flag is equal to 0 and frame_num is not equal to PrevRefFrameNum
and is not equal to ( PrevRefFrameNum + 1 ) % MaxFrameNum, the decoding process should infer an unintentional loss of
pictures.there's a flag in the SEI indicating if frames may be missing or not. If that flag isn't set than the decoder assumes that frames are missing and it signals that situation with the error message above.
Maybe it helps to (as root)
echo 16764854 >/sys/module/amvdec_mh264/parameters/error_proc_policythat would deactivate that gap check.
TheChoconut
commented
1 year ago Does anyone know if there's a way to extract how long it took for the decoder to decode the frame and display it? That would help get some stats and would also help in noticing what frames cause delays and if it's moonlight or the processor's fault for the delay...
tanio99
commented
1 year ago No, there's unfortunately no way to get some metrics out of the decoder/video chain. I did some "performance tests" last year or so by adding some logs. The average decoding time for VP9 sources is about 17 ms per frame. For h.264 sources it varies depending on the frame type. I-Frames for instance can take up to 0.2 seconds, whereas other frames take only 2-5 ms. I did not measure h.265.
samnazarko
commented
1 year ago 
EvgeniySpinov
commented
1 year ago Alternatively https://github.com/LizardByte/Sunshine can be used? I haven't tried it yet, but it seems that it's the way to go.
EvgeniySpinov
commented
1 year ago Was trying to build moonlight from master in order to test it with HEVC, however I'm getting:
RTSP port: 48010
Initializing platform...done
Resolving host name...done
Initializing audio stream...done
Starting RTSP handshake...Audio port: 48000
Video port: 47998
Control port: 47999
done
Initializing control stream...done
Initializing video stream...done
Initializing input stream...done
Starting control stream...done
Error: failed to open X display.
Starting video stream...Video stream start failed: -1
Stopping control stream...ENet peer acknowledged disconnection
done
Cleaning up input stream...done
Cleaning up video stream...done
Cleaning up control stream...done
Cleaning up audio stream...done
Cleaning up platform...done
EVIOCGRAB failed with error 16Moonlight version: Moonlight Embedded 2.5.3-master-00c28f5
Vero 4K, but OSMC version is not the most recent (kernel 3.14.29-160-osmc)
Is there any specific parameters I need to use or I should just upgrade?
samnazarko
commented
1 year ago You will need to be on a newer version of OSMC with the 4.9.269 kernel.
Your OSMC version will be very old (likely pre Aug 2021).
EvgeniySpinov
commented
1 year ago Thank you Sam. I'm tending not to update when everything works.
Right, I've updated to most recent version, got kernel 4.9.269-16-osmc, rebuilt moonlight, however the problem persists.
Also when I install sudo apt-get install rbp-userland-dev-osmc I get:
The following additional packages will be installed:
rbp-userland-osmc
The following NEW packages will be installed:
rbp-userland-dev-osmc rbp-userland-osmc
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/4934 kB of archives.
After this operation, 19.1 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
(Reading database ... 61451 files and directories currently installed.)
Preparing to unpack .../rbp-userland-osmc_3.8.0-2_armhf.deb ...
dpkg-divert: error: 'diversion of /etc/kernel-img.conf to /etc/kernel-img.conf.distributed by rbp-userland-osmc' clashes with 'diversion of /etc/kernel-img.conf to /etc/kernel-img.conf.distributed by vero3-userland-osmc'
dpkg: error processing archive /var/cache/apt/archives/rbp-userland-osmc_3.8.0-2_armhf.deb (--unpack):
new rbp-userland-osmc package pre-installation script subprocess returned error exit status 2
Selecting previously unselected package rbp-userland-dev-osmc.
Preparing to unpack .../rbp-userland-dev-osmc_3.8.0-2_armhf.deb ...
Unpacking rbp-userland-dev-osmc (3.8.0-2) ...
Errors were encountered while processing:
/var/cache/apt/archives/rbp-userland-osmc_3.8.0-2_armhf.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)Not sure if it affects the build, but I was unable to overcome this. Otherwise error looks exactly the same as above.
Thoughts?
samnazarko
commented
1 year ago You shouldn’t have Raspberry Pi packages on this system (rbp-userland), what instructions did you follow?
Sam
EvgeniySpinov
commented
1 year ago Those from the project: https://github.com/moonlight-stream/moonlight-embedded/wiki/Full-details-for-Compilation-and-Installation-in-Raspbian-Or-OSMC
There is separate line for OSMC. Outdated?
EvgeniySpinov
commented
1 year ago Removed rbp- packages and installed vero3- instead. Now it looks like:
# dpkg -l | grep vero3-userland
ii vero3-userland-dev-osmc 2.0.4 armhf Vero 3 userland package with headers
ii vero3-userland-osmc 2.0.4 armhf Vero 3 userland packageRebuilt moonlight, however problem presists, i.e. nothing has changed. Not sure what could be done from here. Strace with start: https://pastebin.com/frj998n3
Need to mention that I'm executing this from SSH with media server shut down. Cannot execute it directly on device, as after shutting down media server I'm getting black screen without any output at all. Probably it's related.
On host machine each connect is detected via adding audio device and even moonlight process termination on Vero is not bringing it's back. Have to restart GameStream.
samnazarko
commented
1 year ago Hi @EvgeniySpinov
Starting from the command line should be fine. From your strace I can see Wayland and GBM libraries being loaded for windowing. On Vero 4K / 4K + we use GLES.
I suspect that your binary has been built against the wrong windowing system; either because of some leftover Raspberry Pi packages or the presence of MESA on your system.
EvgeniySpinov
commented
1 year ago This is not exactly the case:
ldd /usr/local/bin/moonlight | grep vero
libEGL.so.1 => /opt/vero3/lib/libEGL.so.1 (0xf5c0d000)And in strace:
openat(AT_FDCWD, "/opt/vero3/lib/libEGL.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\20|\2\0004\0\0\0tl\21\0\2\0\0\5"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0775, st_size=1387197, ...}) = 0
mmap2(NULL, 1177624, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xf593f000
mprotect(0xf5a51000, 28672, PROT_NONE) = 0
mmap2(0xf5a58000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x111000) = 0xf5a58000
mmap2(0xf5a5e000, 2072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xf5a5e000
mprotect(0xfff5e000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC|PROT_GROWSDOWN) = 0
close(3) = 0
openat(AT_FDCWD, "/opt/vero3/lib/libGLESv2.so.2", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\20|\2\0004\0\0\0tl\21\0\2\0\0\5"..., 512) = 512
....Unless you refer to another part where wayland and GBM are effectively used to determine X Display, since I haven't found such.
Probably there is env variable I need to set for moonlight to pick up display correctly?
samnazarko
commented
1 year ago It's good to see that Moonlight is linked against the correct libraries.
I haven't got an NVIDIA GPU to test this with; but I'm sure there are some compilation instructions somewhere in this repository for AMLogic devices.
EvgeniySpinov
commented
1 year ago Seems like I could use some help from @cgutman
Maybe there are some parameters I need to use to build moonlight in order to have it properly detect X display?
EvgeniySpinov
commented
1 year ago Probably this is the issue with display is about this: https://github.com/moonlight-stream/moonlight-embedded/wiki/Packages#additional-required-setup-steps-for-raspbian-bullseye
@samnazarko @tanio99 What is the way to have config.txt parameters in OSMC? It seems that recent version doesn't have config.txt anymore.
samnazarko
commented
1 year ago If you're referring to Raspberry Pi, we still have config.txt.
However we now recommend editing config-user.txt, as we may overwrite config.txt as we see fit. This file is found on the /boot partition.
EvgeniySpinov
commented
1 year ago Actually I've meant Vero. You're saying I can create /boot/config-user.txt (file is not there), and put those configuration items in it and it will be picked up during boot? Or process is different for Vero?
samnazarko
commented
1 year ago No -- we don't have a config.txt on Vero and haven't in the past. The commands suggested in that guide are for Raspberry Pi and aren't related - won't do anything on OSMC.
From the instructions you've linked to, I can see that it enables a Moonlight repository, however the packages that it makes available will be for Raspberry Pi and not the Vero. I believe you'll need to compile it from source and specify AMLogic support when building so it works on the device.
EvgeniySpinov
commented
1 year ago What I've meant is this part of instructions:
Is there a way to do this on Vero?
samnazarko
commented
1 year ago These instructions are Raspberry Pi specific, they’re not needed nor do they apply to Vero. But the packages the author is offering is built for Pi hardware not Vero.
EvgeniySpinov
commented
1 year ago Ok, given another try to moonlight-embedded. Updated and rebuilt. Tried without any third-party scripts, directly from console, in order to avoid any issues related to them.
Now version is: Moonlight Embedded 2.5.3-master-5bb47ce
Stopped media server manually and running via SSH:
$ moonlight stream -app 5 -verbose
Moonlight Embedded 2.5.3 (AML;SDL;X11;VDPAU;VAAPI;ALSA;PULSE;EMBEDDED)
Searching for server...
Connecting to 10.0.4.21...
GPU: , GFE: 3.23.0.74 (, 7.1.431.-1)
Platform SDL2 (software decoding)
The path /dev/dri/ cannot be opened or is not available
The path /dev/dri/ cannot be opened or is not available
Could not initialize SDL - No available video deviceIf I try with AML:
$ moonlight stream -app 5 -verbose -platform aml
Moonlight Embedded 2.5.3 (AML;SDL;X11;VDPAU;VAAPI;ALSA;PULSE;EMBEDDED)
Searching for server...
Connecting to 10.0.4.21...
GPU: , GFE: 3.23.0.74 (, 7.1.431.-1)
Platform Unknown
Platform 'aml' not foundUPD: It seems that I've found how to make it work - it works when started directly from build directory. Even though after "make install" binaries in /usr/local are the same, including /usr/local/lib/libmoonlight-aml.so and modified ld.so.conf.d
Not sure what is causing this, but for now, I'm going to test it using build directory in order to test all modes first
EvgeniySpinov
commented
1 year ago Ok, I've ran some tests in different modes with Sunshine (0.18.4) + Moonlight and everything seems to work fine. I've tested resolutions from 720p till 2160p with rates from 8Mbps to 150Mbps. Wired network.
The only issue I can see if that HDR is not working. When I try to start HDR stream I get the log: Client side: https://paste.osmc.tv/akocujugax Server side (verbose, Software encoder enforced, NVENC doesn't support HDR on GTX 970): https://paste.osmc.tv/iyafitemuf.hs
Screen shows 1st frame and freezes till moonlight termination in around 10 seconds. Second attempt brings black screen.
HDR is supported on the host by GPU and monitor. And HDR is supported by Vero 4K+ and TV. Also Vero 4K+ can play HDR content from OSMC.
However when moonlight is started with HDR mode - TV is not entering HDR mode.
I've spent some time investigating issue I had with not being able to stream 4K and figured that I can stream 4K only when OSMC is configured to this resolution. Otherwise I get stream the same as initial resolution after boot. So my guess is that moonlight cannot change resolution on Vero 4K+ and switch on HDR as well.
Would be glad to help with experiments in this area.
UPD: Added server side logs.
samnazarko
commented
1 year ago Hi
I’m currently travelling on business and behind on a few things. I’ll make a note to follow up on this early next week.
Sam
From: EvgeniySpinov @.> Reply to: moonlight-stream/moonlight-embedded @.> Date: Tuesday, 11 April 2023 at 23:27 To: moonlight-stream/moonlight-embedded @.> Cc: Sam Nazarko @.>, Mention @.***> Subject: Re: [moonlight-stream/moonlight-embedded] Vero 4K support (#573)
Ok, I've ran some tests in different modes with Sunshine (0.18.4) + Moonlight and everything seems to work fine. I've tested resolutions from 720p till 2160p with rates from 8Mbps to 150Mbps. Wired network.
The only issue I can see if that HDR is not working. When I try to start HDR stream I get the log: https://paste.osmc.tv/akocujugax
Screen shows 1st frame and freezes till moonlight termination in around 10 seconds. Second attempt brings black screen.
HDR is supported on the host by GPU and monitor. And HDR is supported by Vero 4K+ and TV. Also Vero 4K+ can play HDR content from OSMC.
However when moonlight is started with HDR mode - TV is not entering HDR mode.
I've spent some time investigating issue I had with not being able to stream 4K and figured that I can stream 4K only when OSMC is configured to this resolution. Otherwise I get stream the same as initial resolution after boot. So my guess is that moonlight cannot change resolution on Vero 4K+ and switch on HDR as well.
Would be glad to help with experiments in this area.
— Reply to this email directly, view it on GitHubhttps://github.com/moonlight-stream/moonlight-embedded/issues/573#issuecomment-1504206443, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHDWLF3FNWGUMT66NOVTXDXAXLDVANCNFSM4D6F24BA. You are receiving this because you were mentioned.Message ID: @.***>
cgutman
commented
1 year ago Server side (verbose, Software encoder enforced, NVENC doesn't support HDR on GTX 970)
Sunshine doesn't support HDR encoding with the software encoder. That's why you see all those "Warning: No compatible capture formats for this encoder" messages and Moonlight complains that no video was ever received.
EvgeniySpinov
commented
1 year ago That's unfortunate. As even if NVENC would support HDR (in case of GTX 970 it doesn't), I wouldn't use NVENC for streaming - it consumes 10-20% of GPU either in raw performance (my case) or in TDP (more modern GPUs), thus encoding with CPU (which in most cases is 40-60% idle) is more preferred.
Since @cgutman is contributor to HDR support in Sunshine (thank you for that!), may I ask, if software encoder update to support HDR is planned?
And regarding this particular issue - the only thing that remains is installation which doesn't work. I'll try to figure what could be done here and ideally would end up with PR.
For those OSMCers whould would be reading this, now on OSMC your setup might look like this:
cgutman
commented
1 year ago if software encoder update to support HDR is planned?
Not at the moment. Color conversion on the CPU is already really slow even with 8bpc SDR frames, because it's currently single-threaded. HDR frames are also twice the size of SDR frames, which makes the performance problem even worse.
It also doesn't look like libswscale supports doing those linear gamma FP16 to Rec 2100 PQ color conversions that the we need to handle the HDR frames that Windows captures. I haven't looked really deeply though, so I may be wrong.
EvgeniySpinov
commented
1 year ago Thank you for detailed explanation. Probably I'm missing something, but to my knowledge software encoding could be multi-threaded (Sunshine has this setting as well), and overhead is coming from sync of frames provided by multiple streams, which is reduced by optimizations made within x264 encoder, used by ffmpeg library, which Sunshine utilizes. Am I getting it wrong? Or there are some specifics within Sunshine on output pipeline?
From my personal experience, I see no difference in latency between NVENC (x264) and Software encoder (x264). In both cases it's noticable at 4K@80Mbit/s comparing to native display. Reducing bitrate and/or resultion reduces latency for both of them. I'm using Ryzen 3600 - not a most advanced CPU. As well as 2 to 3 encoding threads (still have to find a sweet spot). CPU utilization is around 50-70% during game session, depending on game/scene/etc. More capable CPUs should reduce latency and utilization even further.
To summarize, from my experience, so far, it seems that increasing 20-50% on load for encoder (this is average computational effort increase to encode 10bit HDR frame, even though the size of frame is 1.25 times bigger compared to 8bit SDR frame), should still keep software econding as a viable option. This is in theory, and numbers though are a bit different (in a way of reduction) than you've provided. If you can share more details on why it's 2 times the frame size and what latency it might introduce, even in theory, let's say in 1080p@60fps@40Mbit stream - that would be great.
As for libswscale, it seems that you're right. There is workaround though, through using zscale filter from z.lib, which supports high bit-depth and wide color gamut conversions, including conversions involving FP16 and Rec. 2100 PQ. Not sure how hard it would be to incorporate into Sunshine though.
NVidia Geforce Experience version: 3.9.0.97 Moonlight Embedded version: 2.4.3 Moonlight Embedded source: compiled from source Moonlight Embedded running on: Vero 4K Moonlight Embedded running on distribution: OSMC
Verbose output
-verboseof Moonlight Embedded:I have forked the repo and made the necessary changes for moonlight to compile. AMLogic support is already included in the Vero 4K.
Is someone willing to check what might be wrong? Thanks