Feature request: RPi compatibility

[sergiohf]

I want to request a feature, to make this app compatible with raspberry pi 3 or later. I tried to compile (took a long until i discover that i need minimum Cmake 3.15) but after that when i put "make" the compilation fails at https://github.com/thestr4ng3r/chiaki/blob/master/gui/src/avopenglwidget.cpp . In this file the compiler cannot find glEnableVertexAttribArray and others. Someone tell me that with SDL2 and OpenGL ES that works with a RPI, it might just work, butI dont knowif its possible.

[issue-label-bot[bot]]

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

[Ackater]

I had no issues compiling Chiaki for Arch Linux ARM on my Raspberry Pi 4.

But it doesn't run smooth, and the log is filled entirely with

AVCodec internal buffer is full removing frames before pushing

[Ackater]

I made another attempt, this time with ffmpeg compiled with MMAL enabled to take advantage of the Pi's hardware decoder. There might be less lag, but now there's a lot of artifacts and glitching. Here's what the log looks like:

[2020-01-10 11:07:39:831831] [D] StreamConnection reporting corrupt frame(s) from 1319 to 1321
[2020-01-10 11:07:39:880880] [W] Requested key stream for key pos 0x100000003 on GKCrypt 240, but it's not in the buffer
[2020-01-10 11:07:39:880880] [W] Already requested a higher key pos than in the buffer, skipping ahead from min 0x124e000 to 0x1270000
[2020-01-10 11:07:39:880880] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1120, but it's not in the buffer
[2020-01-10 11:07:39:880880] [I] Frame Processor received 1+0 / 5+1 units, attempting FEC
[2020-01-10 11:07:39:881881] [E] FEC failed
[2020-01-10 11:07:39:881881] [W] Missing unit 0x1
[2020-01-10 11:07:39:881881] [W] Missing unit 0x2
[2020-01-10 11:07:39:881881] [W] Missing unit 0x3
[2020-01-10 11:07:39:881881] [W] Missing unit 0x4
[2020-01-10 11:07:39:885885] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b1, tries: 1
[2020-01-10 11:07:39:885885] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b2, tries: 1
[2020-01-10 11:07:39:886886] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b4, tries: 0
[2020-01-10 11:07:39:972972] [W] Detected missing or corrupt frame(s) from 1324 to 1331
[2020-01-10 11:07:39:972972] [D] StreamConnection reporting corrupt frame(s) from 1324 to 1330
[2020-01-10 11:07:39:986986] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b3, tries: 1
[2020-01-10 11:07:40:008008] [I] Frame Processor received 8+0 / 12+1 units, attempting FEC
[2020-01-10 11:07:40:008008] [E] FEC failed
[2020-01-10 11:07:40:008008] [W] Missing unit 0x8
[2020-01-10 11:07:40:009009] [W] Missing unit 0x9
[2020-01-10 11:07:40:009009] [W] Missing unit 0xa
[2020-01-10 11:07:40:009009] [W] Missing unit 0xb
[2020-01-10 11:07:40:009009] [E] AVCodec internal buffer is full removing frames before pushing
[2020-01-10 11:07:40:009009] [E] AVCodec internal buffer is full removing frames before pushing
[2020-01-10 11:07:40:087087] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b1, tries: 2
[2020-01-10 11:07:40:087087] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b2, tries: 2
[2020-01-10 11:07:40:088088] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b4, tries: 1
[2020-01-10 11:07:40:088088] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b5, tries: 0
[2020-01-10 11:07:40:123123] [W] Detected missing or corrupt frame(s) from 1333 to 1334
[2020-01-10 11:07:40:123123] [D] StreamConnection reporting corrupt frame(s) from 1333 to 1333
[2020-01-10 11:07:40:124124] [I] Frame Processor received 9+1 / 12+1 units, attempting FEC
[2020-01-10 11:07:40:125125] [E] FEC failed
[2020-01-10 11:07:40:125125] [W] Missing unit 0
[2020-01-10 11:07:40:126126] [W] Missing unit 0x1
[2020-01-10 11:07:40:126126] [W] Missing unit 0x2
[2020-01-10 11:07:40:188188] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b3, tries: 2
[2020-01-10 11:07:40:189189] [I] Takion Send Buffer re-sending packet with seqnum 0x5c5b94b6, tries: 0
[2020-01-10 11:07:40:190190] [W] Detected missing or corrupt frame(s) from 1333 to 1335
[2020-01-10 11:07:40:191191] [D] StreamConnection reporting corrupt frame(s) from 1333 to 1334
[2020-01-10 11:07:40:214214] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1120, but it's not in the buffer
[2020-01-10 11:07:40:215215] [W] Detected missing or corrupt frame(s) from 1336 to 1350
[2020-01-10 11:07:40:215215] [D] StreamConnection reporting corrupt frame(s) from 1336 to 1349
[2020-01-10 11:07:40:216216] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1152, but it's not in the buffer
[2020-01-10 11:07:40:217217] [W] Requested key stream for key pos 0x100000003 on GKCrypt 240, but it's not in the buffer
[2020-01-10 11:07:40:218218] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1184, but it's not in the buffer
[2020-01-10 11:07:40:218218] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1152, but it's not in the buffer
[2020-01-10 11:07:40:218218] [W] Requested key stream for key pos 0x100000003 on GKCrypt 240, but it's not in the buffer
[2020-01-10 11:07:40:222222] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1120, but it's not in the buffer
[2020-01-10 11:07:40:222222] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1168, but it's not in the buffer
[2020-01-10 11:07:40:226226] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1168, but it's not in the buffer
[2020-01-10 11:07:40:226226] [W] Already requested a higher key pos than in the buffer, skipping ahead from min 0x1271000 to 0x12cd000
[2020-01-10 11:07:40:227227] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1120, but it's not in the buffer
[2020-01-10 11:07:40:227227] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1152, but it's not in the buffer
[2020-01-10 11:07:40:227227] [W] Requested key stream for key pos 0x9ea24a7200000003 on GKCrypt 1120, but it's not in the buffer
[2020-01-10 11:07:40:228228] [I] Frame Processor received 12+1 / 15+1 units, attempting FEC
[2020-01-10 11:07:40:228228] [E] FEC failed

[juicypop]

@Ackater what rpi 4 version are you running this on? Is this the 4gb version and if not could more ram hopefully solve this problem? And one last question, did you improved your last experience or is it a dead end at the moment?

I'm planning eventually buying a 4gb rpi and hope this will work one day.

Looking forward for your response.

[founta]

I am also attempting to install and run Chiaki on a Raspberry Pi 4B. This is the 2GB RAM edition with Raspbian Buster Desktop installed. I am able to compile without a hitch but run into OpenGL/qt5 errors when starting the application and when starting the remote play session, preventing me from viewing the video stream.

Here are the steps I've taken and the results I'm getting, if anyone is interested:

#install prereqs
sudo apt-get install cmake libqt5opengl5-dev libqt5svg5-dev libqt5multimedia5 qtmultimedia5-dev libavcodec-dev libopus-dev libssl-dev libprotoc-dev protobuf-compiler python3-protobuf libsdl2-dev libswscale-dev
sudo ldconfig

#obtain and build chiaki
cd ~/
git clone https://github.com/thestr4ng3r/chiaki.git
cd chiaki
git submodule update --init
mkdir build && cd build
cmake ..
make

#start program
./gui/chiaki

#get this error on startup -- foreboding but no visible impact
# QEGLPlatformContext: Failed to create context: 3009

#register ps4

#start stream

#these qt5 and/or opengl errors pop up repeatedly and 
#the stream window is not rendered:
# QEGLPlatformContext: Failed to create context: 3009
# composeAndFlush: QOpenGLContext creation failed
# composeAndFlush: makeCurrent() failed

#then these start showing up before I eventually close the stream window:
# [E] AVCodec internal buffer is full removing frames before pushing

I am using qt 5.11.3 (obtained from apt) and here is a chunk from glxinfo:

OpenGL version string: 2.1 Mesa 19.3.2
OpenGL shading language version string: 1.20
OpenGL ES profile version string: OpenGL ES 3.1 Mesa 19.3.2
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

Qt5 as installed on my pi seems to be ok; this Qt5 example compiles and runs as expected: https://code.qt.io/cgit/qt/qtbase.git/tree/examples/gui/openglwindow?h=5.11

Let me know if there's anything I'm missing.

[founta]

@Ackater, friend, how did you get Chiaki to run with OpenGL ES 3.1; that's the highest the Pi4 currently supports, right?

The QEGLPlatformContext: Failed to create context: 3009 error I had was because it was trying to make an OpenGL 3.2 context, which the Pi4 doesn't support. I sort of haphazardly changed it to use OpenGL ES 3.1 by changing the version statement in the shaders, setting float precision in the shaders, setting the AA_UseOpenGLES attribute in main before the QApplication object is created, and doing format.setVersion(3,1) instead of format.setVersion(3,2) in the CreateSurfaceFormat function. I'll put this version up in my fork of Chiaki shortly if anyone cares.

The newer OpenGL functions in use, glMapBufferRange, for example, are supported in ES 3.1 . I see that there was also a OpenGLES Chiaki in the works in the stale branch using OpenGLES 2 where VAOs were not supported. VAOs are now supported as of OpenGLES 3 so I thought things might just kind of work.

I only get this error (repeatedly) when running now, after making the changes listed above: QEGLPlatformContext: eglMakeCurrent failed: 3002 where 3002 is the EGL_BAD_MATCH error.

It has something to do with the surface not being compatible with the generated context (as per https://www.khronos.org/registry/EGL/sdk/docs/man/html/eglMakeCurrent.xhtml). I am not sure how to correct this; I am not very well versed in Qt, OpenGL, or graphics programming in general. Does anyone have any tips?

Edit: I've found the issue. Looking at the QSurfaceFormats in the AVOpenGLWidget's inherited context and surface, the context is created with the requested settings; it's QSurfaceFormat (returned from context()->format()) is set to OpenGL ES, version 3.1. However the surface's QSurfaceFormat indicates that it's using OpenGL ES, version 2.0, despite calling setFormat with version 3.1 and QSurfaceFormat::setDefaultFormat with version 3.1. The docs at https://doc.qt.io/qt-5/qoffscreensurface.html (context()->surface()->surfaceClass() says it's an offscreen surface) even say to set the surface format with the QSurfaceFormat returned from QOpenGLContext::format() to ensure compatibility, so I'm not sure why this is happening. Has anyone run into this sort of disaster before?

Edit again: That might not be the issue; I see the same thing happening with the versions in this qt5 example: https://code.qt.io/cgit/qt/qtbase.git/tree/examples/gui/openglwindow?h=5.11, and that one works as expected

[Questinghero]

I would love for this to work, but I do not have the skills to make it happen. just adding my vote. it would be awesome for this to be integrated into RetroPie (I use Steam Link on my pi all the time)

[djmonkeyuk]

Hey, I've started my own Raspberry Pi 4 B+ project to stream my various consoles. PS4 included. I found this issue and it's helped me get some progress, so just thought I'd share my thoughts so far.

With all these changes below, I managed to get things working, although the quality is variable. I will try to spend time looking into performance.

---

@founta Although the RPi4 is 3.1 compliant, the current version that is supplied and supported it 3.0. I don't think 3.1 stable and being pushed out via updates yet. You can check what you're device is using by running "glxinfo | grep version" in the terminal (make sure mesa-utils is installed first to get the command).

With this info, I checked your fork and saw your diffs, I rebased your changes on top of the current master, and swapped the version from 3.1 to 3.0. This got things working for me.

---

Simple fix for audio: Qt wasn't picking up my hardware until I did "sudo apt-get install libqt5multimedia5-plugins"

---

I checked the gles branch of this repo and saw there were renderer options in the settings, which have since been removed in the master branch. I've brought those settings back, but admittedly I don't think they actually did anything to the software. But never the less - I think it would be a nice thing to bring back to implement the feature so we can switch compatibility with older/new OpenGL versions, and possible Vulkan in the future.

---

One more thing, I recompiled ffmpeg to enable mmal. I followed this guide here : https://ubuntu-mate.community/t/tutorial-build-or-download-ffmpeg-libavcodec-with-mmal-support-hardware-acceleration-video-decoding/3565

I'm not sure this had any bearing on my ability to get this working.

---

A small thing: I have also made my fork load the stream in full screen as soon as you connect to your PS4

---

The fork and branch for my changes are here: https://github.com/charliebanks/chiaki/tree/feature/RPi-Effort

[djmonkeyuk]

And just to prove the above points. Here's a video of it all working.

I was going to say about performance, but it seems the only bottleneck now I believe is Wi-Fi speeds.

This is streaming all via ethernet connection, and has bluetooth joy-cons connected via another driver.

https://www.youtube.com/watch?v=_MN3ovU98Ro&

[thestr4ng3r]

I checked the gles branch of this repo and saw there were renderer options in the settings, which have since been removed in the master branch. I've brought those settings back, but admittedly I don't think they actually did anything to the software. But never the less - I think it would be a nice thing to bring back to implement the feature so we can switch compatibility with older/new OpenGL versions, and possible Vulkan in the future.

These options were not removed, they only never left that branch which I started some time ago but never finished. If anyone wants to finish it, I will be happy to receive their PR!

[djmonkeyuk]

These options were not removed, they only never left that branch which I started some time ago but never finished. If anyone wants to finish it, I will be happy to receive their PR!

Ah fair enough! Sorry, I just looked at the branch and assumed it was an older build - didn't spend the time looking at the details of it! :)

[levis1978]

Hey, I've started my own Raspberry Pi 4 B+ project to stream my various consoles. PS4 included. I found this issue and it's helped me get some progress, so just thought I'd share my thoughts so far.

With all these changes below, I managed to get things working, although the quality is variable. I will try to spend time looking into performance.

@founta Although the RPi4 is 3.1 compliant, the current version that is supplied and supported it 3.0. I don't think 3.1 stable and being pushed out via updates yet. You can check what you're device is using by running "glxinfo | grep version" in the terminal (make sure mesa-utils is installed first to get the command).

With this info, I checked your fork and saw your diffs, I rebased your changes on top of the current master, and swapped the version from 3.1 to 3.0. This got things working for me.

Simple fix for audio: Qt wasn't picking up my hardware until I did "sudo apt-get install libqt5multimedia5-plugins"

I checked the gles branch of this repo and saw there were renderer options in the settings, which have since been removed in the master branch. I've brought those settings back, but admittedly I don't think they actually did anything to the software. But never the less - I think it would be a nice thing to bring back to implement the feature so we can switch compatibility with older/new OpenGL versions, and possible Vulkan in the future.

One more thing, I recompiled ffmpeg to enable mmal. I followed this guide here : https://ubuntu-mate.community/t/tutorial-build-or-download-ffmpeg-libavcodec-with-mmal-support-hardware-acceleration-video-decoding/3565

I'm not sure this had any bearing on my ability to get this working.

A small thing: I have also made my fork load the stream in full screen as soon as you connect to your PS4

The fork and branch for my changes are here: https://github.com/charliebanks/chiaki/tree/feature/RPi-Effort

@charliebanks Hi Mate, thanks for this post! I have compiled your branch but when I launch the app, the screen doesn't render. Meanwhile on the terminal I get 2 errors: QEGLPlatformContext: Failed to create context: 3009 [E] AVCodec internal buffer is full removing frames before pushing

What am I doing wrong?

I have installed the latest Buster on RPI4 4GB.

The App connects to my PS4 and I can see that I'm connected to the PS4 remotely, however, the screen is just frozen.

Thanks for your help! Cheers.

[djmonkeyuk]

@levis1978 when you say my branch, do you mean the feature/RPi-Effort branch on my fork?

The only other thing I can think right now is if you recompiled ffmpeg.

[levis1978]

@charliebanks Update: I have recompiled everything and for some reason it now works. The problem that I have now is that when I'm on the menu, the image is perfect, but when I start a game, the image ghosts and I have no sound at all. For the sound tried your suggestion above but still didn't work. My PS4 and Pi are wired to the router, so it's not the connection. On terminal, I get the following:

Takion Send Buffer re-sending packet with seqnum 0x48ff7581, tries: 0 [W] Detected missing or corrupt frame(s) from 2049 to 2054 [D] StreamConnection reporting corrupt frame(s) from 2049 to 2053 [E] Takion dropping data with seq num 0x70331a1 [I] Frame Processor received 11+0 / 13+1 units, attempting FEC [E] FEC failed [W] Missing unit 0xb [W] Missing unit 0xc [h264 @ 0xdf3900] concealing 223 DC, 223 AC, 223 MV errors in P frame [W] Detected missing or corrupt frame(s) from 2055 to 2057 [D] StreamConnection reporting corrupt frame(s) from 2055 to 2056 [W] Takion received data with unexpected data type 0x7 ] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e bd b0 28 67 01 ab bf 80 00 01 00 .H.t...(g....... [W] 10 10 07 03 31 5c 00 00 00 00 07 00 05 0d ...1........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e 37 51 a3 06 01 59 49 f0 00 01 00 .H.t.7Q...YI.... [W] 10 10 07 03 31 5d 00 00 00 00 07 00 05 10 ...1]........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e 15 2e e4 27 01 88 26 f0 00 01 00 .H.t....'..&.... [W] 10 10 07 03 31 5e 00 00 00 00 07 00 05 12 ...1^........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e 51 35 b7 c8 01 4d ba d0 00 01 00 .H.t.Q5...M..... [W] 10 10 07 03 31 5f 00 00 00 00 07 00 05 15 ...1_........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e d3 a9 77 b4 01 5a c6 b0 00 01 00 .H.t...w..Z..... [W] 10 10 07 03 31 60 00 00 00 00 07 00 06 18 ...1`........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e 5d 85 8b 94 01 4e ae a0 00 01 00 .H.t.]....N..... [W] 10 10 07 03 31 61 00 00 00 00 07 00 07 1a ...1a........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e fa 93 86 10 01 4f 23 20 00 01 00 .H.t......O#.... [W] 10 10 07 03 31 62 00 00 00 00 07 00 08 1d ...1b........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e e7 61 b6 a8 01 5c 3e f0 00 01 00 .H.t..a...>.... [W] 10 10 07 03 31 63 00 00 00 00 07 00 08 20 ...1c........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e 0f c8 7c 8d 01 5c c6 e0 00 01 00 .H.t...|....... [W] 10 10 07 03 31 64 00 00 00 00 07 00 09 26 ...1d.......&
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e c7 e2 77 65 01 7f b5 10 00 01 00 .H.t...we....... [W] 10 10 07 03 31 66 00 00 00 00 07 00 0a 2b ...1f.......+
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e bc 30 a0 b6 01 74 c1 e0 00 01 00 .H.t..0...t..... [W] 10 10 07 03 31 67 00 00 00 00 07 00 0b 31 ...1g.......1
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e db 00 a3 b0 01 8c 5e c0 00 01 00 .H.t.......^.... [W] 10 10 07 03 31 68 00 00 00 00 07 00 0c 35 ...1h.......5
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e f3 d6 5d 05 01 81 0c c0 00 01 00 .H.t...]........ [W] 10 10 07 03 31 69 00 00 00 00 07 00 0d 2e ...1i........
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e b4 7f 6f 30 01 6b 1c c0 00 01 00 .H.t...o0.k..... [W] 10 10 07 03 31 6a 00 00 00 00 07 00 0e 27 ...1j.......'
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 48 ff 74 0e 66 e1 2a c3 01 6b 86 90 00 01 00 .H.t.f.*..k..... [W] 10 10 07 03 31 6b 00 00 00 00 07 00 0e 21 ...1k.......!
[E] Takion dropping data with seq num 0x703315d [E] Takion dropping data with seq num 0x703315e [E] Takion dropping data with seq num 0x703315f [E] Takion dropping data with seq num 0x7033160 [E] Takion dropping data with seq num 0x7033161 [E] Takion dropping data with seq num 0x7033162 [E] Takion dropping data with seq num 0x7033163 [E] Takion dropping data with seq num 0x703317f [E] Takion dropping data with seq num 0x7033190 [E] Takion dropping data with seq num 0x7033191 [E] Takion dropping data with seq num 0x7033192 [E] Takion dropping data with seq num 0x70331a3 [E] Takion dropping data with seq num 0x70331a4 [E] Takion dropping data with seq num 0x70331a5 [E] Takion dropping data with seq num 0x70331a6 [E] Takion dropping data with seq num 0x70331a7 [I] Frame Processor received 6+0 / 10+1 units, attempting FEC [E] FEC failed [W] Missing unit 0x6 [W] Missing unit 0x7 [W] Missing unit 0x8 [W] Missing unit 0x9 [h264 @ 0xdf3900] concealing 1797 DC, 1797 AC, 1797 MV errors in P frame [I] Takion Send Buffer re-sending packet with seqnum 0x48ff7584, tries: 0 [W] Detected missing or corrupt frame(s) from 2058 to 2060 [D] StreamConnection reporting corrupt frame(s) from 2058 to 2059 [E] Takion dropping data with seq num 0x7033194 [I] Frame Processor received 7+1 / 11+1 units, attempting FEC [E] FEC failed [W] Missing unit 0 [W] Missing unit 0x1 [W] Missing unit 0x2 [W] Missing unit 0x3

Any idea what this could be? Thanks & cheers!

[levis1978]

@levis1978 when you say my branch, do you mean the feature/RPi-Effort branch on my fork?

The only other thing I can think right now is if you recompiled ffmpeg.

Hi @charliebanks, yes I meant to say fork, sorry. With regards to ffmpeg, I'm not sure I need to. When compiling your fork, I get: Found FFMPEG: avcodec;avutil (found version "56.22.100") found components: avcodec avutil

Also, if I do: ffmpeg -codecs | grep h264_mmal

I get: DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_mmal ) (encoders: libx264 libx264rgb h264_omx h264_v4l2m2m h264_vaapi )

Will that suffice?

Cheers!

[djmonkeyuk]

The sound stopped working on mine too, I'll have to investigate.

We're going outside the scope of how far I got when I last looked into all this!

[djmonkeyuk]

hey @levis1978 ,

So I just ran a sudo apt-get update and sudo apt-get upgrade and the audio is working again for me.

My audio wasn't working for my entire Pi, so i feel it may be a common problem. You may have some luck following this too...

https://www.element14.com/community/community/raspberry-pi/blog/2019/09/14/asla-audio-glitch-issue-pi-4-fix

[levis1978]

Hi @charliebanks ,

My audio works fine on my RPi4, ran a video and it works and also on RetroPie.

So I tried to compile FFmpeg for RPi following this guide: https://pimylifeup.com/compiling-ffmpeg-raspberry-pi/

After this finished, I tried to compile chiaki again and when performing the "make", I've got the following: /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_mmal_core-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_mmal_util-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_mmal_vc_client-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_bcm_host-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_stdc++-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_mmal_core-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_mmal_util-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_mmal_vc_client-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_bcm_host-NOTFOUND /usr/bin/ld: cannot find -lpkgcfg_lib_FFMPEG_avcodec_stdc++-NOTFOUND collect2: error: ld returned 1 exit status make[2]: [gui/CMakeFiles/chiaki.dir/build.make:474: gui/chiaki] Error 1 make[1]: [CMakeFiles/Makefile2:442: gui/CMakeFiles/chiaki.dir/all] Error 2 make: *** [Makefile:163: all] Error 2

If I do ffmpeg -codecs | grep: ffmpeg version git-2020-07-20-ba31fdf Copyright (c) 2000-2020 the FFmpeg developers built with gcc 8 (Raspbian 8.3.0-6+rpi1) configuration: --extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/lib --extra-libs='-lpthread -lm -latomic' --arch=armel --enable-gmp --enable-gpl --enable-libaom --enable-libass --enable-libdav1d --enable-libdrm --enable-libfdk-aac --enable-libfreetype --enable-libkvazaar --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libssh --enable-libvorbis --enable-libvpx --enable-libzimg --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-mmal --enable-nonfree --enable-omx --enable-omx-rpi --enable-version3 --target-os=linux --enable-pthreads --enable-openssl --enable-hardcoded-tables libavutil 56. 55.100 / 56. 55.100 libavcodec 58. 96.100 / 58. 96.100 libavformat 58. 48.100 / 58. 48.100 libavdevice 58. 11.101 / 58. 11.101 libavfilter 7. 87.100 / 7. 87.100 libswscale 5. 8.100 / 5. 8.100 libswresample 3. 8.100 / 3. 8.100 libpostproc 55. 8.100 / 55. 8.100

Any ideas as to what I'm missing to compile Chiaki now?

Thanks and best regards.

[djmonkeyuk]

Sorry @levis1978 , I'm not really familiar with the issues you have now.

Looks like somewhere it's trying to find the version but returning "NOTFOUND" instead, but that's just a guess.

[bhasselbeck]

I would love an update on this... this sounds like an awesome effort!

[levis1978]

I would love an update on this... this sounds like an awesome effort!

Hi @bhasselbeck, I got it to work on a RPi 4 - 4GB on Buster but at 30 FPS and no audio. And still at those FPS, frames are getting corrupted.

[Fredrum]

@charliebanks Hi! I'm just trying to make this work on my Pi4. The builds finish but the streaming doesn't yet work. Would you mind, if you have time, trying to grab the update from just the other day that add support for PS software update 8.0? I just tried running your version but it crashes out and says,

[I] Connected to 192.168.1.555:9295
[I] Sending session request
[I] Reported RP-Version mismatch. ours = 9.0, server = 10.0
[E] Server RP-Version is unknown
[I] Session has quit

This might be a different issue from the 8.0 thing I'm not sure? I will try and figure this out tomorrow.

[Fredrum]

Btw would someone be able to describe what @charliebanks did to downgrade the OpenGLES requirement to 3.1? Was it just in the shaders? Where are the shaders I can't see them?

[Fredrum]

It seems like its expecting pulseaudio to receive the sound? (I found a mention of '--socket=pulseaudio') I didn't have that installed and I got the message: PulseAudioService: pa_context_connect() failed

So I tried to install pulseaudio and then this was printed in the startup shell: [I] Audio Device alsa_output.platform-bcm2835_audio.analog-mono opened with 2 channels @ 48000 Hz, buffer size 9600

But I still couldn't hear any sound. Maybe I just didn't set up pulseaudio correctly? But also it screwed up my sound generally and I had to remove it again. Maybe others are more savvy with pulseaudio. EDIT: It would be nice not to have to use PA though and just use the basic Alsa bits. It seems like maybe its the Qt side that is not great at finding the Pis audio card/device/?

[Fredrum]

I'm continuing to investigate why we get no audio on Raspberries. I have a thread here with some questions I have about how the Qt5 is working/not working. So if you know about that stuff it'd be great if you could take a look, https://www.raspberrypi.org/forums/viewtopic.php?f=33&t=288917

Btw I have made sure that Iv'e got the libqt5multimedia5-plugins package installed.

[Fredrum]

Hi @thestr4ng3r I was wondering if you might be able to take a look at this pastebin? I have added some extra printouts to your code and made the available devices human readable. I also ran gui/chiaki with export 'QT_DEBUG_PLUGINS=1' to see what loads. I have made sure that I installed the libqt5multimedia5-plugins.

If you are able to take a look, can you see if anything is missing in terms of the qt plugins? It seems to me that Qt just don't want to or cannot connect to the RPi audio device. I have run several checks to make sure that my audio is generally working. But since other Raspberry Pi users are reporting the Audio missing I'm assuming its a wider problem.

Pastebin here: https://pastebin.com/CC8Be4mx

[thestr4ng3r]

It's hard to tell from the log what is going wrong. Maybe it's just not selecting the right device? I am personally more familiar with the PulseAudio API than alsa. This looks already pretty good to me:

So I tried to install pulseaudio and then this was printed in the startup shell: [I] Audio Device alsa_output.platform-bcm2835_audio.analog-mono opened with 2 channels @ 48000 Hz, buffer size 9600

Were you able to play sound through PulseAudio from other applications like paplay after setting up the pi again?

Btw the official wip state to port to GLES is here: https://github.com/thestr4ng3r/chiaki/tree/gles

[Fredrum]

I did spend some time with PA but it didn't play the audio so I uninstalled it as it seemed to mess up my general audio setup on the pi. Maybe I just did it wrong though. But Iv'e had similar experiences with PA before where I ended up sitting for hours testing different setups. That was when trying to get bluetooth speakers to work. I'm going to continue with a purer Alsa goal for now but maybe I'l give in at some point and try PA again. It would just be nice to avoid it, for simplicity.

I'll check out your gles version!! :)

[Fredrum]

I just tested the gles branch @thestr4ng3r but it wasnt working for me. I'm getting a stream of these errors,

composeAndFlush: makeCurrent() failed composeAndFlush: makeCurrent() failed

[thestr4ng3r]

Yeah it's been a while since I last tried it. It's highly wip.

[Fredrum]

I think Iv'e got as close as I can to figuring the sound out @thestr4ng3r and I would need some more information or help for how I can get further.

I managed to build a very small Qt5 example that plays back a wav file. Its also printing out the device its using to do that.
Its the device named 'default'. Chiaki on the other hand returns a 'null' device with exactly the same command.

Now I am building my small test program against Qt5.12 and I'm wondering if that is what's causing this? Which version is Chiaki built against?

The other possibility is that you can see the in the example code when they are doing audio = new QAudioOutput(format, &a); they are passing a QCoreApplication where you pass 'this'. I haven't checked yet what your 'this' actually is. But I guess that's happening after the failing default device so it probably more likely it to do with that.

Update: I just switched Qt5 install to which I'm building to the standard already installed version by doing, export QT_SELECT=qt5-arm-linux-gnueabihf I recompiled my test program and it then behaves like current Chiaki. (ie fails to find a device)

My test program that plays a wav.

#include <QAudioOutput>
#include <QFile>
#include <QDebug>
#include <QCoreApplication>

class AudioOutputTest : public QObject
{
    Q_OBJECT
    public Q_SLOTS:
        void handleStateChanged(QAudio::State state) {
            qWarning() << "state = " << state;
            qApp->exit();
        }
};

#include "main.moc"

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    QFile sourceFile;
    QAudioOutput* audio;

    sourceFile.setFileName("/home/pi/Music/LRMonoPhase4.wav");
    sourceFile.open(QIODevice::ReadOnly);

    QAudioFormat format;
    // Set up the format, eg.
    format.setSampleRate(48000);
    format.setChannelCount(2);
    format.setSampleSize(16);
    format.setCodec("audio/pcm");
    format.setByteOrder(QAudioFormat::LittleEndian);
    format.setSampleType(QAudioFormat::SignedInt);

    QAudioDeviceInfo info(QAudioDeviceInfo::defaultOutputDevice());
    printf("Default Device:  %s\n", info.deviceName().toUtf8().constData()); // 'default'

    if (!info.isFormatSupported(format)) {
        qWarning() << "Raw audio format not supported by backend, cannot play audio.";
        return 1;
    }

    audio = new QAudioOutput(format, &a);
    AudioOutputTest audioOutputTest;
    QObject::connect(audio, SIGNAL(stateChanged(QAudio::State)), &audioOutputTest, SLOT(handleStateChanged(QAudio::State)));
    audio->start(&sourceFile);

    return a.exec();
}

[Fredrum]

Update 2:

How can I change what version of Qt5 I'm building Chiaki against. I just found out about doing ldd chiaki

And that shows me things like,

libQt5Multimedia.so.5 => /usr/lib/arm-linux-gnueabihf/libQt5Multimedia.so.5 (0xb6da5000)
libQt5OpenGL.so.5 => /usr/lib/arm-linux-gnueabihf/libQt5OpenGL.so.5 (0xb6d51000)

Ie the libraries that don't work properly. I need to build chiaki against my Qt5.12 install. I thought I changed a CMakeList file before to find_package(Qt5 5.12 REQUIRED COMPONENTS Core Widgets Gui Multimedia OpenGL Svg)

Should that work or do I need to do something else?

Update3: I had already changed the CMakeLists.txt inside chiaki/gui to find_package(Qt5 5.12 REQUIRED COMPONENTS Core Widgets Gui Multimedia OpenGL Svg)

But it doesn't make any difference. I don't see any complaints from cmake but it also doesn't change the libraries linked.

[Fredrum]

Final update for today! :) Iv'e got the sound working on the raspberry pi4 now! What I had to do was,

1. Install Qt5.12 from https://github.com/koendv/qt5-opengl-raspberrypi (please let me know if this is doable with apt?)
2. Add the following lines to chiaki/gui/CMakeLists.txt

   set(Qt5Core_DIR "/usr/lib/qt5.12/lib/cmake/Qt53DCore")
   set(Qt5_DIR "/usr/lib/qt5.12/lib/cmake/Qt5")
   set(QT_QMAKE_EXECUTABLE "/usr/lib/qt5.12/bin/qmake")

   (assuming that's where the new version got installed on your system)

NOTE: cmake will get confused because its doubling up the paths and including the old paths as well but luckily these new ones have priority. Again please if you know how to not get the doubling up please let me know!

3. Build chiaki as usual.
4. do 'ldd gui/chiaki' and look for the paths to the Qt libraries. They need to point to the 5.12 versions, Like so, libQt5Multimedia.so.5 => /usr/lib/qt5.12/lib/libQt5Multimedia.so.5 (0xb6e6a000)

Now run chiaki and you should have the sound streaming!

[Fredrum]

Iv'e started to look into hardware video decode for my RPi4. It does have that built in and the official specs say 1080p60 with h264. I have tried this and it works really well. But so far I haven't been able to get ffplay to do this and I actually believe that the rpi hw decode support is broken in ffmpeg. Online I find plenty of people who are trying to get it to work but no-one that says that its working for them. Recently that is. I think it was working a while ago but then some update/changes happened on the rpi kernel or libraries and it's now not.

I did follow a fairly recent guide on how to compile ffmpeg on the Pi to get omx/mmal hardware support and the build worked. When I list the available codecs for my own build I get:

DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_mmal ) (encoders: libx264 libx264rgb h264_omx h264_v4l2m2m h264_vaapi )

These actually match the officially apt installed ffmpeg. Some people are saying that it's the '_omx' that's needed for it to work. Unfortunately that looks off the list for decode.

This whole thing seems to be a long story but I'm going to go with the assumption that this does not work with ffmpeg currently.

I did some tests and ffplay does play 1060p h264 but with only ~15fps and then using ~45% of CPU. The MMAL/hardare videoplayer 'omxplayer' meanwhile plays the same video at full speed using only ~15% CPU. Interestingly VLC also plays at full speed and 15% CPU.

I found a project on git called 'moonlight' that has an 'embedded' version that caters for the raspberry. It has files called pi.c and mmal.c and though I haven't managed to get a working build yet (I think I'm close though) I'm going to go with that for now I think (unless someone has a better suggestion) and see if there's a way I can use what they have done to run in chiaki as a test.

[tomblind]

I found a project on git called 'moonlight' that has an 'embedded' version that caters for the raspberry. It has files called pi.c and mmal.c and though I haven't managed to get a working build yet (I think I'm close though) I'm going to go with that for now I think (unless someone has a better suggestion) and see if there's a way I can use what they have done to run in chiaki as a test.

Moonlight is amazing. It even works well on a Pi Zero. I've been wondering for awhile if that code could be adapted for Chiaki. I hope you can get that working!

[Fredrum]

Iv'e got my first success with doing proper working HW decode on my RPi4. Its running full 30fps on currently a ~50% load on the cpu. I believe I should be able to get it to do 60fps too if I learn more about how chiaki works as the pi hw decoder has no problems playing back 1080p. In fact I have an example program that does that at probably 5-10% cpu or maybe even less.

But please could you take a look at this session here and see if you can tell me what I'm doing wrong!? Every time I start I get the same pattern of broken stream. The blocks end up broken in the same way every time. But if I switch screen in the game to say the loading screen the image becomes perfect again. I also get this garbling in game with more movement. Maybe @thestr4ng3r you could take a look? Or anyone else who's familiar with h264 decoding and chiaki.

I have achieved this by hacking it into the chiaki code for now so it's entirely possible that Iv'e broken something [important].

https://www.youtube.com/watch?v=P4o4m5Puf6I&feature=youtu.be

I'm also attaching a 'ps' print log. cpulog.txt

And here are some new error messages that seemed tied to my test:

[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 2a 8e e5 b1 32 62 17 6d 01 68 ee 40 00 01 00 ....2b.m.h.@... [W] 10 10 1b b1 0b c2 00 00 00 00 07 00 32 3f ...........2?
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 2a 8e e5 b1 ad 0f 84 78 01 69 90 a0 00 01 00 .......x.i..... [W] 10 10 1b b1 0b c3 00 00 00 00 07 00 21 2a ...........!
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 2a 8e e5 b1 ce 77 67 34 01 6a 18 40 00 01 00 .....wg4.j.@... [W] 10 10 1b b1 0b c4 00 00 00 00 07 00 10 15 .............
[W] Takion received data with unexpected data type 0x7 [W] offset 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef [W] 0 00 2a 8e e5 b1 0e f0 c1 af 01 6a 9e f0 00 01 00 .*........j..... [W] 10 10 1b b1 0b c5 00 00 00 00 07 00 00 00 .............

[Fredrum]

So here's the latest. Smooth non-glitchy in-game and working sound!

https://www.youtube.com/watch?v=xsVvuWcEPC4

[Fredrum]

Just had to post this too as it's so cool. I just played a couple of minutes of Rayman. (really playable but I'd like to try to improve input latency somehow)

I did a 'top' capture for the duration and it looks like average CPU useage on the RPi4 while playing was about 15% !

cpulog_save2.txt

[Fredrum]

Hi @thestr4ng3r I'm wondering how I should move ahead to get this closer to be possible to merge back into the main project? You mentioned that you wanted to take another look at how the hardware decode paths and maybe also OpenGL variants are getting chosen? I don't know if that's something you would want to set up first or if I should try to add it using the current GUI selections?

Note for the RPis we're going to need to close down the whole current OpenGL upload+draw as that is taking precious CPU resource that is not used at all by the Pis rendering path. I'm also commenting out almost all of VideoDecoder::PushFrame

[thestr4ng3r]

You can open a draft pr with what you have no so I can review it there and suggest changes.

[Fredrum]

@thestr4ng3r sorry I'm not experienced with Git how do I add my code to a draft PR? I'm trying using the web interface but it looks like I can only do it one fork?/branch? against another?

[KurtusCobain]

Everyone’s doing a great job working together. Looks like this project is going great honestly I prefer Chiaki over PS4 remote play! My computer just broke and would love to see this work!

[Fredrum]

@thestr4ng3r I think Iv'e done it!

[Fredrum]

I think this is actually working with 720p60. (60fps) I changed in the main GUI to do 60 and then tried playing Rayman Legends. The Pi hw decode has been set to 60 as default and that seems to work fine with a 30fps source also, as well as a 60fps source. Before when I tried it was breaking with a gui setting of 60 but I think after I optimized (culled) for saving CPU cycles there's enough left to handle the extra flow. The CPU useage goes up from ~15% -> ~30%.

I can't show a video of this as I don't have a way to record the screen that wouldn't oveload the Pi when running at the same time as playing. I tried filming with the phone again but because its not in sync, of course, the result is not a very good proof.

[KurtusCobain]

I have no previous experience compiling, but im compiling the dependencies for this project directly on my raspberry pi definately a learning process. each time i make chiaki on Rpi it spits out errors, which i then hunt down a fix. once i finally get through the dependencies and actually build chiaki ill post my success or failure here, and also any anomalies which i may run into. I have a rpi 4 8gb running raspbian basically working off a fresh install here. Quick question, is this best done on 32 bit rasbian or 64 bit?

[Fredrum]

Hi @KurtusCobain The version I have done this work on is a bit older than the current Chiaki Master. I also got errors when I tried to build the main Chiaki out of the box on my RPi.

If you want you could try to build this copy that I have for testing with. https://github.com/Fredrum/test_01 You could post in the 'issues' area there too if you want. https://github.com/Fredrum/test_01/issues/1

I'd really like to get this working for you!

Btw, I'm only running 32bit so that would be the best bet for getting this to work.

[KurtusCobain]

@fredrum Yes im sorry i wasnt clear, i am using your version of the chiaki to build. I look foward to the success of this branch of chiaki. linux and gaming doesnt always go hand in hand. so these projects always fascinate me. i now also have a motivation for learning compiling. just wanna say thank you guys for all the work. ill follow your advice in posting in your "issues" link once i progress a little further.

[Fredrum]

I figured out how I could record 60fps gameplay! https://www.youtube.com/watch?v=8QH2Ag_CqSI&feature=youtu.be

[Fredrum]

Hi @KurtusCobain I have now got a version that should be solid to install! And this has been tested on completely fresh Raspbian OS installs so if you follow all the steps in the instructions it should work!

https://github.com/Fredrum/chiaki/wiki/Chiaki-for-the-Raspberry-Pi

https://youtu.be/9TIIiuy_Hxk

[KurtusCobain]

This is fantastic I kept messing up with my dependencies. It really was messing with my raspian os not a big deal since that is why I’m using a raspberry pi easy to fix a Linux os esp if I can pop in a new micro card and boom new os. I’m actually going to work on this right now got some time off work! Will post results! And video once I’m able to! May even try on a 64 bit install just to see what happens!! Thank you so much for the hard work and dedication!