Resources to help to port Kha to Raspberry PI

RafaelOliveira commented 9 years ago

The lastest haxe build available for the PI is the 3.1.3. There is a message about a possibility to make a new build around december: https://groups.google.com/d/msg/haxelang/YX1mFgZ5rDU/nQA5Q2g6CQAJ

Topic about the OpenFL port: http://community.openfl.org/t/raspberry-pi-2-anyone/1888

Some highlights from the topic:

the Raspberry Pi falls back to a software OpenGL implementation if use OpenGL code (it need to be OpenGL ES)
it seems there is a rpi-specific videodriver to run the games/apps in console without X11. There is some demos that run video and drawing without X11.
X11 doesn't have hardware acceleration with EGL/GLES (for now). It uses software-gl.

Examples with code showing how to do drawing, 3d, video and others things: https://github.com/raspberrypi/firmware/tree/master/opt/vc/src/hello_pi

run ./rebuild.sh to compile everything

maconbot commented 8 years ago

Hopefully this helps: Using OpenGL ES 2.0 on the Raspberry Pi without X windows. https://benosteen.wordpress.com/2012/04/27/using-opengl-es-2-0-on-the-raspberry-pi-without-x-windows/

First step: setting up a connection to the hardware ‘pipe’, a connection to VCHIQ in your C source code. import "bcm_host.h" ... // early on, perhaps at the start of your 'main.c': bcm_host_init(); The bcm_host.h file is within the ‘/opt/vc/include‘ directory on the Pi, so you will need to add that to your Makefile. The correct EGL libraries are also in a similar location ( ‘/opt/vc/lib‘) so be sure to add that into your linker options or LD path too. Next, create a display surface that you can use to render the EGL surface onto. This is the key setup to make! If the settings here are off then your might find that your EGL code will run, and you’ll get console output, but you won’t get any 3D visuals! static EGL_DISPMANX_WINDOW_T nativewindow;

DISPMANX_ELEMENT_HANDLE_T dispman_element; DISPMANX_DISPLAY_HANDLE_T dispman_display; DISPMANX_UPDATE_HANDLE_T dispman_update; VC_RECT_T dst_rect; VC_RECT_T src_rect;

int display_width; int display_height;

// create an EGL window surface, passing context width/height success = graphics_get_display_size(0 /* LCD */, &display_width, &display_height); if ( success < 0 ) { return EGL_FALSE; }

// You can hardcode the resolution here: display_width = 640; display_height = 480;

dst_rect.x = 0; dst_rect.y = 0; dst_rect.width = display_width; dst_rect.height = display_height;

src_rect.x = 0; src_rect.y = 0; src_rect.width = display_width << 16; src_rect.height = display_height << 16;

dispman_display = vc_dispmanx_display_open( 0 /* LCD */); dispman_update = vc_dispmanx_update_start( 0 );

dispman_element = vc_dispmanx_element_add ( dispman_update, dispman_display, 0/layer/, &dst_rect, 0/src/, &src_rect, DISPMANX_PROTECTION_NONE, 0 /alpha/, 0/clamp/, 0/transform/);

nativewindow.element = dispman_element; nativewindow.width = display_width; nativewindow.height = display_height; vc_dispmanx_update_submit_sync( dispman_update );

... // Pass the window to the display that have been created // to the esContext: esContext->hWnd = &nativewindow; The vc, dispman, and other unusual symbols are taken from the bcm_host.h header so make sure that that is included in the source file that you use to create the context. Final tasks: alter eglGetDisplay and eglCreateWindowSurface initialisation calls. Instead of casting an X display or something similar in eglGetDisplay, you can simply do the following: display = eglGetDisplay(EGL_DEFAULT_DISPLAY); if ( display == EGL_NO_DISPLAY ) { return EGL_FALSE; } The next change is to eglCreateWindowSurface. Remember the esContext->hWnd pointer we set earlier? We’ll use that now: surface = eglCreateWindowSurface(display, config, (EGLNativeWindowType)hWnd, NULL); if ( surface == EGL_NO_SURFACE ) { return EGL_FALSE; }

... // 'context' is returned by a "eglCreateContext" call

// Make the context current if ( !eglMakeCurrent(display, surface, surface, context) ) { return EGL_FALSE; } That seems to be all you need to access and display OpenGL ES visuals directly on the Raspberry Pi without using X. Why some of the steps are the way they are I can only guess, but this at least works for me! A port of the OpenGL ES 2.0 Programming guide examples https://github.com/benosteen/opengles-book-samples/tree/master/Raspi Most of the key changes are in the Common/esUtil.c file where the display surface and so on are set up. This is currently hardcoded to be a 640×480 resolution window, that will appear at the bottom-left of the screen. To build this on your Raspberry Pi: Get the source files onto your Pi. Either: wget --no-check-certificate https://github.com/benosteen/opengles-book-samples/tarball/master; tar -xvzf master Or, sudo apt-get install git-core; git clone git://github.com/benosteen/opengles-book-samples.git Then: pi@raspberrypi:~$ cd opengles-book-samples/

pi@raspberrypi:~/opengles-book-samples$ cd Raspi/

pi@raspberrypi:~/opengles-book-samples/Raspi$ make

gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_2/Hello_Triangle/Hello_Triangle.c -o Chapter_2/Hello_Triangle/CH02_HelloTriangle -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_8/Simple_VertexShader/Simple_VertexShader.c -o ./Chapter_8/Simple_VertexShader/CH08_SimpleVertexShader -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_9/Simple_Texture2D/Simple_Texture2D.c -o ./Chapter_9/Simple_Texture2D/CH09_SimpleTexture2D -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_9/MipMap2D/MipMap2D.c -o ./Chapter_9/MipMap2D/CH09_MipMap2D -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_9/Simple_TextureCubemap/Simple_TextureCubemap.c -o ./Chapter_9/Simple_TextureCubemap/CH09_TextureCubemap -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_9/TextureWrap/TextureWrap.c -o ./Chapter_9/TextureWrap/CH09_TextureWrap -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_10/MultiTexture/MultiTexture.c -o ./Chapter_10/MultiTexture/CH10_MultiTexture -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_11/Multisample/Multisample.c -o ./Chapter_11/Multisample/CH11_Multisample -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_11/Stencil_Test/Stencil_Test.c -o ./Chapter_11/Stencil_Test/CH11_Stencil_Test -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib gcc -DRPI_NO_X ./Common/esShader.c ./Common/esTransform.c ./Common/esShapes.c ./Common/esUtil.c ./Chapter_13/ParticleSystem/ParticleSystem.c -o ./Chapter_13/ParticleSystem/CH13_ParticleSystem -I./Common -I/opt/vc/include -lGLESv2 -lEGL -lm -lbcm_host -L/opt/vc/lib Now, to test it: pi@raspberrypi:~/opengles-book-samples/Raspi$ cd Chapter_2/Hello_Triangle pi@raspberrypi:~/opengles-book-samples/Raspi/Chapter_2/Hello_Triangle$ ./CH02_HelloTriangle 123 frames rendered in 2.0036 seconds -> FPS=61.3894 NB As no X server is involved, you can actually invoke the examples from an ssh connection and have them render out to whatever video device is connected to the Pi :)

RafaelOliveira commented 8 years ago

great, thanks for all the information! I will look everything. Yesterday I compiled haxe 3.2 to the PI with the help of some people in the OpenFL forum (starting here http://community.openfl.org/t/raspberry-pi-2-anyone/1888/127). I found these examples too: https://github.com/peepo/openGL-RPi-tutorial

RafaelOliveira commented 8 years ago

https://github.com/AndrewFromMelbourne/raspidmx https://jan.newmarch.name/RPi/ (complete free ebook about programming with the gpu)

maconbot commented 8 years ago

Glad to see you updating this...here is mine: http://www.opentk.com/node/4024

...note not fully implemented but a start and proof of concept...basically provides 3 functions Initialise(), SwapBuffers() and Cleanup()

Be sure to note the 2 updates: I'm going to modify the ES2Generator in opentk to create static bindings instead which solves the problem (I handcoded a few to check). The generator has options to generate static bindings, static+dynamic bindings (core/extensions) and fully dynamic bindings. I don't remember the exact switch, but we use that for mobile platforms.

From Stack Overflow: I have this working but it took a bit of work. First I created a c library to call through to the native libraries for controlling the gl chipset on the PI and exposed these so that I could call them from C#.

Secondly I had to recompile the opentk with the switch to not use dynamic bindings. In the end I stripped out most of the game window logic and just used the bindings and the maths code from opentk.

More details and code here:

http://www.opentk.com/node/4024

Basically this is enough to get you clearing / swapping the screen etc and you can then add the binding code through to the GL ES 2.0 spec which you can generate in the opentk project / solution tools.

RafaelOliveira commented 8 years ago

Robert did a livestream where he implemented the window initialization and graphics on the Pi. https://www.twitch.tv/robdangerous/v/54008949 (the video has 4h of duration) the code is mostly here https://github.com/KTXSoftware/Kore/blob/master/Backends/Pi/Sources/Kore/System.cpp but the engine was updated to support the new target. The biggest problem is that he tried (and me too) to implement a window without fullscreen, but the opengl is failing if we pass a size smaller than the screen size. But I compiled OpenFrameworks on the Pi, and there is a example where the window is not fullscreen, and you can switch to fullscreen.

He will make another live stream to implement Input, probably he will announce in his twitter https://twitter.com/robdangerous

RafaelOliveira commented 8 years ago

I found this library that implements input using evdev. The library is very small, the code could be copied (if permited). I tried to compile the example but the makefile is missing something. But the code could be adapted. https://github.com/progschj/pigu

gepatto commented 8 years ago

Hey Guys, I have been working on raspberry pi support for OpenFL. Basically I managed to get SDL working specifically for the rpi. I know Kore is not using SDL (yet) but It will probably need to use the same libraries. So you might want to have a look at my lime- sdl-submodule fork

In this fork I tweaked some code the have:

a functioning MousePointer: basically an extra dispmanx element
keyboard support via udev and evdev
raspberry-pi touchscreen multitouch support via evdev and udev.

So if it would help you in anyway for Kha that would be great.

I'm not a hardcore c++ dev (yet). I just tweak things until they seem to work.

RobDangerous commented 8 years ago

OK, it's evdev/udev for input then. And OpenMAX for audio I guess. Any new insights about window mode support?

gepatto commented 8 years ago

SDL is using alsa for audio. Maybe that's less work than openmax?

On the raspberry pi opengl(es) in x11 is only possible with experimental drivers at the moment and there's a lot that's not working (it's killing some parts of openmax and doesn't work for me on the raspberry pi touch screen at all). So for now I think sticking to dispmanx is the safest option since there is no E.T.A on the final driver.

By the way, MinecraftPi is a hack: It's an x11 window underneath a dispmanx element.

So you can have a non-fullscreen window-surface if you set the correct settings for the dispmanx element.

here's a good resource I'll create a small demo, but in short it's just setting the dst_rect

// pseudo code modified  from /opt/vc/src/hello_pi/hello_triangle/hello_traingle.c  
 uint32_t width = 640;
 uint32_t height = 480;
 VC_DISPMANX_ALPHA_T dispman_alpha;
 VC_RECT_T src_rect;
 VC_RECT_T dst_rect;
 dst_rect.x = 0;
 dst_rect.y = 0;
 dst_rect.width = width;
 dst_rect.height = height;

 src_rect.x = 0; src_rect.y = 0;
 src_rect.width = state->screen_width << 16; // not sure why there multplying the src_rect
 src_rect.height = state->screen_height << 16; 

// semitransparent layer
   dispman_alpha.flags = DISPMANX_FLAGS_ALPHA_FROM_SOURCE || DISPMANX_FLAGS_ALPHA_FIXED_NON_ZERO;
   dispman_alpha.opacity = 175;
   dispman_alpha.mask = 0;

dispman_element = vc_dispmanx_element_add ( dispman_update, dispman_display,
      0/*layer*/, &dst_rect, 0/*src*/,
      &src_rect, DISPMANX_PROTECTION_NONE, &dispman_alpha, 0/*clamp*/, 0/*transform*/);

  nativewindow.element = dispman_element;
  nativewindow.width  = width;
  nativewindow.height = height;

//etc etc

RafaelOliveira commented 8 years ago

@RobDangerous the keycodes for the keyboard are in linux/input.h (as #define).

RobDangerous commented 8 years ago

Ah, thanks :)

RafaelOliveira commented 8 years ago

Rendering video in OpenGL with OpenMAX https://jan.newmarch.name/RPi/OpenMAX/EGL/

this is part of a online ebook https://jan.newmarch.name/RPi/index.html

RafaelOliveira commented 7 years ago

@gepatto Could you show in your SDL fork where the input is implemented? Keyboard is implemented in Kha, the code was ported from this library https://github.com/progschj/pigu. Now I'm trying to port mouse, but I'd like to see other implementations.

gepatto commented 7 years ago

@RafaelOliveira, my SDL fork isn't really relevant anymore, everything has now been implemented in lime's SDL https://github.com/native-toolkit/sdl

Input devices are scanned with UDEV mouse is detected here: https://github.com/native-toolkit/sdl/blob/master/src/core/linux/SDL_udev.c#L348

And input events are handled with EVDEV starting here: https://github.com/native-toolkit/sdl/blob/master/src/core/linux/SDL_evdev.c#L422

The mouse is drawn here: https://github.com/native-toolkit/sdl/blob/master/src/video/raspberry/SDL_rpimouse.c

Is this enough info?

mateli commented 5 years ago

@RafaelOliveira I added the separate issues #1055 #1056 which is the only reliable way to play or record video on Raspberry pi. The other option is software playback but the CPU is to weak for this.

@gepatto ALSA audio playback without OpenMAX requires software decoding of compressed audio. While the RPi can handle that we also probably want CPU capacity free for game logic. On a PC it makes sense to decode audio in software since the CPU are usually so powerful that the load on the CPU is insignificant, but on a raspberry pi it may impact performance. Especially on Raspberry Pi Zero that only has one CPU core.

Kode / Kha

Resources to help to port Kha to Raspberry PI #188