search

avafinger / nanopi-m4-ubuntu-base-minimal

Nanopi M4 RK3399 base minimal image for development (mali fbdev / gbm) - Camera support
53 stars 10 forks source link
3d-mali bluetooth camera fbdev gbm kernel kodi kodi-media-center lxde mainline-linux-kernel mali nanopi-m4 opengles ov13850 ov4689 rk3399 rockchip vpu wifi

readme

nanopi-m4-ubuntu-base-minimal

Ubuntu 18.04 / 20.04 base minimal image for RK3399 with camera support (NanoPi M4 v1 - 2GB / 4 GB)

OS Image for development with the following tidbits:

OS Image

BSP Kernel 4.4

Mainline Kernel 5.x

Ubuntu 19.10 - EOAN Ermine

Status of the Mainline Kernel 5.4.y

Fixing Wifi on Mainline Kernel

Status of the Mainline Kernel 5.7.y

BSP Kernel 4.19

Mainline u-boot

Mainline Kernel 5.9

Mainline Kernel 5.10

Camera experiments with Kernel 4.19

Camera experiments with Kernel 5.10

Screenshot comparison - OV13850 / OV4689

Dual-camera with Kernel 4.19.y

Dual camera with Mainline Kernel 5.11.0

Mainline u-boot

Here we are going to fix the reboot issue i had with my u-boot Android like build. We are going to build the latest u-boot on NanoPi M4 (2GB) running from SD CARD.

Mainline Kernel 5.10

Testing and Benchmarking 5.10 RC (WiP) in Ubuntu 20.04 LTS Simple benchmarks in progress for the nanoPi M4 2G (DDR3).

SBC Dev Board tested NanoPi M4
kernel version 5.10.0-rc3
gcc version 10
display hdmi
graphical interface CLI
idle Temp ºC / freq 25 ºC / 408 Mhz
full Temp ºC / freq 55 ºC / 1.5 GHz - 2.0 GHz
RAM memory usage (avg) 97 Mbytes
i2c
spi
hdmi sound out
rt5651
spdif
Camera
Wifi
BT BCM4356A2
ethernet Gbps
sound hdmi-sound and Bluetooth
gpu panfrost
vpu
DVFS
------------------------ ------------------------------

Building Vanilla Kernel natively on NanoPi M4 with gcc 10.2.0

The kernel 5.10.0-rc3

Mainline Kernel 5.9

SBC Dev Board tested NanoPi M4
kernel version 5.9
gcc version 10
display hdmi
graphical interface CLI
idle Temp ºC / freq 25 ºC / 408 Mhz
full Temp ºC / freq 55 ºC / 1.5 GHz - 2.0 GHz
RAM memory usage (avg) 92 Mbytes
i2c yes
spi
hdmi sound out yes
rt5651 yes
spdif
Camera
Wifi yes
BT yes
ethernet Gbps
sound hdmi-sound and Bluetooth
gpu panfrost
vpu wip
DVFS yes
------------------------ ------------------------------

Benchmarks 5.7.9-rkdec

Benchmarks with u-boot mainline and the board with nice cooler, a heatsink and ceramic, just for fun.

buntu@nanopi-m4:~$ 7z b

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,6 CPUs LE)

LE
CPU Freq: - - - - - - - - -

RAM size:    1924 MB,  # CPU hardware threads:   6
RAM usage:   1323 MB,  # Benchmark threads:      6

               Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
     KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       6338   530   1164   6166  |     102348   519   1682   8728
23:       5901   501   1200   6013  |     100372   518   1677   8685
24:       6377   565   1214   6857  |      98519   518   1668   8647
25:       6299   560   1285   7193  |      97049   518   1666   8637
----------------------------------  | ------------------------------
Avr:             539   1216   6557  |              518   1673   8674
Tot:             529   1445   7616

ubuntu@nanopi-m4:~$ openssl speed sha256 aes-128-cbc aes-256-cbc rsa2048
Doing sha256 for 3s on 16 size blocks: 19452925 sha256's in 3.00s
Doing sha256 for 3s on 64 size blocks: 13864911 sha256's in 3.00s
Doing sha256 for 3s on 256 size blocks: 7367936 sha256's in 2.99s
Doing sha256 for 3s on 1024 size blocks: 2553850 sha256's in 3.00s
Doing sha256 for 3s on 8192 size blocks: 359438 sha256's in 3.00s
Doing sha256 for 3s on 16384 size blocks: 180700 sha256's in 2.99s
Doing aes-128 cbc for 3s on 16 size blocks: 14735693 aes-128 cbc's in 3.00s
Doing aes-128 cbc for 3s on 64 size blocks: 4431648 aes-128 cbc's in 3.00s
Doing aes-128 cbc for 3s on 256 size blocks: 1164991 aes-128 cbc's in 3.00s
Doing aes-128 cbc for 3s on 1024 size blocks: 296218 aes-128 cbc's in 2.99s
Doing aes-128 cbc for 3s on 8192 size blocks: 37173 aes-128 cbc's in 3.00s
Doing aes-128 cbc for 3s on 16384 size blocks: 18591 aes-128 cbc's in 3.00s
Doing aes-256 cbc for 3s on 16 size blocks: 11725626 aes-256 cbc's in 2.99s
Doing aes-256 cbc for 3s on 64 size blocks: 3385548 aes-256 cbc's in 3.00s
Doing aes-256 cbc for 3s on 256 size blocks: 879341 aes-256 cbc's in 3.00s
Doing aes-256 cbc for 3s on 1024 size blocks: 222662 aes-256 cbc's in 3.00s
Doing aes-256 cbc for 3s on 8192 size blocks: 27923 aes-256 cbc's in 2.99s
Doing aes-256 cbc for 3s on 16384 size blocks: 13956 aes-256 cbc's in 3.00s
Doing 2048 bits private rsa's for 10s: 2222 2048 bits private RSA's in 9.98s
Doing 2048 bits public rsa's for 10s: 83572 2048 bits public RSA's in 9.99s
OpenSSL 1.1.1c  28 May 2019
built on: Wed May 27 19:04:47 2020 UTC
options:bn(64,64) rc4(char) des(int) aes(partial) blowfish(ptr) 
compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-KZTQCo/openssl-1.1.1c=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128 cbc      78590.36k    94541.82k    99412.57k   101447.23k   101507.07k   101531.65k
aes-256 cbc      62745.82k    72225.02k    75037.10k    76001.96k    76503.42k    76218.37k
sha256          103748.93k   295784.77k   630833.32k   871714.13k   981505.37k   990163.48k
          sign    verify    sign/s verify/s
rsa 2048 bits 0.004491s 0.000120s    222.6   8365.6

BSP Kernel 4.19

Experiments with Kernel 4.19

https://gist.github.com/avafinger/64085a755c717db2263aaa64c9aa91c2

SBC Dev Board tested NanoPi M4
kernel version 4.19.111
gcc version 7.5.0
display hdmi
graphical interface CLI
idle Temp ºC / freq 25 ºC / 408 Mhz
full Temp ºC / freq 55 ºC / 1.4 GHz - 1.8 GHz
RAM memory usage (avg) 71 Mbytes
i2c yes
spi
hdmi sound out yes
rt5651 yes
spdif
Camera
Wifi yes (?)
BT yes
ethernet Gbps / 100Mbps
sound hdmi-sound and Bluetooth
gpu mali T860
vpu
DVFS yes
------------------------ ------------------------------

Kodi GPU (hw accel)

Kodi gpu

Kodi kernel

Mainline 5.7.y status

SBC Dev Board tested NanoPi M4
kernel version 5.7.9
gcc version 9.2.1
display hdmi
graphical interface CLI
idle Temp ºC / freq 33 ºC / 408 Mhz
full Temp ºC / freq 55 ºC / 1.20 GHz - 1.99 GHz
RAM memory usage (avg) 101 Mbytes
i2c yes
spi
hdmi sound out yes
rt5651 yes
spdif yes (need someone to test)
Camera
Wifi yes (new firmware)
BT yes
ethernet Gbps / 100Mbps
sound hdmi-sound and Bluetooth
gpu panfrost
vpu yes (common formats)
DVFS yes
------------------------ ------------------------------

(*) Problems / Issues

Kodi mainline kernel 5.7.9

You can build Kodi for mainline with current kernel using the same instructions used in the kernel 4.4 but you get limited decoding capabilities. Support basic video codecs.

To build Kodi with your own kernel version you will need:

Kodi Panfrost

Kodi Mainline

Mainline Kernel 5.7.9

This is the Image file for booting from SD CARD for NanoPi M4 v1 (2G), with pre-built Kernel 5.7.9. The Ubuntu EOAN 19.10 OS Image has the following working:

This Image is not to be used with NanoPi M4 v2 and possibly with 4GB DRAM. Image file: https://github.com/avafinger/nanopi-m4-ubuntu-base-minimal/releases/tag/v1.36

Mainline Kernel 5.7.1

Bootlog: https://gist.github.com/avafinger/54a1d590308215a37e437d1a93033c15

Support for:

Basic Benchmark

Mainline Kernel 5.7.0

https://github.com/avafinger/nanopi-m4-ubuntu-base-minimal/releases/tag/v1.35

Testing rt5651

There is support for rt5651 but the trick to make it work is described on Armbian forum. Alsamixer must set the following:

amixer set 'HPO L' on
amixer set 'HPO R' on
amixer set 'HPOVOL L' on
amixer set 'HPOVOL R' on
amixer set 'HPO MIX HPVOL' on
amixer set 'OUT MIXL DAC L1' on
amixer set 'OUT MIXR DAC R1' on
amixer set 'Stereo DAC MIXL DAC L1' on
amixer set 'Stereo DAC MIXR DAC R1' on

Playing sound on every device:

rt5651

aplay -D sysdefault:CARD=0 /usr/share/sounds/alsa/Front_Right.wav

SPDIF

aplay -D sysdefault:CARD=1 /usr/share/sounds/alsa/Front_Right.wav

hdmi-sound

aplay -D sysdefault:CARD=2 /usr/share/sounds/alsa/Front_Right.wav

Testing GPU and VPU

There is support for GPU (panfrost) and VPU (hantro and rkvdec). Most H264 (1920x1080) works smooth. You can also build Kodi.

If you output decoded video to a framebuffer:

ffmpeg -hwaccel drm -i Big_Buck_Bunny_1080_10s_30MB.mp4 -pix_fmt bgra -f fbdev /dev/fb0

There is a small penalty to be paid for converting the drm_prime format to bgra that increases CPU usage, while output to a drm_prime surface like Kodi does not.

FFmpeg used to test hw decoding: ffmpeg version 4.2.2-Matrix-Alpha1-26-g6029b361f2-Kodi

You can learn more reading Mainline VPU on Armbian forum

Testing Wifi

Wifi and BT works fine with the latest brcm firmware provided on previous image. 2.4 GHz and 5 GHz has been working without any glitch, read the previous info.

Mainline Kernel 5.7.0-rc4

https://github.com/avafinger/nanopi-m4-ubuntu-base-minimal/releases/tag/v1.34

Panfrost 3D graphics

Panfrost works on linux-image 5.4.0-rc8. It has been tested with latest Mesa 3D and kmscube.

Mesa 3D: https://gitlab.freedesktop.org/mesa/mesa Kmscube: https://gitlab.freedesktop.org/mesa/kmscube

Panfrost Monitoring

Panfrost Output:

ubuntu@nanopi-m4:~/kmscube$ sudo ./kmscube
[sudo] password for ubuntu: 
Using display 0xaaaad69887e0 with EGL version 1.4
===================================
EGL information:
  version: "1.4"
  vendor: "Mesa Project"
  client extensions: "EGL_EXT_client_extensions EGL_EXT_device_base EGL_EXT_device_enumeration EGL_EXT_device_query EGL_EXT_platform_base EGL_KHR_client_get_all_proc_addresses EGL_KHR_debug EGL_EXT_platform_wayland EGL_EXT_platform_x11 EGL_MESA_platform_gbm EGL_MESA_platform_surfaceless EGL_EXT_platform_device"
  display extensions: "EGL_ANDROID_blob_cache EGL_EXT_buffer_age EGL_EXT_image_dma_buf_import EGL_KHR_cl_event2 EGL_KHR_config_attribs EGL_KHR_create_context EGL_KHR_create_context_no_error EGL_KHR_fence_sync EGL_KHR_get_all_proc_addresses EGL_KHR_gl_colorspace EGL_KHR_gl_renderbuffer_image EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_3D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap EGL_KHR_no_config_context EGL_KHR_partial_update EGL_KHR_reusable_sync EGL_KHR_surfaceless_context EGL_EXT_pixel_format_float EGL_KHR_wait_sync EGL_MESA_configless_context EGL_MESA_drm_image EGL_MESA_image_dma_buf_export EGL_MESA_query_driver EGL_WL_bind_wayland_display "
===================================
OpenGL ES 2.x information:
  version: "OpenGL ES 2.0 Mesa 20.0.0-devel (git-02d63ee5a4)"
  shading language version: "OpenGL ES GLSL ES 1.0.16"
  vendor: "panfrost"
  renderer: "panfrost"
  extensions: "GL_EXT_blend_minmax GL_EXT_multi_draw_arrays GL_EXT_texture_format_BGRA8888 GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth24 GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_mapbuffer GL_OES_rgb8_rgba8 GL_OES_stencil8 GL_OES_texture_3D GL_OES_texture_float GL_OES_texture_float_linear GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_EGL_image GL_OES_depth_texture GL_OES_packed_depth_stencil GL_EXT_texture_type_2_10_10_10_REV GL_OES_get_program_binary GL_APPLE_texture_max_level GL_EXT_discard_framebuffer GL_EXT_read_format_bgra GL_EXT_frag_depth GL_NV_fbo_color_attachments GL_OES_EGL_image_external GL_OES_EGL_sync GL_OES_vertex_array_object GL_EXT_occlusion_query_boolean GL_EXT_texture_rg GL_EXT_unpack_subimage GL_NV_draw_buffers GL_NV_read_buffer GL_NV_read_depth GL_NV_read_depth_stencil GL_NV_read_stencil GL_EXT_draw_buffers GL_EXT_map_buffer_range GL_KHR_debug GL_KHR_texture_compression_astc_ldr GL_OES_required_internalformat GL_OES_surfaceless_context GL_EXT_separate_shader_objects GL_EXT_compressed_ETC1_RGB8_sub_texture GL_EXT_draw_elements_base_vertex GL_EXT_texture_border_clamp GL_KHR_context_flush_control GL_OES_draw_elements_base_vertex GL_OES_texture_border_clamp GL_KHR_no_error GL_KHR_texture_compression_astc_sliced_3d GL_KHR_parallel_shader_compile "
===================================
Using modifier ffffffffffffff
Modifiers failed!
Using modifier ffffffffffffff
Modifiers failed!

How to build 3D Mesa 20.0 for Panfrost

Kodi on 5.4.0

I have built Kodi with same parameters as for the Mali way, but for some reason it crashed when launched. It is known to be possible to run Kodi with Panfrost since Kernel 5.2, but something is missing on my Kodi build. No luck yet.

Mainline Kernel 5.4.y follow up

OS Image

then

sudo ./flash_sd.sh /dev/sdX (or /dev/mmcblkY) where X is a letter from b,c.. and Y is a number from 0,1..)

WARNING Find the correct device letter for your USB SD CARD or you may wipe your hdd.

Release v1.0

These are the raw files for building the OS Image with kernel 4.4.166-rockchip

Release v1.1

Get v1.1 files:

    wget $(curl -s https://api.github.com/repos/avafinger/nanopi-m4-ubuntu-base-minimal/releases | grep -oP '"browser_download_url": "\K(.*)(?=")' | grep v1.1)

Boot takes about 10 seconds or less depending on SD card in use. Tip: If you can't see anything on screen, type enter twice!

On first login:

sudo apt-get update
sudo apt-get dist-upgrade

Tip: You may need to update the file resolver.conf to reflect you network (DNS)

type in the shell: df -lh

Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p2  7.3G  582M  6.4G   9% /
devtmpfs        963M     0  963M   0% /dev
tmpfs           964M     0  964M   0% /dev/shm
tmpfs           964M   17M  947M   2% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           964M     0  964M   0% /sys/fs/cgroup
/dev/mmcblk0p1  113M   21M   84M  20% /boot
tmpfs           193M     0  193M   0% /run/user/1000

type in the shell: free -m

              total        used        free      shared  buff/cache   available
Mem:           1926          47        1798          24          80        1793
Swap:             0           0           0

Now we can start installing and/or configuring things:

Instructions and DEB packages coming soon

Remote access the board

Install ssh to be able to login remotely:

    sudo apt-get install ssh

from your PC:

    ssh ubuntu@IP
    where IP is the board IP assigned by DHCP

3D Mali (fbdev)

To be able to use OpenGL ES 2 / 3 we need to install the Mali user space lib.

Wifi

Install packages:

    sudo apt-get install crda wpasupplicant

Enable wifi editing the file /etc/network/interfaces and adding:

    allow-hotplug wlan0
    iface wlan0 inet dhcp
        wpa-ssid SID
        #psk="00012345678901234567890"
        wpa-psk afde07531d767050796db24c3d449e26a449c15b98bfaf20148970ec43242078
        dns-nameservers 8.8.8.8 8.8.4.4
        wireless-power off

Where SID os you AP name and xxxxxxxxx is your encrypted password. Generate encrypted password like so:

    wpa_passphrase SSID xxxxxxxxx

and finally boot and connect to Wifi

GBM

We have now fbdev working and with OpenGLES 3.2 (fbdev) we can go further and install GBM (Graphic Buffer Management) libraries to be able to use DRM for a non-Desktop environment (no X11/Wayland). VPU has support for DRM/GBM and not for fbdev yet.

There is a DEB package ready to install VPU support. Mali user space for GBM is needed in order to be able to use GL and/or GLES 2/3. The nice thing here is that we can have Kodi Media Center to run on NanoPi M4 / NEO / T4 or any other RK3399 if you wish.

Install the gbm libraries and VPU deb package so we can have Hardware Decoding and HW accelerated display with Mali T86x.

    sudo apt-get install libgbm-dev

Test if GBM is fully working with gbm_es2_demo (170 Kb) or kmscube (1.4 Mb) Get file with in https://github.com/avafinger/nanopi-m4-ubuntu-base-minimal/releases/tag/v1.7 

    ./gbm_es2_demo 
    using card: '/dev/dri/card0': Success
    Using display 0x557241ccf0 with EGL version 1.4
    EGL Version "1.4 Midgard-"r14p0-01rel0""
    EGL Vendor "ARM"
    EGL Extensions "EGL_KHR_partial_update EGL_KHR_image_pixmap EGL_EXT_image_dma_buf_import EGL_KHR_config_attribs EGL_KHR_image EGL_KHR_image_base EGL_KHR_fence_sync EGL_KHR_wait_sync EGL_KHR_gl_colorspace EGL_KHR_get_all_proc_addresses EGL_IMG_context_priority EGL_ARM_pixmap_multisample_discard EGL_KHR_gl_texture_2D_image EGL_KHR_gl_renderbuffer_image EGL_KHR_create_context EGL_KHR_surfaceless_context EGL_KHR_gl_texture_cubemap_image EGL_EXT_create_context_robustness EGL_KHR_cl_event2"
    FPS: 0.002751 
    FPS: 59.027754 
    FPS: 59.989142 
    FPS: 60.001338 
    FPS: 60.008244 
    FPS: 60.007122 
    FPS: 59.988592 
    FPS: 60.013970 
    FPS: 59.988422 
    FPS: 60.008008 
    FPS: 60.001338 
    FPS: 60.000630 
    FPS: 59.995440 
    FPS: 60.001102 
    FPS: 60.006768 
    FPS: 59.995967 
    FPS: 60.001397 
    FPS: 60.007831

or kmscube

    ./kmscube 
    Using display 0x55b89fca00 with EGL version 1.4
    ===================================
    EGL information:
      version: "1.4 Midgard-"r14p0-01rel0""
      vendor: "ARM"
      client extensions: "EGL_EXT_client_extensions EGL_EXT_platform_base EGL_KHR_client_get_all_proc_addresses EGL_KHR_platform_gbm EGL_MESA_platform_gbm"
      display extensions: "EGL_KHR_partial_update EGL_KHR_image_pixmap EGL_EXT_image_dma_buf_import EGL_KHR_config_attribs EGL_KHR_image EGL_KHR_image_base EGL_KHR_fence_sync EGL_KHR_wait_sync EGL_KHR_gl_colorspace EGL_KHR_get_all_proc_addresses EGL_IMG_context_priority EGL_ARM_pixmap_multisample_discard EGL_KHR_gl_texture_2D_image EGL_KHR_gl_renderbuffer_image EGL_KHR_create_context EGL_KHR_surfaceless_context EGL_KHR_gl_texture_cubemap_image EGL_EXT_create_context_robustness EGL_KHR_cl_event2"
    ===================================
    OpenGL ES 2.x information:
      version: "OpenGL ES 3.2 v1.r14p0-01rel0-git(966ed26).2e3fa7564ebca70897f04bd9fb7bc67e"
      shading language version: "OpenGL ES GLSL ES 3.20"
      vendor: "ARM"
      renderer: "Mali-T860"
      extensions: "GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_internalformat GL_OES_vertex_array_object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_OES_element_index_uint GL_EXT_shadow_samplers GL_OES_texture_compression_astc GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_texture_compression_astc_sliced_3d GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_multisampled_render_to_texture GL_OES_surfaceless_context GL_OES_texture_stencil8 GL_EXT_shader_pixel_local_storage GL_ARM_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_ARM_mali_program_binary GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OES_texture_storage_multisample_2d_array GL_OES_shader_image_atomic GL_EXT_robustness GL_EXT_draw_buffers_indexed GL_OES_draw_buffers_indexed GL_EXT_texture_border_clamp GL_OES_texture_border_clamp GL_EXT_texture_cube_map_array GL_OES_texture_cube_map_array GL_OES_sample_variables GL_OES_sample_shading GL_OES_shader_multisample_interpolation GL_EXT_shader_io_blocks GL_OES_shader_io_blocks GL_EXT_tessellation_shader GL_OES_tessellation_shader GL_EXT_primitive_bounding_box GL_OES_primitive_bounding_box GL_EXT_geometry_shader GL_OES_geometry_shader GL_ANDROID_extension_pack_es31a GL_EXT_gpu_shader5 GL_OES_gpu_shader5 GL_EXT_texture_buffer GL_OES_texture_buffer GL_EXT_copy_image GL_OES_copy_image GL_EXT_color_buffer_half_float GL_EXT_color_buffer_float GL_EXT_YUV_target GL_OVR_multiview GL_OVR_multiview2 GL_OVR_multiview_multisampled_render_to_texture GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_draw_elements_base_vertex GL_OES_draw_elements_base_vertex "
    ===================================

If the installation is Okay, you will see a spinning 3D cube.

Kodi Media Center on RK3399

Kodi has support for GBM (DRM) and can run on our NanoPi M4 setup. Kodi 18-RC5 has been compiled succesfully. Building instructions are provided here: https://github.com/xbmc/xbmc/blob/master/docs/README.Linux.md#4-build-kodi

Tips to build Kodi v18.0 RC5 on RK3399:

Splash screen Kodi 18rc5 on NanoPi M4 (RK3399) Kodi 1

Kodi 18rc5 on NanoPi M4 Kodi NanoPi M4 1

Kodi 18rc5 on NanoPi M4 with kernel 4.4.169-rk3399 Kodi NanoPi M4 2

Kodi 18rc5 screenshot 1 Kodi NanoPi M4 2

Kodi 18rc5 screenshot 2 Kodi NanoPi M4 2

Kodi 18rc5 screenshot 3 Kodi NanoPi M4 2

Kodi 18rc5 screenshot 4 Kodi NanoPi M4 2

Kodi 18rc5 screenshot 5 Kodi NanoPi M4 2

Splash screen Kodi 18b5 Kodi 1

ScreenShot 1 Kodi 1

ScreenShot 2 Kodi 1

ScreenShot 3 Kodi 1

ScreenShot 4 Kodi 1

ScreenShot 5 Kodi 1

ScreenShot 6 Kodi 1

Release v1.3

Release v1.4

Release v1.8 (OS Image for eMMC only)

Release v1.9

Release v1.10

Release v1.11

Release v1.13

Release v1.15

Release v1.18

Release v1.19

Bluetooth

We are going to set up the BT in an old fashioned way to bypass some limitations and later install a systemd service to boot with BT enabled. Kernel has been patched to hopefully fix the "timeout" issue with Bluez but still have some issues with hctool. For this we will use brcm_patchram_plus to load the firmware and turn ON the BT and have a stable BT experience.

LibreElec or not LibreElec

If you want to run Kodi automatically without the need of logging in you can do this:

Mainline Kernel on NanoPi M4

A litle follow-up on the status of Mainline kernel 5.0.0-rc2 on RK3399.

What works:

Increase oop to 2.0 / 1.5 GHz 7z b

Kodi 1

Mainline Kernel 5.0-rc2 (Ubuntu minimal 18.04)

I have provided an OS Image tested on NanoPi M4:

Download from: https://github.com/avafinger/nanopi-m4-ubuntu-base-minimal/releases/tag/v1.12

Release v1.14 (Experimental)

Release v1.16 (Experimental)

Release v1.17 (Experimental)

Release v1.20 (Experimental)

Release v1.21 (Experimental)

Release v1.22 (Experimental)

Mainline Kernel 5.1-rc6 - Release 1.23 (experimental)

Mainline Kernel 5.2.0

Mainline kernel 5.2 has been released, here is what you can expect:

Bootlog: https://gist.github.com/avafinger/97d7faea424e386219ea4193a787d3b5

DVFS DVFS

Mainline Kernel 5.6.2

Mainline kernel 5.6.2 (status update) has some improvments:

i was surprised by some nice improvments on DVFS and Wifi and BT are working together. The missing pieces are the rt5651 codec and HW decoding.

7z b

ubuntu@nanopi-m4:~$ 7z b

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=C.UTF-8,Utf16=on,HugeFiles=on,64 bits,6 CPUs LE)

LE
CPU Freq: 64000000 - - - - - - - -

RAM size:    1920 MB,  # CPU hardware threads:   6
RAM usage:   1323 MB,  # Benchmark threads:      6

               Compressing  |                  Decompressing
Dict     Speed Usage    R/U Rating  |      Speed Usage    R/U Rating
     KiB/s     %   MIPS   MIPS  |      KiB/s     %   MIPS   MIPS

22:       5049   450   1092   4912  |      98605   505   1665   8409
23:       5796   522   1131   5906  |      96652   506   1652   8363
24:       5369   497   1161   5773  |      94737   504   1649   8315
25:       5838   550   1212   6666  |      93054   507   1633   8281
----------------------------------  | ------------------------------
Avr:             505   1149   5814  |              506   1650   8342
Tot:             505   1399   7078

Bootlog: https://gist.github.com/avafinger/eb14cc0d7fde87223e3c8fa7766ac238

Build Mainline Kernel on NanoPi M4 (on board)

Requirements:

Building KODI 19

Instructions to build Kodi 19.0 ALPHA1

KODI 19.0 ALPHA1 Kodi 1

KODI 19.0 ALPHA1 - mali Kodi 2

KODI 19.0 ALPHA1 - Web Inteerface Config Kodi 3

KODI 19.0 ALPHA1 - system Kodi 4

Install dependencies if not already installed, add or change if desired:

sudo apt-get install autoconf automake autopoint gettext autotools-dev cmake curl default-jre gawk gcc g++ cpp gdc gperf     libasound2-dev libass-dev libavahi-client-dev libavahi-common-dev libbluetooth-dev libbluray-dev libbz2-dev libcdio-dev     libcec-dev libp8-platform-dev libcrossguid-dev libcurl4-openssl-dev libcwiid-dev libdbus-1-dev libegl1-mesa-dev libenca-     dev libflac-dev libfontconfig1-dev libfmt-dev libfreetype6-dev libfribidi-dev libfstrcmp-dev libgcrypt20-dev libgif-dev     libgles2-mesa-dev libglew-dev libglu1-mesa-dev libgnutls28-dev libgpg-error-dev libiso9660-dev libjpeg-dev liblcms2-dev     liblirc-dev libltdl-dev liblzo2-dev libmicrohttpd-dev libmysqlclient-dev libnfs-dev libogg-dev libomxil-bellagio-dev         libpcre3-dev libplist-dev libpng-dev libpulse-dev libsmbclient-dev libsqlite3-dev libssl-dev libtag1-dev libtiff5-dev       libtinyxml-dev libtool libudev-dev libva-dev libvdpau-dev libvorbis-dev libxkbcommon-dev libxmu-dev libxrandr-dev           libxslt1-dev libxt-dev netcat lsb-release nasm python-dev python-pil python-minimal rapidjson-dev swig unzip uuid-dev       yasm zip zlib1g-dev

Build the Kodi on-board

mkdir -p kodi
cd kodi
mkdir -p kodi-build
git clone https://github.com/xbmc/xbmc kodi
cd kodi
sudo make -C tools/depends/target/crossguid PREFIX=/usr
sudo make -C tools/depends/target/flatbuffers PREFIX=/usr
sudo make -C tools/depends/target/libfmt PREFIX=/usr
cd ../kodi-build
cmake ../kodi -DCMAKE_INSTALL_PREFIX=/usr/local -DCORE_PLATFORM_NAME=gbm -DGBM_RENDER_SYSTEM=gles
cmake --build . -- VERBOSE=0 -j4
sudo make install
cd ../kodi
sudo make -j4 -C tools/depends/target/binary-addons PREFIX=/usr/local

Testing

kodi

Pre-built Kodi 19 RK3399

Fix for memory leak and random crahes

Kernel 4.4.177 fix https://github.com/avafinger/nanopi-m4-ubuntu-base-minimal/releases/tag/v1.25

nanopi-m4-ubuntu-base-minimal 19.10 development

Deploying Ubuntu 19.10 Eoan Ermine (development branch)

Welcome to Ubuntu Eoan Ermine (development branch) (GNU/Linux 5.3.0-rc5-rk3399 aarch64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

Last login: Wed Aug 21 23:13:19 2019
ubuntu@nanopi-m4:~$

Mainline Linux Kernel 5.3.0-rc7

Bootlog: https://gist.github.com/avafinger/84aa9fac1c810765ec84844cb0da94b6

*Compiled with gcc 9.2.1

Bootlog: https://gist.github.com/avafinger/5d2e7586b4b10cb45a8a97f9d33d093b

Mainline Linux Kernel 5.3.1

Bootlog: https://gist.github.com/avafinger/28d976dbe48bc92580efba3fffc37078

Mainline Linux Kernel 5.4.0-rc4

Bootlog: https://gist.github.com/avafinger/0d3d90ad478fb3457707b9652c9710ea

7z b stress

Mainline Bluetooth

Mailine Bluetooth works out-of-the-box on 5.3.y and 5.4.y

Here is how to play music with a Bluetooth speaker like JBL CHARGE3

Build instructions

You can also monitor the health of your board with htop 2.2.2 from here:

https://github.com/avafinger/htop_2.2.2

Fix Wifi on Mainline Kernels

Until recently i was able to use Wifi or Bluetooth but not both together. This has been fixed following some simple instructions.

I used the old way (the ugly way) to set the Wifi and Ethernet, editing the file /etc/network/interfaces instead of netplan, the recommended way which i found a bit hard to configure.

For some reason, the wifi works smooth with power save off in 2.4 GHz and 5 GHz, and now Bluetooth can work together. Setting the wireless-power off in /etc/network/interfaces does not work but using iw works:

sudo apt-get update
sudo apt-get install iw
sudo iw dev wlan0 set power_save off

Now Wifi and Bluetooth work smooth.

Also for a better performance you can update the firmware from here (which is more up-to-date):

https://people.linaro.org/~manivannan.sadhasivam/rock960_wifi/

You can add the power save command at boot time adding the command to rc.local file:

#!/bin/bash
iw dev wlan0 set power_save off
exit 0

You might need to add the rc.local service if you have not done it yet before you edit the rc.local file above;

sudo systemctl status rc-local
sudo systemctl enable rc-local
printf '%s\n' '#!/bin/bash' 'exit 0' | sudo tee -a /etc/rc.local
sudo chmod +x /etc/rc.local
sudo systemctl enable rc-local

Check for SSID(s):

sudo iw dev wlan0 scan | egrep "signal:|SSID:" | sed -e "s/\tsignal: //" -e "s/\tSSID: //" | awk '{ORS = (NR % 2 == 0)? "\n" : " "; print}' | sort

Enjoy the maximum performance of both Wifi and BT.

MIPI Camera

Experiments with MIPI camera in Kernel 4.19. In these experiments i will try to grab images from the cameras with the kernel 4.19.111

MIPI Camera Mainline

There is a porting of the OV13850 driver to the mainline kernel, so I decided to take a look and see if we can pull the raw image from the sensor using mjpg-streamer on NanoPi M4. For one reason, i used Kernel v5.10.0-rc1 to test its work since my kernel v5.10.0, which has some of my hacks just failed to load the ISP.

Now we have our character device as i expected to have it in kernel 4.19:

ubuntu@nanopi-m4:~$ cat /sys/dev/char/81:131/device/name
ov13850

Doing some check to find out about the sensor and rkisp1:

ubuntu@nanopi-m4:~$ /usr/local/bin/media-ctl -d /dev/media0 -p|grep enti
- entity 1: rkisp1_isp (4 pads, 5 links)
- entity 6: rkisp1_resizer_mainpath (2 pads, 2 links)
- entity 9: rkisp1_resizer_selfpath (2 pads, 2 links)
- entity 12: rkisp1_mainpath (1 pad, 1 link)
- entity 16: rkisp1_selfpath (1 pad, 1 link)
- entity 20: rkisp1_stats (1 pad, 1 link)

Configuring the sensor:

/usr/local/bin/media-ctl --device "platform:rkisp1" --reset
# connect pad 0 (sink) of the ISP with pad 0 of the camera and enable the link
/usr/local/bin/media-ctl --device "platform:rkisp1" --links "'ov13850 1-0010':0 -> 'rkisp1_isp':0 [1]"
# create a link between the selfpath (preview) and the ISP output on pad 2 (source), but keep it deactivated
/usr/local/bin/media-ctl --device "platform:rkisp1" --links "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [0]"
# create a link between the mainpath and the ISP output on pad 2 (source) and enable the link
/usr/local/bin/media-ctl --device "platform:rkisp1" --links "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [1]"

# Set the video format on the camera (this is very dependent on your camera)
/usr/local/bin/media-ctl --device "platform:rkisp1" --set-v4l2 '"ov13850 1-0010":0 [fmt:SBGGR10_1X10/2112x1568]'
# Set the input video format for the ISP, this must match the video format of the camera, crop it down to 1920x1080
/usr/local/bin/media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_isp":0 [fmt:SBGGR10_1X10/2112x1568 crop: (0,0)/1920x1080]'
# Set the output video format of the ISP, the maximum size was propagated from the sink pad, and the format size is taken from the crop format
/usr/local/bin/media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_isp":2 [fmt:YUYV8_2X8/1920x1080 crop: (0,0)/1920x1080]'

# Set the input format for the mainpath resizer
/usr/local/bin/media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_resizer_mainpath":0 [fmt:YUYV8_2X8/1920x1080]'
# Set the output format for the mainpath resizer
/usr/local/bin/media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/1280x720]'

# Configure the format at the mainpath DMA-engine, which is the point that is accessed by user-space
/usr/local/bin/v4l2-ctl --media-bus-info "platform:rkisp1" --device "rkisp1_mainpath" --set-fmt-video "width=1280,height=720,pixelformat=422P"

Check if this is working:

ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/video1 -vwidth=1280,height=720,pixelformat=YUYV --stream-mmap --stream-skip=3
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<^C
ubuntu@nanopi-m4:~$ 
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/v4l-subdev3 --all

User Controls

               exposure 0x00980911 (int)    : min=4 max=1660 step=1 default=1536 value=1536

Image Source Controls

          vertical_blanking 0x009e0901 (int)    : min=96 max=31199 step=1 default=96 value=96
        horizontal_blanking 0x009e0902 (int)    : min=2688 max=2688 step=1 default=2688 value=2688 flags=read-only
          analogue_gain 0x009e0903 (int)    : min=16 max=248 step=1 default=16 value=16

Image Processing Controls

         link_frequency 0x009f0901 (intmenu): min=0 max=0 default=0 value=0 (300000000 0x11e1a300) flags=read-only
                0: 300000000 (0x11e1a300)
             pixel_rate 0x009f0902 (int64)  : min=0 max=120000000 step=1 default=120000000 value=120000000 flags=read-only
           test_pattern 0x009f0903 (menu)   : min=0 max=4 default=0 value=0 (Disabled)
                0: Disabled
                1: Vertical Color Bar Type 1
                2: Vertical Color Bar Type 2
                3: Vertical Color Bar Type 3
                4: Vertical Color Bar Type 4
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/v4l-subdev2 --all
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.10.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.10.0
Interface Info:
    ID               : 0x03000030
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000009 (9)
    Name             : rkisp1_resizer_selfpath
    Function         : Video Scaler
    Pad 0x0100000a   : 0: Sink, Must Connect
      Link 0x02000024: from remote pad 0x1000004 of entity 'rkisp1_isp': Data
    Pad 0x0100000b   : 1: Source, Must Connect
      Link 0x02000026: to remote pad 0x1000013 of entity 'rkisp1_selfpath': Data, Enabled, Immutable
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/v4l-subdev1 --all
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.10.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.10.0
Interface Info:
    ID               : 0x0300002e
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000006 (6)
    Name             : rkisp1_resizer_mainpath
    Function         : Video Scaler
    Pad 0x01000007   : 0: Sink, Must Connect
      Link 0x02000020: from remote pad 0x1000004 of entity 'rkisp1_isp': Data, Enabled
    Pad 0x01000008   : 1: Source, Must Connect
      Link 0x02000022: to remote pad 0x100000f of entity 'rkisp1_mainpath': Data, Enabled, Immutable
ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl -d /dev/v4l-subdev0 --all
Media Driver Info:
    Driver name      : rkisp1
    Model            : rkisp1
    Serial           : 
    Bus info         : platform:rkisp1
    Media version    : 5.10.0
    Hardware revision: 0x00000000 (0)
    Driver version   : 5.10.0
Interface Info:
    ID               : 0x0300002c
    Type             : V4L Sub-Device
Entity Info:
    ID               : 0x00000001 (1)
    Name             : rkisp1_isp
    Function         : Video Pixel Formatter
    Pad 0x01000002   : 0: Sink, Must Connect
      Link 0x0200001e: from remote pad 0x100001d of entity 'ov13850 1-0010': Data, Enabled
    Pad 0x01000003   : 1: Sink
      Link 0x02000028: from remote pad 0x1000019 of entity 'rkisp1_params': Data, Enabled, Immutable
    Pad 0x01000004   : 2: Source
      Link 0x02000020: to remote pad 0x1000007 of entity 'rkisp1_resizer_mainpath': Data, Enabled
      Link 0x02000024: to remote pad 0x100000a of entity 'rkisp1_resizer_selfpath': Data
    Pad 0x01000005   : 3: Source
      Link 0x0200002a: to remote pad 0x1000015 of entity 'rkisp1_stats': Data, Enabled, Immutable
ubuntu@nanopi-m4:~$

Testing the performance again:

ubuntu@nanopi-m4:~$ /usr/local/bin/v4l2-ctl --media-bus-info "platform:rkisp1" --device "rkisp1_mainpath" --stream-mmap --stream-count=150
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.05 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Media order keeps changing on every boot, here you can find the sensor name and the corresponding /dev/videoX:

ubuntu@nanopi-m4:~$ ls /dev/media*
/dev/media0  /dev/media1  /dev/media2  /dev/media3

Finding the sensor in use:

ubuntu@nanopi-m4:~$ media-ctl -p -d 1 |grep ENABLED |grep -v "rkisp"|grep -o "\".*\""|tr -d \"
ov4689 1-0036

where -d 1 is the media index, searching from media0,media1,...media3

Finding the video device for the sensor:

ubuntu@nanopi-m4:~$ media-ctl -e "rkisp1_mainpath" -d 1
/dev/video2

Build mjpg-streamer from source and run:

mjpg_streamer -i "./input_uvc.so -y -r 1280x720 -d /dev/video1" -o "./output_http.so -w ./www"
MJPG Streamer Version: git rev: 58e952383cbe973641a3ce6c6a738bafc1605337
 i: Using V4L2 device.: /dev/video1
 i: Desired Resolution: 1280 x 720
 i: Frames Per Second.: -1
 i: Format............: YUYV
 i: JPEG Quality......: 80
 i: TV-Norm...........: DEFAULT
Using mplane plugin for capture 
UVCIOC_CTRL_ADD - Error at Pan (relative): Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Tilt (relative): Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Pan Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Tilt Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Pan/tilt Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Focus (absolute): Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Pan (relative): Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Tilt (relative): Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Pan Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Tilt Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Pan/tilt Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Focus (absolute): Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at LED1 Mode: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at LED1 Frequency: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Disable video processing: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Raw bits per pixel: Inappropriate ioctl for device (25)
 o: www-folder-path......: ./www/
 o: HTTP TCP port........: 8080
 o: HTTP Listen Address..: (null)
 o: username:password....: disabled
 o: commands.............: enabled

As we can see by the errors, we still need a way to use ISP while grabbing the frames, they are dark and greenish:

mjpg-streamer pulling raw images - 1280x720 ~29 FPS (80% quality) streaming

Monitoring mjpg-streamer in action mjpg-streamer

Mainline 5.11.0-rc2

With the help of Sebastian (the author), it was possible to make use of ISP. Grabbing images still greenish but far better quality than pulling raw images directly from the sensor. Unfortunately Mjpg-streamer dropped from 29 FPS to 15 FPS and CPU usage seems a bit higher.

mjpg-streamer 1280x720 ~15 FPS (80% quality) streaming 15 FPS

Monitoring traffic and CPU usage mjpg-streamer isp

Streaming 1920x1080 with Mainline

Experimenting with OV13850 in low light condition with window size 1920x1080 10 FPS.

/usr/local/bin/media-ctl --device "platform:rkisp1" --set-v4l2 '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/1920x1080]'
/usr/local/bin/v4l2-ctl --media-bus-info "platform:rkisp1" --device "rkisp1_mainpath" --set-fmt-video "width=1920,height=1080,pixelformat=YUYV"

Mjpg-streamer command line:

mjpg_streamer -i "./input_uvc.so -y -r 1920x1080 -d /dev/video1" -o "./output_http.so -w ./www"

Mjpg-streamer 1920x1080 OV13850 mjpg-streamer isp 1920x1080

Mjpg-streamer 1920x1080 OV4689 (IR cut - Indirect light) mjpg-streamer isp 1920x1080

Streaming video with Mjpg-streamer 2688x1520 gives 5 FPS over Gbit ethernet.

mjpg_streamer -i "./input_uvc.so -y -r 2688x1520 -d /dev/video2" -o "./output_http.so -w ./www"
MJPG Streamer Version: git rev: 58e952383cbe973641a3ce6c6a738bafc1605337
 i: Using V4L2 device.: /dev/video2
 i: Desired Resolution: 2688 x 1520
 i: Frames Per Second.: -1
 i: Format............: YUYV
 i: JPEG Quality......: 80
 i: TV-Norm...........: DEFAULT
Using mplane plugin for capture 
UVCIOC_CTRL_ADD - Error at Pan (relative): Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Tilt (relative): Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Pan Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Tilt Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Pan/tilt Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_ADD - Error at Focus (absolute): Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Pan (relative): Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Tilt (relative): Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Pan Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Tilt Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Pan/tilt Reset: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Focus (absolute): Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at LED1 Mode: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at LED1 Frequency: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Disable video processing: Inappropriate ioctl for device (25)
UVCIOC_CTRL_MAP - Error at Raw bits per pixel: Inappropriate ioctl for device (25)
 o: www-folder-path......: ./www/
 o: HTTP TCP port........: 8080
 o: HTTP Listen Address..: (null)
 o: username:password....: disabled

mjpg-streamer isp 2688x1520

MIPI Dual Camera

Initial support for dual-camera has been done in Kernel 4.19.111 and the Kernel is made available here: https://github.com/avafinger/kernel-rockchip-dual-camera

You can attach two cameras (OV13850 or OV4689) to the NanoPi M4 and stream video.

MIPI Dual Camera Mainline

Initial support for dual-camera has been introduced in Kernel 5.11.0 with pre-built camera application

The setup must follow one of this:

Limitations:

The camera sensor tested with ov4689 is the camera w/o IR cut filter

Pulling video from the cameras

8GB SD CARD Image https://github.com/avafinger/nanopi-m4-ubuntu-base-minimal/releases/tag/1.42

Mainline Porting

You can get further info about the porting:

Driver:

https://sebastianfricke.me/porting-the-ov13850-camera/

Obs: Porting OV4689 is very similar to the OV13850.

rkisp1 (Helen Koike):

https://www.collabora.com/news-and-blog/news-and-events/kernel-5.10-rockchip-h264-bifrost-more.html

MIPI Camera Experiments

Here we can compare some photos taken with FE Kernel 4.4.179, Kernel 4.19.111 and mainline 5.11.0-rc2 (OV13850). I have the impression that in BSP, the image is taken at a higher resolution and then resized to fit 1920x1080 (16:9). It looks a little distorted. You can draw your conclusions.

1 - Photo in 1920x1080 size This is the image taken with cam application from FE 4.4.179. Real Image 1920x1080

2 - Screenshot mjpg-streamer with FE 4.4.179 This is a screenshot of mjpg-streamer while streaming video in 1920x1080 taken from FE 4.4.179. mjpg-streamer FE 1920x1080

3 - Screenshot YU12 1920x1080 with KFE Kernel 4.19.111 This is a screenshot of YU12 1920x1080 grabed image with Kernel 4.19.11, better focus but no Gain / Exposure controls. YU12 1920x1080

4 - Screenshot mjpg-streamer with Mainline 5.11.0-rc2 This is a screenshot of mjpg-streamer while streaming video in 1920x1080 taken from the mainline kernel 5.11.0-rc2. mjpg-streamer 1920x1080

OV4689 w/o IR cut filter with mainline kernel 5.11.0-rc6

This will configure the sensor OV4689 but grabbed images will appear green and dark, so we need a trick to bypass this limitation, run the cam application to grab a few images and adjust exposure automatically for our v4l2 application and then repeat the previous config.

cam --camera=1 --capture=15 --file=/dev/null -s height=1520,width=2688,pixelformat=NV21

Now we can run mjpg streamer and stream video to our web clients. Build the latest mjpg streamer from source, install and run like:

mjpg_streamer -i "input_uvc.so -y -r 1920x1080 -d /dev/video1" -o "output_http.so -w /usr/local/share/mjpg-streamer/www"

ov4689 1920x1080

Credits