search

OpenVoiceOS / VocalFusionDriver

Raspberry Pi VocalFusion linux driver for kernel 5.10
4 stars 5 forks source link

Modernize the XMOS VocalFusion 3510-INT linux driver #4

Open j1nx opened 7 months ago

j1nx commented 7 months ago

The XVF 3510-INT as used within the SJ-201 Mark2 HAT is based on a rather old i2c audio driver; https://github.com/PaulCreaser/rpi-i2s-audio

The below repository is used for (the in my oppinion hacky way of implementing a driver) their board/HAT. https://github.com/xmos/vocalfusion-rpi-setup

If you go through their code you will discover the following aspects of the XVF3510-INT board.

1) It uses a slightly modified my_loader.c driver and Makefile system to support both the RPI3 and RPI4 and configured them both as master and as slave (clock driver) 2) It only supports 48 kHz because of a fixed I2S BCLK of 3072 kHz (BCM 18) 3) To get the above done, it needs a MCLK set at 12288 kHz (4 times division) and it want that at BCM4 4) The board does not have an EEPROM/ROM to keep the firmware saved, so it needs to get the firmware at power up of the board. 5) The board has a Reset and Power GPIO pin to get it started. Both need to be set to output direction and pulled up to reset the board and then power it up.

Above point 1 is obviously done at kernel space being it a kernel driver, however for point 2 - 5 they decided to do that in userspace? To do that, they again used a rather old library called pigpio; https://github.com/xmos/vocalfusion-rpi-setup/blob/master/resources/clk_dac_setup/setup_mclk_bclk.c

What I did back then is moving points 2, 3 and 5 towards the kernel driver (point 4 was still on the todo). Below some old bookmarks I saved back then;

The original my_loader.c https://github.com/PaulCreaser/rpi-i2s-audio/blob/master/my_loader.c

Documentation about the different Raspberry PI clocks https://www.raspberrypi.com/documentation/computers/configuration.html#clock-configuration

How to configure those RPI clocks in a kernel / dtoverlay way https://forums.raspberrypi.com/viewtopic.php?f=107&t=136988

https://forums.raspberrypi.com/viewtopic.php?f=107&t=193550

As all of the above is very old or at least based on very old stuff, below some pointers I had saved about how it should be done with the newer kernels and associated code.

Following the below forum thread on the raspberry pi forums will reveal that the old rpi-i2s-audio driver is no longer needed. https://forums.raspberrypi.com/viewtopic.php?t=8496

With the below Pull Request merged https://github.com/raspberrypi/linux/pull/2104

We can now use the mainline simple card bindings / driver https://github.com/raspberrypi/linux/blob/rpi-6.6.y/Documentation/devicetree/bindings/sound/simple-card.yaml

At page 41 of the above forum thread their are two links pointing to a typical master I2S sound TDM driver and a typical slave I2S sound TDM driver. As we do not have a clock on the HAT we need to use the RPI clocks and therefor the I2S is master in our case; https://github.com/AkiyukiOkayasu/RaspberryPi_I2S_Master/blob/master/i2smaster.dts

I am not really sure, how to do all of the above but I guess together we can figure things out.

Below is a DTS code snippet I had still saved. Updated it a little bit, however not finished but culd be a start or at least a pointer to get started.

/*
 *Device Tree overlay for SJ-201 Rev6&10 - XMOS VocalFusion 3510-INT
 *Compile with: dtc -@ -H epapr -O dtb -o sj201.dtbo -Wno-unit_address_vs_reg sj201.dts
 */

/dts-v1/;
/plugin/;

/ {
    compatible = "brcm,bcm2835";

    fragment@0 {
        target = <&sound>;
        __overlay__ {
            compatible = "simple-audio-card";
            simple-audio-card,name = "sj201";
            status="okay";

            clocks = <&clocks 38>;
            clock-names = "mclk";
            clock-frequency = <24576000>;

            pinctrl-names = "default";
            pinctrl-0 = <&xvf_clk_pin &xvf_ctrl_pins>;

            // GPIO properties
            pwr-gpio = <&gpio 16 0>; // GPIO 16, active high
            rst-gpio = <&gpio 27 0>; // GPIO 27, active high

            capture_link: simple-audio-card,dai-link@0 {
                format = "i2s";
                bitclock-master = <&r_cpu_dai>;
                frame-master = <&r_cpu_dai>;

                r_cpu_dai: cpu {
                    sound-dai = <&i2s>;

                    dai-tdm-slot-num = <2>;
                    dai-tdm-slot-width = <32>;
                };

                r_codec_dai: codec {
                    sound-dai = <&codec_in>;
                };
            };

            playback_link: simple-audio-card,dai-link@1 {
                format = "i2s";

                p_cpu_dai: cpu {
                    sound-dai = <&i2s>;

                    dai-tdm-slot-num = <2>;
                    dai-tdm-slot-width = <32>;
                };

                p_codec_dai: codec {
                    sound-dai = <&codec_out>;
                };
            };
        };
    };

    fragment@1 {
        target-path = "/";
        __overlay__ {
            codec_out: spdif-transmitter {
                #address-cells = <0>;
                #size-cells = <0>;
                #sound-dai-cells = <0>;
                compatible = "linux,spdif-dit";
                status = "okay";
            };

            codec_in: spdif-receiver {
                #address-cells = <0>;
                #size-cells = <0>;
                #sound-dai-cells = <0>;
                compatible = "linux,spdif-dir";
                status = "okay";
            };
        };
    };

    fragment@2 {
        target = <&i2s_clk_consumer>;
        __overlay__ {
            #sound-dai-cells = <0>;
            status = "okay";
        };
    };

    /* set GPIO4 to ALT0 (GPCLK0)  */
    fragment@3 {
        target = <&gpio>;
        __overlay__ {
            xvf_clk_pin: xvf_clk_pin {
                brcm,pins = <4>;
                brcm,function = <4>;
            };
            xvf_ctrl_pins: xvf_ctrl_pins {
                brcm,pins = <16 27>;
                brcm,function = <1 1>; // Set as output
                // Assuming active high, no pull-up/down needed
            };
        };
    };
};
j1nx commented 7 months ago

BTW, untested however I do have a slightly newer firmware for the board with the below changelog from XMOS.

sw_xvf3510 change log
=====================

4.2.0
-----

  * CHANGED: Update MCLK divider in I2S slave data partition to support
    Raspberry Pi4B
  * FIXED: Make timing of startup control commands more deterministic
  * FIXED: Check order of control commands added to data partition in generator
    script
  * CHANGED: tidy Jenkinsfile
  * REMOVED: dependency to libreadline-dev in VfCtrl host app
  * REMOVED: not necessary cpanfile
  * ADDED: Range check of input arguments of SET_IO_MAP command in VfCtrl host
    app

  * Changes to dependencies:

    - audio_test_tools: 4.2.0 -> 4.3.0

      + ADDED: test_wav_xscope test feature
      + FIXED: test_wav_xx build failure for non-xscope (axe) apps
      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_aec: 8.0.4 -> 8.0.5

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_agc: 7.0.1 -> 7.0.2

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_ai: 1.0.0 -> 1.0.1

      + REMOVED: not necessary cpanfile

    - lib_audio_pipelines: 1.3.1 -> 1.3.2

      + CHANGED: Refactored stage C XC code
      + CHANGED: Refactored stage B XC code
      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_device_control: 4.0.1 -> 4.0.2

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_dfu: 1.0.5 -> 1.0.6

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_dsp: 6.0.1 -> 6.0.2

      + CHANGED: use XS2 version of platform-specific functions on XS3
      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_flash_data_partition: 2.2.0 -> 2.2.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_i2c: 6.0.0 -> 6.0.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_i2s: 4.1.0 -> 4.1.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_interference_canceller: 5.1.1 -> 5.1.2

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_logging: 3.0.0 -> 3.0.2

      + REMOVED: not necessary cpanfile
      + CHANGED: Pin Python package versions

    - lib_mic_array: 4.2.1 -> 4.2.2

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_ndp: 2.0.0 -> 2.0.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_noise_suppression: 2.1.0 -> 2.1.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_spdif: 4.0.0 -> 4.0.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_spi: 3.1.0 -> 3.1.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_src: 2.0.0 -> 2.0.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_vad: 1.0.3 -> 1.0.4

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_voice_toolbox: 8.0.0 -> 8.0.1

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_xassert: 4.0.0 -> 4.0.2

      + REMOVED: not necessary cpanfile
      + CHANGED: Pin Python package versions

    - lib_xua: 1.1.0 -> 1.1.1

      + RESOLVED: Zero length input packets generated before enumeration causing
        I2S timing pushout at startup
      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile

    - lib_xud: 1.0.0 -> 1.1.2

      + CHANGED: Pin Python package versions
      + REMOVED: not necessary cpanfile
      + RESOLVED:   Cases where disabling RxError caused firmware to crash
      + RESOLVED:   USB Disconnect on self-powered devices intermittently
        causing EP set to not-ready indefinitely
      + RESOLVED:   Disabled erroneous handling of Rx Error line

app_xvf3510_int_spi_boot_v4_2_0.zip

The older 4.1.0 version remian here; https://github.com/OpenVoiceOS/ovos-buildroot/tree/develop/buildroot-external/package/vocalfusion However while we are at it, I guess the firmware and xvf3510flash script should be merged into this repo.

For point 4 above, we might need to look into how certain wifi drivers flash their firmware at driver init, such that we can do the same. https://docs.kernel.org/driver-api/firmware/fw_upload.html

However, one step at a time. Let's get the mics working first

j1nx commented 7 months ago

This PR just surfaced and looks ike it has quite some similarities. https://github.com/raspberrypi/linux/pull/5949

BohdanBuinich commented 7 months ago

https://github.com/OpenVoiceOS/VocalFusionDriver/pull/5

Part of the work I've done. not tested yet