pico-sdk is missing tinyusb submodule

[icodeforyou-dot-net]

Describe the bug

pico-sdk as provided by nixpkgs is missing the tinyusb submodule needed for usb functionality and serial console via pico_enable_stdio_usb().

Steps To Reproduce

Steps to reproduce the behavior:

1. Create nix-shell with pico-sdk available in shell via shellHook export PICO_SDK_PATH="${pkgs.pico-sdk}/lib/pico-sdk"
2. Create a pi pico project with cmake using the PICO_SDK_PATH in the nix store
3. Cmake will issue the following warning:

[cmake] TinyUSB submodule has not been initialized; USB support will be unavailable [cmake] hint: try 'git submodule update --init' from your SDK directory

Expected behavior

TinyUSB submodule should be available to cmake via pico-sdk.

Notify maintainers

@muscaln @musfay

[muscaln]

Thanks for reporting!

TinyUSB submodule is rather huge. When I enable fetchSubmodules, closure size rises to 1,6 GiB. Main reason to package pico-sdk is picotool. Could you fetch submodules locally instead?

[icodeforyou-dot-net]

Yes, it is not so tiny after all then!

I tried using fetchSubmodules locally, but it did not work as expected. So I must be doing something wrong. Since I am not very deep into writing nix derivations, I decided to file this bug report. Maybe we can work on finding a setup that works and document a workaround here?

The alternative would be two create to separate derivations in nixpkgs, like pico-sdk and pico-sdk-tinyusb or something like this. Or to simply use the sdk entirely outside of nix. Which is sad because it means the setup will be far less reproducible.

My local setup right now is a flake.nix looking like this which I run with nix develop:

{
  description = "Basic rpi pico development shell";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/master";
  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system: 
    let
      pico-sdk131 = with pkgs; (pico-sdk.overrideAttrs (o:
        rec {
        pname = "pico-sdk";
        version = "1.3.1";
        src = fetchFromGitHub {
          fetchSubmodules = true;
          owner = "raspberrypi";
          repo = pname;
          rev = version;
          sha256 = "sha256-Yf3cVFdI8vR/qcxghcx8fsbYs1qD8r6O5aqZ0AkSLDc=";
        };
        }));
      pkgs = nixpkgs.legacyPackages.${system};
    in {
        devShell = pkgs.mkShell {
          buildInputs = with pkgs; [
            cmake
            gcc-arm-embedded
            libusb1
            openocd
            pico-sdk131
            picotool
            ];
          shellHook = with pkgs; ''
            export PICO_SDK_PATH="${pico-sdk131}/lib/pico-sdk"
            '';
          };
        });
}

So as you can see I override a number of the attributes of your derivation to get the latest version of the sdk. And I try to fetch the submodules. But I am not sure why it isn't working. Because cmake is unimpressed so far. When I give it a local $PICO_SDK_PATH and use the 'git submodule update --init command inside the repo, it works flawlessly.

Using the sdk in the nix store, cmake still tells me

[cmake] CMake Warning at /nix/store/9zwkkbkgjvpdmsfjddm64qfsdmjxf8nw-pico-sdk-1.3.1/lib/pico-sdk/src/rp2_common/tinyusb/CMakeLists.txt:10 (message):
[cmake]   TinyUSB submodule has not been initialized; USB support will be unavailable
[cmake]   hint: try 'git submodule update --init' from your SDK directory
[cmake]   (/nix/store/9zwkkbkgjvpdmsfjddm64qfsdmjxf8nw-pico-sdk-1.3.1/lib/pico-sdk).

[brodul]

@icodeforyou-dot-net make sure to remove files in your build directory.

I managed to get things to work. This is my shell.nix:

{ pkgs ? import <nixpkgs> {} }:
  let
    pico-sdk140 = with pkgs; pico-sdk.overrideAttrs(oldAttrs:
        rec {
        pname = "pico-sdk";
        version = "1.4.0";
        src = fetchFromGitHub {
          fetchSubmodules = true;
          owner = "raspberrypi";
          repo = pname;
          rev = version;
          sha256 = "sha256:1wihm752wm3mnnrqnr7vjvrlzrhpd418jgf9i5bfajri45qrl6vs";
        };
        });
  in
  pkgs.mkShell {
    nativeBuildInputs = with pkgs; [ pico-sdk140 cmake gcc-arm-embedded git ];

    shellHook = ''
    export PICO_SDK_PATH=${pico-sdk140.out}/lib/pico-sdk/
    '';
}

Then I have:

nix-shell
git clone https://github.com/raspberrypi/pico-examples
cd pico-examples
mkdir build
cd build
cmake -DPICO_BOARD=pico_w -DWIFI_SSID="Your Network" -DWIFI_PASSWORD="Your Password" ..

An I get:

Using PICO_SDK_PATH from environment ('/nix/store/ws1s6in360l1wkm1237ii2hwz7xqs9ym-pico-sdk-1.4.0/lib/pico-sdk/')
PICO_SDK_PATH is /nix/store/ws1s6in360l1wkm1237ii2hwz7xqs9ym-pico-sdk-1.4.0/lib/pico-sdk
Defaulting PICO_PLATFORM to rp2040 since not specified.
Defaulting PICO platform compiler to pico_arm_gcc since not specified.
-- Defaulting build type to 'Release' since not specified.
PICO compiler is pico_arm_gcc
-- The C compiler identification is GNU 10.3.1
-- The CXX compiler identification is GNU 10.3.1
-- The ASM compiler identification is GNU
-- Found assembler: /nix/store/2xk4mcxwh6c3xbk68x07w0pjndy6c0wx-gcc-arm-embedded-10.3.1/bin/arm-none-eabi-gcc
Build type is Release
PICO target board is pico_w.
Using CMake board configuration from /nix/store/ws1s6in360l1wkm1237ii2hwz7xqs9ym-pico-sdk-1.4.0/lib/pico-sdk/src/boards/pico_w.cmake
Using board configuration from /nix/store/ws1s6in360l1wkm1237ii2hwz7xqs9ym-pico-sdk-1.4.0/lib/pico-sdk/src/boards/include/boards/pico_w.h
-- Found Python3: /home/brodul/.nix-profile/bin/python3.11 (found version "3.11.0") found components: Interpreter 
TinyUSB available at /nix/store/ws1s6in360l1wkm1237ii2hwz7xqs9ym-pico-sdk-1.4.0/lib/pico-sdk/lib/tinyusb/src/portable/raspberrypi/rp2040; enabling build support for USB.
cyw43-driver available at /nix/store/ws1s6in360l1wkm1237ii2hwz7xqs9ym-pico-sdk-1.4.0/lib/pico-sdk/lib/cyw43-driver
lwIP available at /nix/store/ws1s6in360l1wkm1237ii2hwz7xqs9ym-pico-sdk-1.4.0/lib/pico-sdk/lib/lwip
Enabling build support for Pico W wireless.
Skipping tcp_client example as TEST_TCP_SERVER_IP is not defined
Skipping Pico W FreeRTOS examples as FREERTOS_KERNEL_PATH not defined
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/banana/pico-examples/build

Lost a bunch of time because CMake is caching a lot of stuff in the build folder.

[icodeforyou-dot-net]

@brodul thanks for reply! I will give it a try tomorrow and see if I get the desired results.

[brodul]

pico-sdk could have a input parameter e.g. (tinyusbSupport). Then we have 2 packages in all-packages. I guess we can exclude the one with tinyusb from being build on Hydra? What do you think?

[muscaln]

I can create a setup hook for pico-sdk package to set proper PICO_SDK_PATH. You can also use pkgsCross.arm-embedded.buildPackages.gcc for compiling.

[icodeforyou-dot-net]

@brodul I finally had the time to get back to my pico stuff. It appears to work now, both with your nix-shell based approach, as well as with my modified flake based nix develop setup.

My working flake.nix now looks like this:

{
  description = "Basic rpi pico development shell";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/master";
  inputs.flake-utils.url = "github:numtide/flake-utils";

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system: 
    let
      pico-sdk140 = with pkgs; (pico-sdk.overrideAttrs (o:
        rec {
        pname = "pico-sdk";
        version = "1.4.0";
        src = fetchFromGitHub {
          fetchSubmodules = true;
          owner = "raspberrypi";
          repo = pname;
          rev = version;
          sha256 = "sha256:1wihm752wm3mnnrqnr7vjvrlzrhpd418jgf9i5bfajri45qrl6vs";
        };
        }));
      pkgs = nixpkgs.legacyPackages.${system};
    in {
        devShell = pkgs.mkShell {
          nativeBuildInputs = with pkgs; [
            pico-sdk140 
            cmake
            clang-tools
            gcc-arm-embedded
            ];
          shellHook = ''
            export PICO_SDK_PATH="${pico-sdk140}/lib/pico-sdk"
            '';
          };
      });
}

For me personally this issue is done now, if no one has any objections I would close it in maybe a week from now. Perhaps I will find some time and document this setup in the nixos wiki.

[danwdart]

I tried to just do a simple import of pico-sdk but it said that since the submodules weren't included, it wouldn't do the Pico W examples, because of the lack of the support driver. Is this meant to happen or do I override at that point?

[chkno]

It would be nice to have the proper, full pico-sdk available, even if it is excluded from hydra builds over size concerns. Currently, anyone that wants to actually build software with pico-sdk has to overrideAttrs it and change the src to add fetchSubmodules = true;. We could rename the current thing pico-sdk-minimal.

Here's a concrete example of how to build a Pico W image with nix/nixpkgs today. I wasn't able to get muscaln's suggestion above of using pkgsCross.arm-embedded.buildPackages.gcc to work. I resorted to using runInLinuxImage to build with Debian's gcc-arm-none-eabi

[hitsmaxft]

the latest pico-idk 1.5.1 has tinyusb as submodule, and tinyusb have lots of submodules, that's the cause of huge file size if fetchgit with fetchSubmodule=true (there's not an option to disable recursive fetching)

[vymiheev]

git submodule update --init --recursive will solve problem