flutter: fetching dependencies via `dart pub get` results in TLS error if run in sandbox

HannesGitH commented 6 months ago

Describe the bug

Inside a flake sandbox, with access to the internet, i.e. a fixed-output derivation, having cacert as a build dependeny doesnt suffice, as dart pub get still results in a TLS error.

flutter doctor fails the network with

✗ A cryptographic error occurred while checking "https://pub.dev/": Handshake error in client
    You may be experiencing a man-in-the-middle attack, your network may be compromised, or you may have malware installed on your computer.

and dart pub get -vv with

Got TLS error trying to find package intl at https://pub.dev.

which is caused by

Wrapped exception: HandshakeException: Handshake error in client 
(OS Error: CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate(handshake.cc:393))

as wget is working it seems like dart is looking for certificates in the wrong places, not provided by cacert?

Steps To Reproduce

Steps to reproduce the behavior:

have a flake with following fixed-output derivation package (inside packages):

default = pkgs.stdenv.mkDerivation {
        name = "fetchDeps";
        buildInputs = with pkgs; [ wget cacert git flutter ];
        src = frontend-dir;
        buildPhase = ''
          export HOME=$(mktemp -d)
          chmod -R 777 $HOME
          export DART_VM_OPTIONS="--root-certs-file=/etc/ssl/certs/ca-certificates.crt"
          wget https://nixos.org/manual/ # just to check, internet and access to https works
          flutter doctor
          dart pub get --enforce-lockfile -vv
        '';
        installPhase = ''
          mkdir -p $out
          cp -r $HOME/.pub-cache $out/.pub-cache
        '';
        outputHashAlgo = "sha256";
        outputHashMode = "recursive";
        outputHash = pkgs.lib.fakeHash; #only used that the derivation has internet
      };

nix build
...

Expected behavior

The command should run similarly as to when I type dart pub get inside nix develop, i.e. fetch the dependencies and add them to .pub-cache

Additional context

I have posted a closely related question in the nixos discourse

Notify maintainers

@babariviere @ericdallo @FlafyDev @hacker1024

Metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.25, NixOS, 24.05 (Uakari), 24.05.20240408.4cba8b5`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.2`
 - channels(root): `"home-manager-23.11.tar.gz, nixos-23.11"`
 - nixpkgs: `/nix/store/450afzqlzzgw6wnyc3dwysf3i5yxyqkr-source`

Add a :+1: reaction to issues you find important.

nixos-discourse commented 6 months ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/reproducable-flutter-app-with-flake/44156/2

FlafyDev commented 6 months ago

You can use buildFlutterApplication and buildDartApplication to build Flutter and Dart apps in derivations: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/dart.section.md. buildFlutterApplication only supports Linux and Web currently.

As for running dart pub get in a derivation, this is how it's currently done:

 dart --root-certs-file=${cacert}/etc/ssl/certs/ca-bundle.crt pub get -v

for how to do it with flutter: https://github.com/NixOS/nixpkgs/blob/6d1c562d34b80f81165430c0e6c4c66c02c1d69d/pkgs/build-support/flutter/default.nix#L29-L51

Though I'd recommend to just use buildFlutterApplication or buildDartApplication rather than making your own derivation.

HannesGitH commented 6 months ago

That's exactly the hints I was looking for, thanks a lot! I'll check that out soon. Would PRs adding new targetPlatforms be appreciated (if there is a good way and I can find it) then?

FlafyDev commented 6 months ago

Would PRs adding new targetPlatforms be appreciated (if there is a good way and I can find it) then?

Yeah, absolutely! Are you looking to add apk support for buildFlutterApplication? For building apks with Flutter outside of a derivation you can use a flake like this(took it from this issue):

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixpkgs-unstable";

    flake-parts = {
      url = "github:hercules-ci/flake-parts";
      inputs.nixpkgs-lib.follows = "nixpkgs";
    };
  };

  outputs = {
    self,
    nixpkgs,
    flake-parts,
    ...
  } @ inputs:
    flake-parts.lib.mkFlake {inherit inputs;} {
      systems = ["x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin"];
      perSystem = {
        pkgs,
        system,
        lib,
        ...
      }: {
        devShells.default = let
          pkgs = import nixpkgs {
            inherit system;
            config = { allowUnfree = true; android_sdk.accept_license = true; };
          };
          buildToolsVersionForAapt2 = "34.0.0";
          androidComposition = pkgs.androidenv.composeAndroidPackages {
            # Installing both version for aapt2 and version that flutter wants
            buildToolsVersions = [buildToolsVersionForAapt2 "30.0.3"];
            platformVersions = ["34" "33" "31" "30"];
            abiVersions = ["armeabi-v7a" "arm64-v8a" "x86" "x86_64"];
            toolsVersion = "26.1.1";
            platformToolsVersion = "33.0.3";
            extraLicenses = [
              "android-googletv-license"
              "android-sdk-arm-dbt-license"
              "android-sdk-license"
              "android-sdk-preview-license"
              "google-gdk-license"
              "intel-android-extra-license"
              "intel-android-sysimage-license"
              "mips-android-sysimage-license"
            ];
          };
          androidSdk = androidComposition.androidsdk;
        in
          pkgs.mkShell {
            ANDROID_SDK_ROOT = "${androidSdk}/libexec/android-sdk";
            ANDROID_HOME = "${androidSdk}/libexec/android-sdk";
            GRADLE_OPTS = "-Dorg.gradle.project.android.aapt2FromMavenOverride=${androidSdk}/libexec/android-sdk/build-tools/${buildToolsVersionForAapt2}/aapt2";
            buildInputs = with pkgs; [
              flutter
              pkg-config
              jdk17
              androidSdk
              android-tools
            ];
          };
        formatter = pkgs.alejandra;
      };
    };
}

hacker1024 commented 6 months ago

I'll also point out that using pub get in a FOD is not trivial, and took months to get right in Nixpkgs.

Pub likes to create many temporary files and leaves timestamps everywhere. Git dependencies are also not stable.

If you must use it, I recommend taking a look at the Nixpkgs implementation before the switch to Nix in #263345.

HannesGitH commented 6 months ago

I'll provide my progress in this PR: https://github.com/NixOS/nixpkgs/pull/307983

I havent tested anything yet, and let you know when its ready for review or when I need more help :)

edit: i seem to have a problem with ncurses (which does not arise when using nixpkgs 23.11), if someone could check out my PR and/or provide help, that would be greatly appreciated :)

nixos-discourse commented 1 month ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flutter-flake-for-development/54017/1

NixOS / nixpkgs