search

cideM / solana-nix

The Solana CLI tools packaged up with Nix
GNU Affero General Public License v3.0
14 stars 4 forks source link

missing BPF binary #3

Open sfultong opened 3 years ago

sfultong commented 3 years ago

I'm trying to get started with Solana dev, so I may be doing something wrong here. But when I try to run cargo build-bpf for a tutorial using this flake, I get:

BPF SDK path does not exist: /nix/store/2zgi0xf7byrqz8v8hj87pfvhjl8w7lyz-solana-1.7.15/bin/sdk/bpf: No such file or directory (os error 2)

hhefesto commented 3 years ago

Same error:

$ nix shell
 ...
 $ solana --version                                                                                                                                      nix-shell
solana-cli 1.7.15 (src:devbuild; feat:2995838076)
$ cargo-build-bpf                                                                                                                                       nix-shell
BPF SDK path does not exist: /nix/store/2zgi0xf7byrqz8v8hj87pfvhjl8w7lyz-solana-1.7.15/bin/sdk/bpf: No such file or directory (os error 2)
$ which cargo-build-bpf                                                                                                                                 nix-shell
/nix/store/2zgi0xf7byrqz8v8hj87pfvhjl8w7lyz-solana-1.7.15/bin/cargo-build-bpf
cideM commented 3 years ago

I'll look into it!

cideM commented 3 years ago

@sfultong @hhefesto can you try the fix-build-bpf branch? The BPF SDK is now copied into the output directory. I can run cargo-build-bpf --help and you should be able to run it with the CLI flags shown in the shell script of the same name https://github.com/solana-labs/solana/blob/master/cargo-build-bpf which, I'm guessing, achieves the same result?

sfultong commented 3 years ago

Thanks for the quick response @cideM !

Now I get a different error:

BPF SDK: /nix/store/nxkbyijrrzjv8j4sszig7ya4695njxw9-solana-1.7.15/bin/sdk/bpf
thread 'main' panicked at 'Failed to install bpf-tools: "Read-only file system (os error 30)"', sdk/cargo-build-bpf/src/main.rs:440:6
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
hhefesto commented 3 years ago

I managed to get other errors:

First I cloned https://github.com/solana-labs/solana. There is a sdk directory on the top level of that repository. I copied pasted that directory and specified the route with:

╭─hhefesto@olimpo ~/src/solana-nix ‹main*› 
╰─$ cargo-build-bpf --bpf-sdk ./sdk                                                                                                                           nix-shell
Failed to obtain package metadata: Error during execution of `cargo metadata`: error: could not find `Cargo.toml` in `/home/hhefesto/src/solana-nix` or any parent directory

Which prompted me to provide cargo and do:

╭─hhefesto@olimpo ~/src/solana-nix ‹main*› 
╰─$ cargo init                                                                                                                                                cargo
     Created binary (application) package
╭─hhefesto@olimpo ~/src/solana-nix ‹main*› 
╰─$ cargo-build-bpf --bpf-sdk ./sdk                                                                                                                           cargo
Note: solana-nix crate does not contain a cdylib target
BPF SDK: /home/hhefesto/src/solana-nix/sdk
thread 'main' panicked at 'Failed to install bpf-tools: "Unable to write \"/home/hhefesto/.cache/solana/v1.12/bpf-tools/tmp-solana-bpf-tools-linux.tar.bz2\": Custom { kind: Other, error: reqwest::Error { kind: Decode, source: TimedOut } }"', sdk/cargo-build-bpf/src/main.rs:440:6
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Then I noticed your answer here and tried in the fix-biuld-bpf branch:

╭─hhefesto@olimpo ~/src/solana-nix ‹fix-build-bpf*› 
╰─$ cargo-build-bpf --bpf-sdk ./sdk                                                                                                                           cargo
Note: solana-nix crate does not contain a cdylib target
BPF SDK: /home/hhefesto/src/solana-nix/sdk
Running: rustup toolchain list -v
Failed to execute rustup: No such file or directory (os error 2)

So I provided rustup and:

╭─hhefesto@olimpo ~/src/solana-nix ‹fix-build-bpf*› 
╰─$ nix-shell -p rustup                                                                                                                                       cargo
this path will be fetched (2.65 MiB download, 9.19 MiB unpacked):
  /nix/store/2xqcbb5dqzqrkqyypw1qcdlh8ym0qzcr-rustup-1.24.2
copying path '/nix/store/2xqcbb5dqzqrkqyypw1qcdlh8ym0qzcr-rustup-1.24.2' from 'https://cache.nixos.org'...
╭─hhefesto@olimpo ~/src/solana-nix ‹fix-build-bpf*› 
╰─$ cargo-build-bpf --bpf-sdk ./sdk                                                                                                                    cargo rustup
Failed to obtain package metadata: Error during execution of `cargo metadata`: error: no override and no default toolchain set

I'm unsure if this last error is an expected error: my rust knowledge is quite superficial.

hhefesto commented 3 years ago

And also thanks for the quick response and this flake <3

hhefesto commented 3 years ago

same error without --bpf-sdk on fix-build-bpf branch

cideM commented 3 years ago

I had to read up on what cargo-build-bpf does. It seems that it's used to compile on-chain programs as shown https://github.com/solana-labs/break/blob/main/README.md

I'll try to run this tutorial myself and see if I can fix the issues.

cideM commented 3 years ago

Wanted to give everyone an update. It's really busy at work so I couldn't spend much time on it. But this line in the SDK is likely the problem:

 136   │     // Make a symbolic link source_path -> target_path in the
 137   │     // sdk/bpf/dependencies directory if no valid link found.
 138   │     let source_base = config.bpf_sdk.join("dependencies");
 139   │     if !source_base.exists() {

This will make a symlink into the source folder, which is read-only, hence the Permission denied

Good thing is, https://github.com/solana-labs/bpf-tools/releases/download/v1.12/solana-bpf-tools-osx.tar.bz2 this is the URL that is used to download the missing tools. It's slightly different for Linux.

But I think it should be pretty easy to handle this download with Nix, as it should be. Hopefully the Rust code will accept the resulting symlink. If not, patching it is always an option. I'll get something going this weekend.

cideM commented 3 years ago

@hhefesto @sfultong could you pull from the branch and try again? I'm using this flake for the https://github.com/solana-labs/break/blob/main/README.md project:

{
  description = "Solana CLI";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
    solanaNix.url = "path:/Users/fbs/private/solana-nix";
    fenix = {
      url = "github:nix-community/fenix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, flake-utils, solanaNix, fenix }:
    flake-utils.lib.eachSystem [ "x86_64-linux" "x86_64-darwin" "aarch64-darwin" ] (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ fenix.overlay ];
        };

      in
      rec {
        devShell = pkgs.mkShell {
          buildInputs = with pkgs; [
            libiconv
            solanaNix.packages.${system}.solana
            rustup
            (fenix.packages.${system}.complete.withComponents [
              "cargo"
              "clippy"
              "rust-src"
              "rustc"
              "rustfmt"
            ])
          ];
        };
      }
    );
}

you'll have to adjust the path to my solana-nix repo. Here's what I then do:

~/private/solana-nix-issue/break on main •
$ nix develop
warning: Git tree '/Users/fbs/private/solana-nix-issue/break' is dirty
Florians-MBP:break fbs$ env -C program cargo-build-bpf
BPF SDK: /nix/store/cahp4fsddad29fbarmydyl8gi1231422-solana-1.7.15/bin/sdk/bpf
Running: rustup toolchain list -v
Running: cargo +bpf build --target bpfel-unknown-unknown --release
   Compiling proc-macro2 v1.0.24
   Compiling syn v1.0.67
   Compiling serde_derive v1.0.125
   Compiling serde v1.0.125
   Compiling typenum v1.12.0
   Compiling rustversion v1.0.3
   Compiling log v0.4.14
   Compiling byteorder v1.3.4

This fails for me with (https://github.com/solana-labs/bpf-tools/issues/25)

          ld: warning: ignoring file /Users/fbs/private/solana-nix-issue/break/program/target/release/build/log-942b38fc0d7404da/build_script_build-942b38fc0d7404da.build_script_build.62arnz6f-cgu.9.rcgu.o, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
          ld: entry point (_main) undefined. for architecture arm64
          clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

but I think that's just me being on M1 when the BPF tools are for x86. I'll look into this further but maybe you can already try if this helps.

For now I patched out the installation of the BPF tools since they're provided with my flake on my branch

sfultong commented 3 years ago

Thanks @cideM !

I get error: no override and no default toolchain set now, but I think that's not a hard fix.

The Solana toolchain needs a specific version of rust (1.53 or 1.56?) and I assume fenix does not provide that. I'm going to try switching that out for another rust flake

sfultong commented 3 years ago

Hmmm, I'm still getting the same error with oxalica/rust-overlay and rust 1.56.0

I'm not sure what to try next

cideM commented 3 years ago

I can take a look on my NixOS desktop machine on Monday or Tuesday. I'm at my parents place right now and only have my MacBook with me.

I did notice that you talk about cargo build-bpf so I just want to make sure that that's not a typo. Because I'm always running cargo-build-bpf

sfultong commented 3 years ago

I think cargo build-bpf and cargo-build-bpd are the same

cideM commented 3 years ago

@sfultong 

{
  description = "Solana CLI";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
    solanaNix.url = "path:/Users/fbs/private/solana-nix";
    fenix = {
      url = "github:nix-community/fenix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, flake-utils, solanaNix, fenix }:
    flake-utils.lib.eachSystem [ "x86_64-linux" "x86_64-darwin" ] (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ fenix.overlay ];
        };

      in
      rec {
        devShell = pkgs.mkShell {
          buildInputs = with pkgs; [
            libiconv
            solanaNix.packages.${system}.solana
            rustup
            (fenix.packages.${system}.complete.withComponents [
              "cargo"
              "clippy"
              "rust-src"
              "rustc"
              "rustfmt"
            ])

            (pkgs.writeScriptBin "run" ''
              env -C program ${solanaNix.packages.x86_64-darwin.solana}/bin/cargo-build-bpf
            '')
          ];
        };
      }
    );
}

On my Darwin M1 machine, using the .#devShell.x86_64-darwin, I can run run and it successfully compiles the program. Using the latest changes on the fix-build-bpf branch that is. I will also start testing this on NixOS tomorrow.

~/private/solana-nix-issue/break on main •
❯ run
BPF SDK: /nix/store/ns6xkm0jbkz80h0skpzh4phrpvpn64zl-solana-1.7.15/bin/sdk/bpf
Running: rustup toolchain list -v
Running: cargo +bpf build --target bpfel-unknown-unknown --release
   Compiling proc-macro2 v1.0.24
   Compiling syn v1.0.67
   Compiling serde v1.0.125
   Compiling serde_derive v1.0.125
   Compiling typenum v1.12.0
   Compiling byteorder v1.3.4
   Compiling log v0.4.14
   Compiling rustversion v1.0.3
   Compiling opaque-debug v0.3.0
   Compiling ahash v0.4.7
   Compiling lazy_static v1.4.0
   Compiling either v1.5.3
   Compiling bs58 v0.3.1
   Compiling hex v0.4.2
   Compiling semver v0.9.0
   Compiling generic-array v0.14.4
   Compiling bv v0.11.1
   Compiling num-traits v0.2.11
   Compiling itertools v0.9.0
   Compiling hashbrown v0.9.1
   Compiling rustc_version v0.2.3
   Compiling solana-frozen-abi-macro v1.6.6
   Compiling solana-frozen-abi v1.6.6
   Compiling solana-program v1.6.6
   Compiling quote v1.0.2
   Compiling borsh-schema-derive-internal v0.8.2
   Compiling borsh-derive-internal v0.8.2
   Compiling thiserror-impl v1.0.9
   Compiling num-derive v0.3.0
   Compiling solana-sdk-macro v1.6.6
   Compiling thiserror v1.0.9
   Compiling serde_bytes v0.11.3
   Compiling bincode v1.3.1
   Compiling toml v0.5.8
   Compiling block-buffer v0.9.0
   Compiling digest v0.9.0
   Compiling sha2 v0.9.3
   Compiling proc-macro-crate v0.1.5
   Compiling borsh-derive v0.8.2
   Compiling borsh v0.8.2
   Compiling break-solana-program v0.1.0 (/Users/fbs/private/solana-nix-issue/break/program)
    Finished release [optimized] target(s) in 35.69s
Running: /nix/store/ns6xkm0jbkz80h0skpzh4phrpvpn64zl-solana-1.7.15/bin/sdk/bpf/scripts/strip.sh /Users/fbs/private/solana-nix-issue/break/program/target/bpfel-unknown-unknown/release/break_solana_program.so /Users/fbs/private/solana-nix-issue/break/program/target/deploy/break_solana_program.so
+ wget https://github.com/Snaipe/Criterion/releases/download/v2.3.2/criterion-v2.3.2-osx-x86_64.tar.bz2 -O criterion-v2.3.2-osx-x86_64.tar.bz2 --progress=dot:giga --retry-connrefused --read-timeout=30
--2021-11-07 21:02:07--  https://github.com/Snaipe/Criterion/releases/download/v2.3.2/criterion-v2.3.2-osx-x86_64.tar.bz2
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/30111969/33613116-1c60-11e7-8934-6166ffd90477?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211107%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211107T200208Z&X-Amz-Expires=300&X-Amz-Signature=33a3753454a6436d73c60892d73fc42e396c35a6ec046fa50cdd3b6f3cb8bad0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=30111969&response-content-disposition=attachment%3B%20filename%3Dcriterion-v2.3.2-osx-x86_64.tar.bz2&response-content-type=application%2Foctet-stream [following]
--2021-11-07 21:02:08--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/30111969/33613116-1c60-11e7-8934-6166ffd90477?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211107%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211107T200208Z&X-Amz-Expires=300&X-Amz-Signature=33a3753454a6436d73c60892d73fc42e396c35a6ec046fa50cdd3b6f3cb8bad0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=30111969&response-content-disposition=attachment%3B%20filename%3Dcriterion-v2.3.2-osx-x86_64.tar.bz2&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.111.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 180979 (177K) [application/octet-stream]
Saving to: ‘criterion-v2.3.2-osx-x86_64.tar.bz2’

     0K                                    100% 3.33M=0.05s

2021-11-07 21:02:08 (3.33 MB/s) - ‘criterion-v2.3.2-osx-x86_64.tar.bz2’ saved [180979/180979]

+ tar --strip-components 1 -jxf criterion-v2.3.2-osx-x86_64.tar.bz2
ln: failed to create symbolic link 'criterion': Permission denied
Running: /nix/store/ns6xkm0jbkz80h0skpzh4phrpvpn64zl-solana-1.7.15/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/llvm-readelf --dyn-symbols /Users/fbs/private/solana-nix-issue/break/program/target/deploy/break_solana_program.so

To deploy this program:
  $ solana program deploy /Users/fbs/private/solana-nix-issue/break/program/target/deploy/break_solana_program.so
The program address will default to this keypair (override with --program-id):
  /Users/fbs/private/solana-nix-issue/break/program/target/deploy/break_solana_program-keypair.json
sfultong commented 3 years ago

Yeah, I'm stilling getting the same error. I copied your flake, except I changed to solanaNix.url = github:cideM/solana-nix/fix-build-bpf and I changed the x86_64-darwin to x86-64-linux in the run script

cideM commented 3 years ago

So a little update. I can reproduce the error on NixOS. It's simply because I stuck a rustup in there. Without it, the error goes away, but the build command fails later because cargo in the same shell is broken as long as rustup is there too. I don't know why though.

The toolchain is used later during cargo build +bpf which selects the bpf toolchain

I created an issue https://github.com/nix-community/fenix/issues/58 because I'm not sure how to best make the custom BPF toolchain available.

sfultong commented 2 years ago

I was playing around with this, and seem to have gotten a bit further

[nix-shell:~/code/playnft/solana]$ cargo build-bpf
BPF SDK: /nix/store/fyhkchqfmx5mjhqb4rg1mzggvzd832kk-solana-1.7.15/bin/sdk/bpf
Running: rustup toolchain list -v
Running: rustup toolchain link bpf /nix/store/fyhkchqfmx5mjhqb4rg1mzggvzd832kk-solana-1.7.15/bin/sdk/bpf/dependencies/bpf-tools/rust
Running: cargo +bpf build --target bpfel-unknown-unknown --release
error: no such subcommand: `+bpf`

I'm using https://github.com/oxalica/rust-overlay/blob/master/docs/reference.md for a rust overlay rather than fenix

macalinao commented 2 years ago

I was playing around with this, and seem to have gotten a bit further

[nix-shell:~/code/playnft/solana]$ cargo build-bpf
BPF SDK: /nix/store/fyhkchqfmx5mjhqb4rg1mzggvzd832kk-solana-1.7.15/bin/sdk/bpf
Running: rustup toolchain list -v
Running: rustup toolchain link bpf /nix/store/fyhkchqfmx5mjhqb4rg1mzggvzd832kk-solana-1.7.15/bin/sdk/bpf/dependencies/bpf-tools/rust
Running: cargo +bpf build --target bpfel-unknown-unknown --release
error: no such subcommand: `+bpf`

I'm using https://github.com/oxalica/rust-overlay/blob/master/docs/reference.md for a rust overlay rather than fenix

This issue has to do with +bpf being part of Rustup