Inconsistent behavior with 'dev' versions of a package

janosimas commented 1 month ago

What happened?

Note: I have no knowledge on Nix, all here is based on google.

I'm trying to build (cargo install) a package that depends on openssl. To add the development package of openssl, I added openssl.dev to my devbox.json.

I have two problems here:

I'm unable to set the version of the package This works:
```
"packages": [
"openssl.dev"
],
```
But his doesn't:
```
"packages": [
"openssl.dev@3.0.10"
],
```
devbox update also complains of openssl.dev.
I have to manually set the environment variables for openssl. Is this a devbox or a nix package issue?

Steps to reproduce

devbox add openssl.dev@3.0.10

Command

add, run, shell, update

devbox.json

{
  "packages": [
    "rustup@latest",
    "libiconv@latest"
  ],
  "env": {
    "OPENSSL_DIR":     "/nix/store/npgyxjszxhdd6lwhxxw3wpxq76qqspx6-openssl-3.0.11-dev",
    "OPENSSL_LIB_DIR": "/nix/store/ld5jfd5y6a0sx4r8rassg6j2dcldms6s-openssl-3.0.11/lib/"
  },
  "shell": {
    "init_hook": [
      "rustup default nightly"
    ],
    "scripts": {
      "update_tools": "cargo install-update --all"
    }
  }
}

Devbox version

0.12.0

Nix version

nix (Nix) 2.18.1

What system does this bug occur on?

Linux (x86-64)

Debug logs

time=2024-08-01T06:34:43.176Z level=DEBUG source=go.jetpack.io/devbox/internal/devbox/dir.go:22 msg="finding devbox config" path="" time=2024-08-01T06:34:43.177Z level=DEBUG source=go.jetpack.io/devbox/internal/devbox/dir.go:66 msg="finding devbox config" dir=/home/jsimas/tools/devbox/rust time=2024-08-01T06:34:43.177Z level=DEBUG source=go.jetpack.io/devbox/internal/devbox/dir.go:22 msg="finding devbox config" path="" time=2024-08-01T06:34:43.177Z level=DEBUG source=go.jetpack.io/devbox/internal/devbox/dir.go:66 msg="finding devbox config" dir=/home/jsimas/tools/devbox/rust time=2024-08-01T06:34:43.221Z level=DEBUG source=go.jetpack.io/devbox/internal/nix/nix.go:337 msg="nix --version --debug output" out="nix (Nix) 2.18.1\nSystem type: x86_64-linux\nAdditional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux\nFeatures: gc, signed-caches\nSystem configuration file: /etc/nix/nix.conf\nUser configuration files: /home/jsimas/.config/nix/nix.conf:/etc/xdg/nix/nix.conf\nStore directory: /nix/store\nState directory: /nix/var/nix\nData directory: /nix/store/azvn85cras6xv4z5j85fiy406f24r1q0-nix-2.18.1/share\n" time=2024-08-01T06:34:43.221Z level=DEBUG source=go.jetpack.io/devbox/internal/nix/command.go:60 msg="nix command starting" cmd.args="nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command flakes fetch-closure' eval --impure --raw --expr builtins.currentSystem" cmd.path=/nix/var/nix/profiles/default/bin/nix time=2024-08-01T06:34:43.258Z level=DEBUG source=go.jetpack.io/devbox/internal/nix/command.go:67 msg="nix command exited" cmd.args="nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command flakes fetch-closure' eval --impure --raw --expr builtins.currentSystem" cmd.path=/nix/var/nix/profiles/default/bin/nix cmd.pid=1845434 cmd.code=0 cmd.dur=37.073604ms time=2024-08-01T06:34:43.258Z level=DEBUG source=go.jetpack.io/devbox/internal/devbox/dir.go:22 msg="finding devbox config" path="" time=2024-08-01T06:34:43.258Z level=DEBUG source=go.jetpack.io/devbox/internal/devbox/dir.go:66 msg="finding devbox config" dir=/home/jsimas/tools/devbox/rust time=2024-08-01T06:34:43.408Z level=DEBUG source=go.jetpack.io/devbox/internal/nix/command.go:60 msg="nix command starting" cmd.args="nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command flakes fetch-closure' search 'github:NixOS/nixpkgs/75a52265bda7fd25e06e3a67dee3f0354e73243c#openssl.dev@3.0.10' ^ --json" cmd.path=/nix/var/nix/profiles/default/bin/nix time=2024-08-01T06:34:43.479Z level=DEBUG source=go.jetpack.io/devbox/internal/nix/command.go:67 msg="nix command exited" cmd.args="nix --extra-experimental-features ca-derivations --option experimental-features 'nix-command flakes fetch-closure' search 'github:NixOS/nixpkgs/75a52265bda7fd25e06e3a67dee3f0354e73243c#openssl.dev@3.0.10' ^ --json" cmd.path=/nix/var/nix/profiles/default/bin/nix cmd.stderr="flake 'github:NixOS/nixpkgs/75a52265bda7fd25e06e3a67dee3f0354e73243c' does not provide attribute 'packages.x86_64-linux.openssl.dev@3.0.10', 'legacyPackages.x86_64-linux.openssl.dev@3.0.10' or 'openssl.dev@3.0.10'" cmd.pid=1845443 cmd.code=1 cmd.dur=70.36248ms

Error: Package openssl.dev@3.0.10 not found

time=2024-08-01T06:34:43.479Z level=ERROR source=go.jetpack.io/devbox/internal/boxcli/midcobra/debug.go:70 msg="command error" execid=48d29b9af8894e66821c5c636a0d6a58 stack="Package openssl.dev@3.0.10 not found\ngo.jetpack.io/devbox/internal/boxcli/usererr.New\n\tgo.jetpack.io/devbox/internal/boxcli/usererr/usererr.go:29\ngo.jetpack.io/devbox/internal/devbox.(Devbox).Add\n\tgo.jetpack.io/devbox/internal/devbox/packages.go:105\ngo.jetpack.io/devbox/internal/boxcli.addCmdFunc\n\tgo.jetpack.io/devbox/internal/boxcli/add.go:88\ngo.jetpack.io/devbox/internal/boxcli.addCmd.func1\n\tgo.jetpack.io/devbox/internal/boxcli/add.go:47\ngithub.com/spf13/cobra.(Command).execute\n\tgithub.com/spf13/cobra@v1.8.0/command.go:983\ngithub.com/spf13/cobra.(Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.8.0/command.go:1115\ngithub.com/spf13/cobra.(Command).Execute\n\tgithub.com/spf13/cobra@v1.8.0/command.go:1039\ngo.jetpack.io/devbox/internal/boxcli/midcobra.(*midcobraExecutable).Execute\n\tgo.jetpack.io/devbox/internal/boxcli/midcobra/midcobra.go:61\ngo.jetpack.io/devbox/internal/boxcli.Execute\n\tgo.jetpack.io/devbox/internal/boxcli/root.go:115\ngo.jetpack.io/devbox/internal/boxcli.Main\n\tgo.jetpack.io/devbox/internal/boxcli/root.go:138\nmain.main\n\t./main.go:11\nruntime.main\n\truntime/proc.go:271\nruntime.goexit\n\truntime/asm_amd64.s:1695" time=2024-08-01T06:34:43.479Z level=DEBUG source=go.jetpack.io/devbox/internal/devbox/dir.go:22 msg="finding devbox config" path="" time=2024-08-01T06:34:43.479Z level=DEBUG source=go.jetpack.io/devbox/internal/devbox/dir.go:66 msg="finding devbox config" dir=/home/jsimas/tools/devbox/rust

Lagoja commented 1 month ago

Hey @janosimas,

The best way to install alternate outputs like dev is not with the package.output format, but rather using the --outputs flag. For example:

devbox add openssl@3.0.10 --outputs=out,dev

This should update your packages to include the following:

"packages": {
    "openssl": {
      "version": "3.0.10",
      "outputs": ["out", "dev"]
    }
}

see https://www.jetify.com/devbox/docs/faq/#a-package-i-installed-is-missing-header-files-or-libraries-i-need-for-development-where-do-i-find-them for more details

Reading the docs, I think we can cover this better. I'll work on an update to the docs

janosimas commented 1 month ago

Thanks! It seams to be working fine this way. I'll do some more tests and add a note here if I find anything.

How do I know what is the output of an output? I checked the nix package for openssl but it doesn't give much information.

Lagoja commented 1 month ago

It's unfortunately not very well documented or standardized, and you would need to check the source code in Nixpkgs.

Generally libraries are in lib or dev, header files are in dev, and the main binaries for the package are in out

jetify-com / devbox