flake: make it easier to use devenv in Nix flake

[bobvanderlinden]

Currently it isn't trivial to start using devenv inside of a Nix flake. This PR attempts to ease the use.

First it introduces a mkShell lib function. This makes it possible to use devenv using:

devShells.x86_64-linux.default = devenv.lib.mkShell {
  inherit inputs pkgs;
  modules = [
    {

    }
  ];
};

In addition, I thought some documentation was useful. However I didn't want to confuse people with Nix flake things and conflate it with devenv. As a first step I added a Nix template that allows starting a Nix flake + direnv + devenv project using:

nix flake init --template cachix/devenv#simple

Are these things that you'd accept in devenv? What would be a good place to document this?

[hugosenari]

You're in the right path, maybe you need some inspiration. http://mourlot.free.fr/english/fmtaureau.html :taurus:

[bobvanderlinden]

You're in the right path, maybe you need some inspiration. http://mourlot.free.fr/english/fmtaureau.html ♉

What part are you referring to specifically?

[domenkozar]

This is both scary and exciting :)

If we do support this, it should be documented and tested.

It will be quite a degraded experience, all the fixes that come with a fork of Nix are lost, on top of the UX cli improvements.

I think a better approach will be for devenv.nix to use the local flake.nix as an input to consume packages, etc - this is something I'm working on.

Would you prefer flake.nix wiring over flake.nix as an input?

[domenkozar]

It looks like people are excited about this, we should have a test to check it works and a document page to show how it can be used :)

[hugosenari]

What part are you referring to specifically?

Expectation

devShells.x86_64-linux.default = devenv.lib.mkShell {
  inherit inputs pkgs;
  modules = [ ./myModule.nix  ];
};

Reality:

    let
      systems = [ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-linux" "aarch64-darwin" ];
      forAllSystems = f: builtins.listToAttrs (map (name: { inherit name; value = f name; }) systems);
    in
    {
      devShells = forAllSystems
        (system:
          let
            pkgs = import nixpkgs {
              inherit system;
            };
          in
          {
            default = devenv.lib.mkShell {
              inherit inputs pkgs;
              modules = [ ./myModule.nix  ];
            };
          });
    };

I think most in cases developer will not require other devShells then maybe we can assume default:

    let
      systems = [ "x86_64-linux" "i686-linux" "x86_64-darwin" "aarch64-linux" "aarch64-darwin" ];
      forAllSystems = f: builtins.listToAttrs (map (name: { inherit name; value = f name; }) systems);
    in
    {
      devShells = forAllSystems
        (system:
          let
            pkgs = import nixpkgs {
              inherit system;
            };
          in
            devenv.lib.mkShell {
              inherit inputs pkgs;
              modules = [ ./myModule.nix  ];
            });
    };

The systems you defined in template can also be assumed as default or we could add all without drawbacks, then we can hide forAllSystems

    {
      devShells = devenv.lib.mkShellForAll {
         inherit inputs;
         modules = [ ./myModule.nix  ];
      };
    };

As default statement re applied, we could assume it returns devShells

    outputs = { devenv, ... }@inputs: devenv.lib.devShells {
       inherit inputs;
       modules = [ ./myModule.nix  ];
    };

That can be used like this if any user wants defines its own other flake outputs attrs

    outputs = { devenv, ...}@inputs: 
      devenv.lib.devShells {
        inherit inputs;
        modules = [ ./myModule.nix  ];
      }
      // # ... we don't care how following value are generated
      { 
        checks."<system>"."<name>" = derivation;
        packages."<system>"."<name>" = derivation;
        apps."<system>"."<name>" = {};
        # ...
      }
;

We can keep going...

    outputs = inputs: inputs.devenv.lib.devShells {
       inherit inputs;
       modules = [ ./myModule.nix  ];
    };

Why stop there...

    outputs = inputs: inputs.devenv.lib.devShells inputs [ ./myModule.nix  ];

Then tell Eelco we have the only output that matters:

    outputs = [ ./myModule.nix ];

And that isn't crazy as it sounds :wink:

Ok, I can't tell what version actually better fits our users

See also:

• meme

[bobvanderlinden]

This is both scary and exciting :)

I agree :sweat_smile: Still unsure whether it's a good thing for devenv.

If we do support this, it should be documented and tested.

Right! Should I wait for #98 or do you have something else in mind?

As for documentation, what would be a good place to do so? I didn't want to clutter any of the existing getting started guides with choices 'use Nix as-is' or 'use devenv cli'. It should be somewhere separate for people who are already invested in using Nix CLI and flakes.

It will be quite a degraded experience, all the fixes that come with a fork of Nix are lost, on top of the UX cli improvements.

Fully agreed. Maybe it's good to know why I'd like to use devenv in Nix flakes.

Currently I have a 'company-wide' repository that includes extra modules specific to the applications I work on. This was based on dev-shell. However, I ran into limitations of dev-shell, namely having make, cc, pkg-config, etc available for building native Ruby gems or JS packages. devenv solves this part of the problem.

In addition, I find it a good idea in general to simplify usage of devenv by having a custom devenv cli in addition to using its own Nix CLI to extend some of the functionality. It makes it easier to pick up for newcomers and I think it'll help adoption of devenv in general compared to dev-shell.

What I am missing though is integration with the rest of the Nix ecosystem. I'd like to define modules to extend existing functionality. That part needs to be done using Nix flakes. To be able to instruct collegues to do so, I'd have to explain to install/use devenv as well as explain Nix, with experimental-features and how flakes work. Since I'm already explaining flakes, I'd like to lessen confusion by also introducing a separate command.

Flakes do allow adding new packages. Having a common home-manager module. Introducing common functions (in lib). That is currently not possible with just devenv. Maintaining these do require Nix/flake knowledge.

Lastly, I currently have some projects where the maintainer does not know about flakes, but I do want to introduce a devshell for these. That's what I'm doing with the single flake repository that includes multiple devshells. I can nix develop mycompanyrepo#myproject to open the devShell for myproject.

I think a better approach will be for devenv.nix to use the local flake.nix as an input to consume packages, etc - this is something I'm working on.

Nice :+1: I'm interested!

Would you prefer flake.nix wiring over flake.nix as an input?

Not 100% sure what you mean here, but since I have explained (and know) a bit about flakes, I'd like to expand on that knowledge. Especially since devenv only cares about development environments and not the many other use-cases Nix flakes covers. Nothing needs to change about this I think, because for most people it's already a huge win to have development dependencies be locked in place with an easy to use cli.

@hugosenari

Ah, like that. It can indeed be simpler. I thought this was the most straightforward and often used structure for a flake. Something that is clear for someone already familiar with flakes. When someone is not familiar with flakes I presume they will adopt devenv's way of working.

That said, I do like having some helper functions to simplify the flake definition. However, I do think mkShell (and maybe mkConfig) still need to be there as well so that you can create multiple devShells. And do so with custom pkgs. It is needed to change nixpkgs's config to set allowInsure and allowUnfree, in addition to extending/overriding with overlays.

[domenkozar]

This is both scary and exciting :)

I agree sweat_smile Still unsure whether it's a good thing for devenv.

Others have convinced me it is :)

If we do support this, it should be documented and tested.

Right! Should I wait for #98 or do you have something else in mind?

For now you can add something to scripts.devenv-run-tests just to make sure our tests will catch any regressions.

As for documentation, what would be a good place to do so? I didn't want to clutter any of the existing getting started guides with choices 'use Nix as-is' or 'use devenv cli'. It should be somewhere separate for people who are already invested in using Nix CLI and flakes.

I'd start a new section called guides for this. Something like docs/guides/using-with-flakes-devshell.md.

It will be quite a degraded experience, all the fixes that come with a fork of Nix are lost, on top of the UX cli improvements.

Fully agreed. Maybe it's good to know why I'd like to use devenv in Nix flakes.

Currently I have a 'company-wide' repository that includes extra modules specific to the applications I work on. This was based on dev-shell. However, I ran into limitations of dev-shell, namely having make, cc, pkg-config, etc available for building native Ruby gems or JS packages. devenv solves this part of the problem.

In addition, I find it a good idea in general to simplify usage of devenv by having a custom devenv cli in addition to using its own Nix CLI to extend some of the functionality. It makes it easier to pick up for newcomers and I think it'll help adoption of devenv in general compared to dev-shell.

What I am missing though is integration with the rest of the Nix ecosystem. I'd like to define modules to extend existing functionality. That part needs to be done using Nix flakes. To be able to instruct collegues to do so, I'd have to explain to install/use devenv as well as explain Nix, with experimental-features and how flakes work. Since I'm already explaining flakes, I'd like to lessen confusion by also introducing a separate command.

That's something that still needs to be done on the devenv side. While devenv allows consuming flakes, this PR allows devenv to be consumed by flakes :+1:

Flakes do allow adding new packages. Having a common home-manager module. Introducing common functions (in lib). That is currently not possible with just devenv. Maintaining these do require Nix/flake knowledge.

That's going to change in the upcoming months, but step by step :)

Lastly, I currently have some projects where the maintainer does not know about flakes, but I do want to introduce a devshell for these. That's what I'm doing with the single flake repository that includes multiple devshells. I can nix develop mycompanyrepo#myproject to open the devShell for myproject.

I need to write a guide how to create custom options for devenv.nix and how to share that around.

I think a better approach will be for devenv.nix to use the local flake.nix as an input to consume packages, etc - this is something I'm working on.

Nice +1 I'm interested!

Would you prefer flake.nix wiring over flake.nix as an input?

Not 100% sure what you mean here, but since I have explained (and know) a bit about flakes, I'd like to expand on that knowledge. Especially since devenv only cares about development environments and not the many other use-cases Nix flakes covers. Nothing needs to change about this I think, because for most people it's already a huge win to have development dependencies be locked in place with an easy to use cli.

@hugosenari

Ah, like that. It can indeed be simpler. I thought this was the most straightforward and often used structure for a flake. Something that is clear for someone already familiar with flakes. When someone is not familiar with flakes I presume they will adopt devenv's way of working.

That said, I do like having some helper functions to simplify the flake definition. However, I do think mkShell (and maybe mkConfig) still need to be there as well so that you can create multiple devShells. And do so with custom pkgs. It is needed to change nixpkgs's config to set allowInsure and allowUnfree, in addition to extending/overriding with overlays.

[bobvanderlinden]

I've added a minimal test for using nix flake init + nix develop. I also added guide on how to use devenv with nix flake's devShells. I tried to explain this without explaining all of flakes. Let me know what you think. It includes some explanation of multiple devShells. I started with explaining using multiple devenv modules (and module files), but found I was going down the rabbit-hole of explaining everything, so I skipped multiple modules for now. For now this should be usable as a start for people who have some sense of flakes.

[roman]

Do we have instructions on how to run other devenv commands?

Wondering if I need to define a package to run devenv up for example.

This is very exciting, thanks for the PR again.

[bobvanderlinden]

Do we have instructions on how to run other devenv commands?

Wondering if I need to define a package to run devenv up for example.

This is very exciting, thanks for the PR again.

I have the following module in my flake:

{ lib, config, ... }:
{
  config = {
    scripts.up.exec = ''
      ${config.procfileScript}
    '';
  };
}

So that I can run up as a separate script. I think if devenv subcommands as options are going to be created it might be easier to add a devenv(-like) executable with those subcommands as well. Not sure yet how good of an idea this is :sweat_smile:

[bobvanderlinden]

@domenkozar thanks for the ping on Discord and your suggestions. Your suggestions have been incorporated and I rebased on main. I couldn't run devenv up docs because of a missing bin/mkdocs? I couldn't figure out how to host the docs locally yet.

[roman]

devenv(-like) executable with those subcommands as well. Not sure yet how good of an idea this is 😅

I think this is a great idea, the challenging part may be the name, ideally it would be devenv, but that would be confused with the full porcelain binary.

Having the same subcommands or most of them makes total sense to me.