Update docs so users don't configure a disabled process manager

[MatrixManAtYrService]

Describe the bug If bar depends on foo's "process_completed_successfully" then bar should not start until foo completes. Instead they both start together.

To reproduce

Include this processes attribute in devenv.nix, then run devenv up

  processes = {
    foo.exec = ''
      sleep 5
      echo foo
    '';

    bar = {
      exec = ''
        sleep 5
        echo bar
      '';
      process-compose.depends_on.foo.condition = "process_completed_successfully";
    };
  };

Here's what I see:

❯ devenv up
Building shell ...
hello from devenv
git version 2.41.0
04:45:18 system | bar.1 started (pid=69161)
04:45:18 system | foo.1 started (pid=69162)
04:45:23 foo.1  | foo
04:45:23 bar.1  | bar
04:45:23 system | foo.1 stopped (rc=0)
04:45:23 system | sending SIGTERM to bar.1 (pid 69161)
04:45:23 system | bar.1 stopped (rc=0)

Note that there is not a 5 second gap between when the two processes start, as there should be.

Version

devenv: 0.6.3

Attempted Workaround

I've had some luck just installing process-compose as a package and using a process-compose.yaml file instead of declaring the dependency in devenv.nix. That prevents the bug described above. But when I do this, I'm not sure how to make the declared processes depend on postgres, which is not part of the process-compose.yaml, but is declared in devenv.nix

[domenkozar]

Does it come in the difference of process-compose versions?

[MatrixManAtYrService]

That will be an easy question to answer once I return from some travel, it's set up on my home machine. I'll let you know in a week or so.

[penguincoder]

I have seen some similar behavior with a complex dependency tree. I have also seen process-compose keep a bunch of processes in pending state, even though they don't have any dependencies. I had to go and manually restart them with ^R a couple of times to get the stack un-wedged. This might be something I'm doing wrong, but I've seen this happen, too.

[thenonameguy]

@F1bonacc1 anecdotal evidence of some dependency tracking issues. I see it sometimes too with a simple Postgres readiness probe with 3-4 dependent process_healthy processes. I doubt the issue is devenv related.

[F1bonacc1]

If you can reproduce it with the latest version, please open a bug in the process-compose repo with a short list of repro steps. I'll gladly take a look at it.

[MatrixManAtYrService]

I cannot reproduce it when I install the latest version of process-compose, so it seems likely that it's a version thing. But then again when I add pkgs.process-compose to my list of packages and then invoke process-compose up, I'm not really testing the same thing, am I?

If there's a way to inject the latest version into devenv so that devenv up uses it, I'm not aware of that. I'd be happy to give it a shot with a little guidance though.

[domenkozar]

devenv up uses pkgs.process-compose so those two version should be exactly the same.

[MatrixManAtYrService]

I figured it out. I was passing the depends on information to process-compose but it was actually honcho that was running (I had been wondering why the UI looked different...)

This fixed it:

process-managers.honcho.enable = false;
process-managers.process-compose.enable = true;

I couldn't just enable process-compose, I also had to explicitly disable honcho, otherwise I'd get this:

error: The option `processManagerCommand' has conflicting definition values:
       - In `/nix/store/xc9cp3hl78zcwrzhp03dh7pbi2fvijs5-source/src/modules/process-managers/overmind.nix':
           ''
             OVERMIND_ENV=/nix/store/ii7pip2m2vnb7gn02kg7qj9qfdxmlkk7-procfile-env /nix/store/7d1xf24215x6ya4w6kypc2i9rmjjaf2k-overmind-2.4.0/bin/overmind start --root /home/matt/src/ar-engine --procfile /nix/store/5v9kysjdzpmfjzxlinliwzjf0vddbh72-procfile "$@" &
           ''
       - In `/nix/store/xc9cp3hl78zcwrzhp03dh7pbi2fvijs5-source/src/modules/process-managers/honcho.nix':
           ''
             /nix/store/p8da7srfdx9siza0cby78h6q5gm72xn1-honcho-1.1.0/bin/honcho start -f /nix/store/5v9kysjdzpmfjzxlinliwzjf0vddbh72-procfile --env /nix/store/ii7pip2m2vnb7gn02kg7qj9qfdxmlkk7-procfile-env "$@" &
           ''
       Use `lib.mkForce value` or `lib.mkDefault value` to change the priority on any of these definitions.
(use '--show-trace' to show detailed location information)
error: getting status of '/home/matt/src/ar-engine/.direnv/flake-profile.30942': No such file or directory

I'm unblocked and happy, thanks everyone. I'm not closing this because maybe steps could be taken to prevent others from having this problem?

[domenkozar]

We used to have it as a string, but it's now separate enable options.

Maybe honcho should be enable only if any of the other enable options are false?

[domenkozar]

Implementation: https://github.com/cachix/devenv/blob/main/src/modules/process-managers/process-compose.nix

[MatrixManAtYrService]

I like your suggestion re: honcho's enable behavior, but there's still the problem of the user configuring one manager and using the other.

Maybe it was a dumb move on my part, but when I started out I didn't know that process-compose was a separate project at all, I just thought it was a clever way for devenv to hint that this functionality will resemble docker-compose.

I bet there's a reason it's set up the way it is and I'm simply not seeing it, but as a newcomer I find it a bit strange that I can start configuring my processes in a manager-agnostic way and then I have to switch to the manager-specific way once I want to do something that extends beyond the basics. I'd find my way to the right documentation faster if the process manager in use was explicit:

 process-manager.process-compose.config = {
    foo.cmd = ''
      sleep 5
      echo foo
    '';

    bar = {
      cmd = ''
        sleep 5
        echo bar
      '';
      depends_on.foo.condition = "process_completed_successfully";
    };
  };

That way I don't have to enable or disable anything. I'm just configuring the one that I want to use. I don't know how much work I just proposed... just dreaming.

My near-term goal is something that waits for postgres to be ready and then initializes a database. It seems like that would fit right in here: https://devenv.sh/processes/ Once I have it working I can contribute an addition to that page if you think it would be helpful.

[domenkozar]

That documentation is welcome and specifically documenting that process.implementation = "process-compose" is the right way to switch.