Ignore or invert derivation status

[roberth]

Sometimes test etc are incomplete.

Possible design:

• magic attribute on derivation (passthru): ignoreFailure = true
• magic attribute expectFailure = true

expectFailure inverts the status of a derivation. This is a good idea when some feature has not been implemented yet, but the derivation represents a test case for that feature. When the feature is implemented, the attribute must be cleared, which requires manual intervention and therefore a negative status.

[angerman]

I keep wondering if I’d rather want to set the list of derivations as an attribute to ci.nix

{ commit, repo, branch, author, ... }: rec {
__expectFailure = [ ... ];
__ignoreFailure = [ ... ];
__requiredToPass = [ ... ];
}

[roberth]

This can be customized at the Nix level as desired, by calling conversion functions written in Nix. That allows to keep the evaluation interface simple.

This should also include expectShallowFailure to disallow dependency failures. We may want to detect mkShell too.

[angerman]

@roberth yes very much so! I'm always a bit concerned about the proliferation of nix code in repositories, and subsequently try to figure out how a UI for someone who couldn't care less about nix could look like.

[roberth]

This will be available in the upcoming 0.8 release of hercules-ci-agent, by setting attributes. They only have to be present on the return value of the derivation. This can be achieved with passthru or the // operator.

let
  base = mkDerivation {
    # ...
  };
in {
  regular = base;
  # mkDerivation's passthru can be used instead
  deps-only = base // { buildDependenciesOnly = true; };
  effect = base // { isEffect = true; };
  ignore-fail = base // { ignoreFailure = true; };
  require-fail = base // { requireFailure = true; };
  shell = base // { phases = ["nobuildPhase"]; };
}

You could set it on the derivation itself (as in mkDerivation { requireFailure = true; /* bad */ } instead of passthru.requireFailure), but it's not recommended because it will cause a needless rebuild and it doesn't have any meaning at the derivation level.

It will show as follows, with tooltips to explain the attribute status. In this case the derivation for the require-fail attribute failed (as expected) and ignore-fail succeeded. The latter has a warning sign because it didn't need to be ignored.

[roberth]

This is supported now, even if you track stable.