search

NixOS / ofborg

@ofborg tooling automation https://monitoring.ofborg.org/dashboard/db/ofborg
https://ofborg.org
MIT License
232 stars 166 forks source link

How to reproduce jobs locally? -> improve logs #671

Open DavHau opened 8 months ago

DavHau commented 8 months ago

I just had a failing instance of ofborg-eval-lib-tests: https://gist.github.com/GrahamcOfBorg/85196b733b08a4c1c29909c2a1a5af42

... and I'm having a hard time trying to debug this.

Several things are missing here:

  1. Which commands were executed: The logs do not state which command was executed.
  2. Definition unclear: grepping for ofborg-eval-lib-tests in either nixpkgs or nixos/ofborg leads to zero results. Where is the job defined?
  3. Reproduce locally: There are no instructions in the logs how I can reproduce that specific job locally.

All of the above should be easy fixes and would make debugging this stuff a lot easier.

EDIT: I just noticed that it displays the command in github next to the job. So that helps. But still it would be nice to have this clearly stated in the logs as well. Also the command displayed in github nix-build --arg pkgs 'import ./. {}' ./lib/tests/release.nix doesn't actually seem to work and fails with:

error: store path 'q7375xm60a582561sq84mxyx13j79cjb-.version' starts with illegal character '.'
hraban commented 7 months ago

Related, though not exactly the same as your question: I'd love an easy, one-shot way to run an entire hermetic environment exactly equal to the ofborg runners that run as nixos/nixpkgs CI. E.g. a full nixos config so you can repro CI failures locally and iterate on it before pushing a fix.

raboof commented 7 months ago

I agree this would be very helpful.

I just noticed that it displays the command in github next to the job. So that helps. But still it would be nice to have this clearly stated in the logs as well. Also the command displayed in github nix-build --arg pkgs 'import ./. {}' ./lib/tests/release.nix doesn't actually seem to work and fails

Additionally, it seems GitHub doesn't show the ' characters, making this rather confusing.

mweinelt commented 7 months ago

Most of the commands ofBorg runs can be seen with proper tokenization in the code here:

https://github.com/NixOS/ofborg/blob/eb120ff74c155c990b4f0987649cf4b541d7a17b/ofborg/src/tasks/eval/nixpkgs.rs#L433-L562