search

nix-community / linuxkit-nix

An easy to use Linux builder for macOS [maintainer=@nicknovitski]
Other
132 stars 26 forks source link

Doesn't work on my system #7

Open yurrriq opened 6 years ago

yurrriq commented 6 years ago

How can I debug the following?

$ nix-store -r /nix/store/1f5zgx8qykz2fxzhqphmsfp6cvpnfc94-linuxkit-builder

warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/1f5zgx8qykz2fxzhqphmsfp6cvpnfc94-linuxkit-builder
$ nix-env -i /nix/store/1f5zgx8qykz2fxzhqphmsfp6cvpnfc94-linuxkit-builder
replacing old 'linuxkit-builder'
installing 'linuxkit-builder'
$ nix-linuxkit-configure

$HOME/.cache/nix-linuxkit-builder/gcroot
Setting up /var/root//.ssh/nix-linuxkit-ssh-config...
Password:
Ok, try it out!

    nix-build $HOME/.cache/nix-linuxkit-builder/example.nix

If this doesn't work right away, maybe wait a 10+ seconds and try again.
$ nix-build $HOME/.cache/nix-linuxkit-builder/example.nix
these derivations will be built:
  /nix/store/b2gyqwbg8nx1qlhdn2v3gj0872j7j6ar-hello-1535529157.drv
error: a 'x86_64-linux' is required to build '/nix/store/b2gyqwbg8nx1qlhdn2v3gj0872j7j6ar-hello-1535529157.drv', but I am a 'x86_64-darwin'
nicknovitski commented 5 years ago

I checked the plist file for the service.

$ cat ~/Library/LaunchAgents/org.nix-community.linuxkit-builder.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>ExitTimeOut</key>
        <integer>30</integer>
        <key>Label</key>
        <string>org.nix-community.linuxkit-builder</string>
        <key>Program</key>
        <string>/nix/store/9ri2i7r780aghp3bh6nr8sqhi9qqigqz-shellchecked-nix-linuxkit-builder</string>
        <key>RunAtLoad</key>
        <true/>
</dict>

Then I tried running the Program manually to see what the problem might be, and found that the ui.sh script was trying to execute a rust binary with a _-separated name when it actually was _-separated. This might the problem you had too; it's reproducible with the current installation instructions since the hash in the readme hasn't been updated with the recently-committed fix.

This is what I did in the meantime: 

# current instructions
nix-store -r /nix/store/1f5zgx8qykz2fxzhqphmsfp6cvpnfc94-linuxkit-builder
nix-env -i /nix/store/1f5zgx8qykz2fxzhqphmsfp6cvpnfc94-linuxkit-builder
nix-linuxkit-configure
# scripted fix
builderPlist=~/Library/LaunchAgents/org.nix-community.linuxkit-builder.plist
builderProgram=$(/usr/libexec/PlistBuddy "$builderPlist" -c "Print :Program")
/usr/libexec/PlistBuddy "$builderPlist" \
  -c "delete :Program" \
  -c "add :ProgramArguments array" \
  -c "add :ProgramArguments: string '/bin/bash'" \
  -c "add :ProgramArguments: string '-c'" \
  -c "add :ProgramArguments: string 'sed \'s|nix_linuxkit_runner|nix-linuxkit-runner|\' $builderProgram | bash'"
launchctl unload "$builderPlist"
launchctl load "$builderPlist"

I think it would also be possible to add entries to the plist file to log output/error to a file, making it a little easier to answer questions like this.

puffnfresh commented 5 years ago

Thanks for debugging this! That was a change in a more recent version of nixpkgs I think. We should update the plist!

grahamc commented 5 years ago

I just updating the install instructions to use /nix/store/v4i5gx94r2qxs91mfy8sz4mmnigzravy-linuxkit-builder which includes the fixes already in the repo for that problem. Sorry for the delay!

yurrriq commented 5 years ago

Thanks! I look forward to trying it out.

yurrriq commented 5 years ago

I finally tried this out and it still doesn't work on my system.

$ nix-build ~/.cache/nix-linuxkit-builder/example.nix
these derivations will be built:
  /nix/store/ack9492w9k7n5016af33jrw06l71dbzq-hello-1541003208.drv
these paths will be fetched (57.87 MiB download, 362.46 MiB unpacked):
  /nix/store/2wdj4rznh37jd44lxz9nn6ymnbqs9axp-diffutils-3.6
  /nix/store/3jg1sj3va3rmm4fhw5xkybq9q3v8wnyi-pcre-8.42
  /nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz
  /nix/store/5hir9w0mnrw86yv7sf4qghsard0ccwl1-glibc-2.27-bin
  /nix/store/61m8xswm9xxwb3ddw1za2cxvcikgk656-patch-2.7.6
  /nix/store/63j8mi5vjyxfc89rhjknwxaxqg3a2sx2-paxctl-0.9
  /nix/store/6yz7851vibc1xjxpiyfzqqi2ksbv6qah-binutils-wrapper-2.30
  /nix/store/9f89z51na7w931aja8lqlmhqny9h16cj-gnugrep-3.1
  /nix/store/akak0rxhbi4n87z3nx78ipv76frvj841-glibc-2.27-dev
  /nix/store/bqp3d4yhpx6h60fv8jww5id0nkm44034-patchelf-0.9
  /nix/store/g5dlpwd44kd75i71nwzii8w4bp4inxwk-findutils-4.6.0
  /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30
  /nix/store/i6vl5lwlz5jbkg4r6p340dwmj6fha3xq-stdenv-linux
  /nix/store/iw94llkj05wgaz268mlzvgx8jkbi1ss0-gcc-wrapper-7.3.0
  /nix/store/kspghzdcy9wfdikc81mm8lm2gg3zn2w4-ed-1.14.2
  /nix/store/ny5p32137wfyzdm485xfdck21w1gyl3g-gnused-4.5
  /nix/store/ql3azyviwyc6w73h6prk1j2945skfapf-gnumake-4.2.1
  /nix/store/vv4r320p5yd1k01kld62q1lppjxcswhb-gcc-7.3.0
  /nix/store/x4n2gl488ir336hd5nq992fldx7yikfg-linux-headers-4.15
  /nix/store/ynb9h0q59ykdpmsn5m2qrkmnq1p683wr-expand-response-params
  /nix/store/zdadpaj5z8k9ifkgsc16yvfjn4wdv0r2-gawk-4.2.1
...
error: a 'x86_64-linux' is required to build '/nix/store/rhczg0sbvwzw3si2s3vx9wbbhmk20rjh-hello-1541003310.drv', but I am a 'x86_64-darwin'

Is there something I can try to get it working?

Mic92 commented 5 years ago

I have a belief that you are not the only one.

zimbatm commented 5 years ago

Just got this error as well, it's as if it wasn't reading /etc/nix/machines at all. Usually when the remote builder is failing it would print a message about the connection attempt.

My mac is managed by nix-darwin which might change a few rules about how Nix is being installed.

yurrriq commented 5 years ago

My mac is managed by nix-darwin...

Mine too.. 🤔

zimbatm commented 5 years ago

nix defaults to load /etc/nix/machines unless the builders option is set in /etc/nix/nix.conf. When using nix-darwin, it sets builders = in the nix.conf by default, which explains the issue.

yurrriq commented 5 years ago

Even with this config:

{ ... }:
{
  nix.extraOptions = ''
      builders = @/etc/nix/machines
  '';
}

... I still can't get it to work.

$ nix-build ~/.cache/nix-linuxkit-builder/example.nix
these derivations will be built:
  /nix/store/zvqgv8zksk1v8i2wal6k0czd0wkanmfy-hello-1542662200.drv
cannot build on 'ssh://nix-linuxkit': cannot connect to 'nix-linuxkit': ssh: connect to host localhost port 24083: Connection refused
error: a 'x86_64-linux' is required to build '/nix/store/zvqgv8zksk1v8i2wal6k0czd0wkanmfy-hello-1542662200.drv', but I am a 'x86_64-darwin'
cbarrett commented 5 years ago

I got that error message, and running the following commands helped me:

rm -rf ~/.cache/nix-linuxkit-builder
nix-linuxkit-configure
groodt commented 5 years ago

This doesn't work for me either on Mojave 10.14.4.

I get:

these derivations will be built:
  /nix/store/7j1jycc5nxxgd6ln547rl3g5856fr8ng-hello-1559370521.drv
cannot build on 'ssh://nix-linuxkit': cannot connect to 'nix-linuxkit': ssh: Could not resolve hostname nix-linuxkit: nodename nor servname provided, or not known
error: a 'x86_64-linux' with features {} is required to build '/nix/store/7j1jycc5nxxgd6ln547rl3g5856fr8ng-hello-1559370521.drv', but I am a 'x86_64-darwin' with features {benchmark, big-parallel, nixos-test}
groodt commented 5 years ago

Running the script from the plist directly /nix/store/z2qfwj97dh2yc43x42pbrp7i1qmhbrp4-shellchecked-nix-linuxkit-builder

COM1 linked to /Users/groodt/.cache/nix-linuxkit-builder/nix-state/tty

vpnkit: [ERROR] Vmnet.Server.listen: read EOF so closing connection
vpnkit: [INFO] Vmnet.Server.listen returning Ok()
FATA[0008] Cannot run hyperkit: exit status 2
linuxkit's ending state: ExitStatus(ExitStatus(256))
Bye!
forficate commented 5 years ago

I'm having the same issue as @groodt

adrian-gierakowski commented 5 years ago

same problem as @groodt and @forficate

yurrriq commented 5 years ago

(Un)fortunately, I almost never touch a Darwin system these days, so I can't be of much help.

AlexChalk commented 5 years ago 
cannot build on 'ssh://nix-linuxkit': cannot connect to 'nix-linuxkit': ssh: Could not resolve hostname nix-linuxkit: nodename nor servname provided, or not known

Adding the ~root ssh config to my local config results in the example building.

groodt commented 5 years ago

Thanks @adc17 that worked for me too.

For anyone else finding this issue, after runninng nix-linuxkit-configure you will see that it creates an SSH config at: /var/root/.ssh/nix-linuxkit-ssh-config

Copy the contents for that SSH config into your regular SSH config located at: ~/.ssh/config

Then you should be able to build the example!

AlexChalk commented 5 years ago

Worth mentioning that I believe this will be addressed in the next release: https://github.com/nix-community/linuxkit-nix/pull/29

groodt commented 5 years ago

Good to know! It really was a puzzler, but after reading the error messages, it made a bit more sense what was going wrong.

I think fixing this in the next release will help a lot of future travellers.

shmish111 commented 4 years ago

This caught me too, could it at least be added to the README?

ysndr commented 4 years ago

Hej Is this issue supposed to be fixed by now? I still have the same problems like others in this issue and neither deleting ~/.cache/nix-linuxkit-builder and reconfiguring or even completely reinstalling made any difference. If there is a new version, I also guess the derivation referenced in the README is outdated.

adrian-gierakowski commented 4 years ago

@ysndr /nix/store/jgq3savsyyrpsxvjlrz41nx09z7r0lch-linuxkit-builder has been working for me, you can install it with:

nix-env -i /nix/store/jgq3savsyyrpsxvjlrz41nx09z7r0lch-linuxkit-builder
ysndr commented 4 years ago

@adrian-gierakowski This is actually the version that, I've got installed. I found to have the same symptom as @groodt had before, yet the solution of configuring the ssh connection in my user config did not help. In fact, as soon as I try to connect via ssh (which works as root), the service shuts down. I don't really know where to debug this, though launchctl has no way to retrieve logs has it?