search

slackhq / nebula

A scalable overlay networking tool with a focus on performance, simplicity and security
MIT License
14.59k stars 981 forks source link

MIPS64 #11

Closed swampmonster closed 4 years ago

swampmonster commented 5 years ago

Are there any plans on supporting mips64?

nbrownus commented 4 years ago

@swampmonster any chance you can test that branch? I don't have access to a mips64 box at the moment.

swampmonster commented 4 years ago

@swampmonster any chance you can test that branch? I don't have access to a mips64 box at the moment.

I think so. I’ll have a go and get back to you.

swampmonster commented 4 years ago

Cross compiles fine and does run but errors out on the ubiquiti device i'm trying it on. Looks to be related to something missing in the kernel on the device but i can quite work out what.

INFO[0000] Firewall rule added                           firewallRule="map[caName: caSha: direction:outgoing endPort:0 groups:[] host:any ip:<nil> proto:0 startPort:0]"
INFO[0000] Firewall rule added                           firewallRule="map[caName: caSha: direction:incoming endPort:0 groups:[] host:any ip:<nil> proto:1 startPort:0]"
INFO[0000] Firewall rule added                           firewallRule="map[caName: caSha: direction:incoming endPort:443 groups:[laptop home] host: ip:<nil> proto:6 startPort:443]"
INFO[0000] Firewall started                              firewallHash=853d3005de969aa0cb1100731e983a740ab4218f89c78189edd389ff5e05ae99
FATA[0000] protocol not available

Turning on debug logging doesn't produce any additional output that could give a hint.

rawdigits commented 4 years ago

@nbrownus do you think it is related to endinness? A quick glance at the docs seems to indicate ubiquiti gear with mips64 runs as big endian vs little.

rawdigits commented 4 years ago

@swampmonster I used to own an EdgeMax, but can't find it. Which device are you using? I'll buy one, as I'd love to see nebula run well on these.

swampmonster commented 4 years ago

endinness

# echo -n I | od -to2 | head -n1 | cut -f2 -d" " | cut -c6
0

Would seem to indicate big endian.

@swampmonster I used to own an EdgeMax, but can't find it. Which device are you using? I'll buy one, as I'd love to see nebula run well on these.

This is on a ubiquiti security gateway, https://www.ui.com/unifi-routing/usg/

nbrownus commented 4 years ago

Going to see if I can get this working in a VM.

swampmonster commented 4 years ago

Going to see if I can get this working in a VM.

If I can help out with any testing let me know

nbrownus commented 4 years ago

@swampmonster can you test this branch again? I got access to a mips64le system and it presented with the same error. Got that bit fixed and nebula is working there, curious if there is more to solve in big endian land.

swampmonster commented 4 years ago

@swampmonster can you test this branch again? I got access to a mips64le system and it presented with the same error. Got that bit fixed and nebula is working there, curious if there is more to solve in big endian land.

INFO[0000] Firewall rule added                           firewallRule="map[caName: caSha: direction:outgoing endPort:0 groups:[] host:any ip:<nil> proto:0 startPort:0]"
INFO[0000] Firewall rule added                           firewallRule="map[caName: caSha: direction:incoming endPort:0 groups:[] host:any ip:<nil> proto:1 startPort:0]"
INFO[0000] Firewall rule added                           firewallRule="map[caName: caSha: direction:incoming endPort:443 groups:[laptop home] host: ip:<nil> proto:6 startPort:443]"
INFO[0000] Firewall started                              firewallHash=853d3005de969aa0cb1100731e983a740ab4218f89c78189edd389ff5e05ae99
INFO[0000] Main HostMap created                          network=192.168.100.1/24 preferredRanges="[]"
INFO[0000] UDP hole punching enabled
FATA[0000] invalid argument

It got a little further now

nbrownus commented 4 years ago

Just added another few changes, if this doesn't work hopefully Ryans gear shows up soon :)

swampmonster commented 4 years ago

Just added another few changes, if this doesn't work hopefully Ryans gear shows up soon :)

Errors out in the same place as before sadly (same error message too).

nbrownus commented 4 years ago

Very helpful, thanks! The error is somewhere in https://github.com/slackhq/nebula/blob/mips64/tun_linux.go#L143

I'll try and read through the code more thoroughly when I get the chance, as well as improve the error messages here.

nbrownus commented 4 years ago

Just made a quick stab at error message improvement, if you get the chance, another run would be great. Should tell us which part of the activation is failing.

swampmonster commented 4 years ago

Just made a quick stab at error message improvement, if you get the chance, another run would be great. Should tell us which part of the activation is failing.

FATA[0000] failed to set tun mtu: invalid argument

nbrownus commented 4 years ago

Got access to a UBNT security gateway, did some very initial testing and the latest version of this branch looks much better.

swampmonster commented 4 years ago

Got access to a UBNT security gateway, did some very initial testing and the latest version of this branch looks much better.

Can confirm. Builds and seems to run now! Haven't tried actually connecting any clients yet but i'm assuming that should work as well.

nbrownus commented 4 years ago

This should be working on master now. Thanks for reporting!