add arm linux soft float build target ?

comicfans commented 5 years ago

Hello, there're many ARM routers which lacks hard float (for example Netgear R7000/R6300v2, running bcm 4708/4709, Cortex A9) , and they're running old kernel (can not use kernel implementation), it will be very helpful if boringtun can provide soft float build for these devices.

vkrasnov commented 5 years ago

What happens if you try to build it with --target arm-unknown-linux-gnueabi?

comicfans commented 5 years ago

I'm trying. some distro (archlinux) only provided hardfloat toolchain, makes cross compile painful. so I wonder if build CI can provide such variant directly.

----update----

successfully build binary with arm-unknown-linux-gnueabi but can't run on my router. seems that my router (tomatousb) use uclibc but my boringtun is dynamic linked to glibc.

arm-unknown-linux-musleabi can't compile due to error

Compiling boringtun v0.2.0 (/root/boringtun) error[E0308]: mismatched types --> src/device/tun_linux.rs:125:31 | 125 | if unsafe { ioctl(fd, SIOCGIFMTU, &ifr) } < 0 { | ^^^^^^^^^^ expected i32, found u32

error: aborting due to previous error

vkrasnov commented 5 years ago

Can you check #47?

comicfans commented 5 years ago

build arm-unknown-linux-musleabi but can't setup device successfully.

./boringtun -v --foreground wg0
Timer("Invalid argument")

gdb shows that program received SIGILL

Breakpoint 1 at 0xade84: file src/device/epoll.rs, line 105.
(gdb) c
Continuing.

Breakpoint 1, >::new_periodic_event (
    self=0x331c18, handler=..., period=...) at src/device/epoll.rs:105
105                 -1 => return Err(Error::Timer(errno_str())),
(gdb) s
boringtun::device::tun::errno_str () at src/device/tun_linux.rs:13
13          let strerr = unsafe { strerror(*__errno_location()) };
(gdb) s
__errno_location () at src/errno/__errno_location.c:5
5       src/errno/__errno_location.c: No such file or directory.
(gdb) n
6       in src/errno/__errno_location.c
(gdb) n
7       in src/errno/__errno_location.c
(gdb) n

Program received signal SIGILL, Illegal instruction.
0x000a0d20 in boringtun::device::tun::errno_str ()
    at src/device/tun_linux.rs:13
13          let strerr = unsafe { strerror(*__errno_location()) };

my device is running

Linux unknown 2.6.36.4brcmarm #17 SMP PREEMPT Wed Apr 19 15:29:02 CEST 2017 armv7l GNU/Linux

vkrasnov commented 5 years ago

That looks like #51

comicfans commented 5 years ago

CLOCK_BOOTTIME (requires 3.15) is not supported by 2.6, but after changed to CLOCK_MONOTONIC, boringtun still crash

root@unknown:/tmp# RUST_BACKTRACE=full ./boringtun -f wg0
thread '' panicked at 'called `Result::unwrap()` on an `Err` value: IOCtl("Resource busy")
', src/libcore/result.rs:997:5
stack backtrace:
thread '' panicked at 'called `Result::unwrap()` on an `Err` value: IOCtl("Resource busy")
', src/libcore/result.rs:997:5
thread '' panicked at 'called `Result::unwrap()` on an `Err` value: IOCtl("Resource busy")
', src/libcore/result.rs:997:5
Poll error Interrupted system call
Poll error Interrupted system call
   0:   0x2b8c97 - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::hdf951eb9f43ea27e
                       at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1:   0x2b5537 - std::sys_common::backtrace::_print::h2b5fdbbb5020ccf8
                       at src/libstd/sys_common/backtrace.rs:70
   2:   0x2b7b8f - std::panicking::default_hook::{{closure}}::h18d8c59665918d69
                       at src/libstd/sys_common/backtrace.rs:58
                       at src/libstd/panicking.rs:200
   3:   0x2b77c7 - std::panicking::default_hook::h1fd1ad969274543a
                       at src/libstd/panicking.rs:215
   4:   0x2b82fb - std::panicking::rust_panic_with_hook::h40a77253872948e8
                       at src/libstd/panicking.rs:478
   5:   0x2b7e7b - std::panicking::continue_panic_fmt::hec94fc8e5daf641b
                       at src/libstd/panicking.rs:385
   6:   0x2b7d6b - rust_begin_unwind
                       at src/libstd/panicking.rs:312
   7:   0x2daac3 - core::panicking::panic_fmt::h74ee8034b317ceed
                       at src/libcore/panicking.rs:85
   8:    0x9e66f - core::result::unwrap_failed::hb36dadfcc8acd779
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/macros.rs:16
   9:    0x9be33 - >::unwrap::h617eb29dccd6522d
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/result.rs:798
  10:    0xbeb93 - boringtun::device::DeviceHandle::event_loop::h030f918ff4d6f8f5
                       at src/device/mod.rs:189
  11:    0x4d4b3 - boringtun::device::DeviceHandle::new::{{closure}}::h8e3e4b72820b214b
                       at src/device/mod.rs:155
  12:    0xbcecb - std::sys_common::backtrace::__rust_begin_short_backtrace::h5ea805a93a43420b
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/sys_common/bac
ktrace.rs:135
  13:    0x7e217 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::hc0e7469f7b1a2c
67
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/thread/mod.rs:
469
  14:    0xad9f7 -  as core::ops::function::FnOnce<()>>::call_once:
:he8042073ae72f85b
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panic.rs:309
  15:    0x1b53f - std::panicking::try::do_call::h0ad164a7905978c8
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panicking.rs:2
97
  16:   0x2bbc57 - __rust_maybe_catch_panic
                       at src/libpanic_unwind/lib.rs:92
  17:    0x1b463 - std::panicking::try::h6377848ab231a9f5
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panicking.rs:2
76
  18:    0xada1f - std::panic::catch_unwind::h4e2430f8aabc9f87
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panic.rs:388
  19:    0x7e08b - std::thread::Builder::spawn_unchecked::{{closure}}::h07220290f9aadc7f
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/thread/mod.rs:
468
  20:    0x7e56b - >::call_box::hd1eed6082b7a88f7
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/liballoc/boxed.rs:734
  21:   0x2bb347 - std::sys::unix::thread::Thread::new::thread_start::h561dd350a6a5bdaa
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/liballoc/boxed.rs:744
                       at src/libstd/sys_common/thread.rs:14
                       at src/libstd/sys/unix/thread.rs:81
stack backtrace:
   0:   0x2b8c97 - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::hdf951eb9f43ea27e
                       at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1:   0x2b5537 - std::sys_common::backtrace::_print::h2b5fdbbb5020ccf8
                       at src/libstd/sys_common/backtrace.rs:70
   2:   0x2b7b8f - std::panicking::default_hook::{{closure}}::h18d8c59665918d69
                       at src/libstd/sys_common/backtrace.rs:58
                       at src/libstd/panicking.rs:200
   3:   0x2b77c7 - std::panicking::default_hook::h1fd1ad969274543a
                       at src/libstd/panicking.rs:215
   4:   0x2b82fb - std::panicking::rust_panic_with_hook::h40a77253872948e8
                       at src/libstd/panicking.rs:478
   5:   0x2b7e7b - std::panicking::continue_panic_fmt::hec94fc8e5daf641b
                       at src/libstd/panicking.rs:385
   6:   0x2b7d6b - rust_begin_unwind
                       at src/libstd/panicking.rs:312
   7:   0x2daac3 - core::panicking::panic_fmt::h74ee8034b317ceed
                       at src/libcore/panicking.rs:85
   8:    0x9e66f - core::result::unwrap_failed::hb36dadfcc8acd779
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/macros.rs:16
   9:    0x9be33 - >::unwrap::h617eb29dccd6522d
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/result.rs:798
  10:    0xbeb93 - boringtun::device::DeviceHandle::event_loop::h030f918ff4d6f8f5
                       at src/device/mod.rs:189
  11:    0x4d4b3 - boringtun::device::DeviceHandle::new::{{closure}}::h8e3e4b72820b214b
                       at src/device/mod.rs:155
  12:    0xbcecb - std::sys_common::backtrace::__rust_begin_short_backtrace::h5ea805a93a43420b
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/sys_common/bac
ktrace.rs:135
  13:    0x7e217 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::hc0e7469f7b1a2c
67
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/thread/mod.rs:
469
  14:    0xad9f7 -  as core::ops::function::FnOnce<()>>::call_once:
:he8042073ae72f85b
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panic.rs:309
  15:    0x1b53f - std::panicking::try::do_call::h0ad164a7905978c8
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panicking.rs:2
97
  16:   0x2bbc57 - __rust_maybe_catch_panic
                       at src/libpanic_unwind/lib.rs:92
  17:    0x1b463 - std::panicking::try::h6377848ab231a9f5
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panicking.rs:2
76
  18:    0xada1f - std::panic::catch_unwind::h4e2430f8aabc9f87
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panic.rs:388
  19:    0x7e08b - std::thread::Builder::spawn_unchecked::{{closure}}::h07220290f9aadc7f
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/thread/mod.rs:
468
  20:    0x7e56b - >::call_box::hd1eed6082b7a88f7
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/liballoc/boxed.rs:734
  21:   0x2bb347 - std::sys::unix::thread::Thread::new::thread_start::h561dd350a6a5bdaa
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/liballoc/boxed.rs:744
                       at src/libstd/sys_common/thread.rs:14
                       at src/libstd/sys/unix/thread.rs:81
stack backtrace:
   0:   0x2b8c97 - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::hdf951eb9f43ea27e
                       at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1:   0x2b5537 - std::sys_common::backtrace::_print::h2b5fdbbb5020ccf8
                       at src/libstd/sys_common/backtrace.rs:70
   2:   0x2b7b8f - std::panicking::default_hook::{{closure}}::h18d8c59665918d69
                       at src/libstd/sys_common/backtrace.rs:58
                       at src/libstd/panicking.rs:200
   3:   0x2b77c7 - std::panicking::default_hook::h1fd1ad969274543a
                       at src/libstd/panicking.rs:215
   4:   0x2b82fb - std::panicking::rust_panic_with_hook::h40a77253872948e8
                       at src/libstd/panicking.rs:478
   5:   0x2b7e7b - std::panicking::continue_panic_fmt::hec94fc8e5daf641b
                       at src/libstd/panicking.rs:385
   6:   0x2b7d6b - rust_begin_unwind
                       at src/libstd/panicking.rs:312
   7:   0x2daac3 - core::panicking::panic_fmt::h74ee8034b317ceed
                       at src/libcore/panicking.rs:85
   8:    0x9e66f - core::result::unwrap_failed::hb36dadfcc8acd779
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/macros.rs:16
   9:    0x9be33 - >::unwrap::h617eb29dccd6522d
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/result.rs:798
  10:    0xbeb93 - boringtun::device::DeviceHandle::event_loop::h030f918ff4d6f8f5
                       at src/device/mod.rs:189
  11:    0x4d4b3 - boringtun::device::DeviceHandle::new::{{closure}}::h8e3e4b72820b214b
                       at src/device/mod.rs:155
  12:    0xbcecb - std::sys_common::backtrace::__rust_begin_short_backtrace::h5ea805a93a43420b
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/sys_common/bac
ktrace.rs:135
  13:    0x7e217 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::hc0e7469f7b1a2c
67
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/thread/mod.rs:
469
  14:    0xad9f7 -  as core::ops::function::FnOnce<()>>::call_once:
:he8042073ae72f85b
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panic.rs:309
  15:    0x1b53f - std::panicking::try::do_call::h0ad164a7905978c8
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panicking.rs:2
97
  16:   0x2bbc57 - __rust_maybe_catch_panic
                       at src/libpanic_unwind/lib.rs:92
  17:    0x1b463 - std::panicking::try::h6377848ab231a9f5
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panicking.rs:2
76
  18:    0xada1f - std::panic::catch_unwind::h4e2430f8aabc9f87
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/panic.rs:388
  19:    0x7e08b - std::thread::Builder::spawn_unchecked::{{closure}}::h07220290f9aadc7f
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/thread/mod.rs:
468
  20:    0x7e56b - >::call_box::hd1eed6082b7a88f7
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/liballoc/boxed.rs:734
  21:   0x2bb347 - std::sys::unix::thread::Thread::new::thread_start::h561dd350a6a5bdaa
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/liballoc/boxed.rs:744
                       at src/libstd/sys_common/thread.rs:14
                       at src/libstd/sys/unix/thread.rs:81
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Any', src/libcore/result.rs
:997:5
stack backtrace:
   0:   0x2b8c97 - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::hdf951eb9f43ea27e
                       at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1:   0x2b5537 - std::sys_common::backtrace::_print::h2b5fdbbb5020ccf8
                       at src/libstd/sys_common/backtrace.rs:70
   2:   0x2b7b8f - std::panicking::default_hook::{{closure}}::h18d8c59665918d69
                       at src/libstd/sys_common/backtrace.rs:58
                       at src/libstd/panicking.rs:200
   3:   0x2b77c7 - std::panicking::default_hook::h1fd1ad969274543a
                       at src/libstd/panicking.rs:215
   4:   0x2b82fb - std::panicking::rust_panic_with_hook::h40a77253872948e8
                       at src/libstd/panicking.rs:478
   5:   0x2b7e7b - std::panicking::continue_panic_fmt::hec94fc8e5daf641b
                       at src/libstd/panicking.rs:385
   6:   0x2b7d6b - rust_begin_unwind
                       at src/libstd/panicking.rs:312
   7:   0x2daac3 - core::panicking::panic_fmt::h74ee8034b317ceed
                       at src/libcore/panicking.rs:85
   8:    0x9e7a3 - core::result::unwrap_failed::hc21e51f708aadfb7
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/macros.rs:16
   9:    0x9c10b - >::unwrap::h8235b69d0d3c975a
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libcore/result.rs:798
  10:    0xbe7c3 - boringtun::device::DeviceHandle::wait::hb8c8d86fd2655d0e
                       at src/device/mod.rs:167
  11:    0xa0557 - boringtun::main::hcd1878d4b4189ada
                       at src/main.rs:155
  12:    0x6e7cf - std::rt::lang_start::{{closure}}::h378379d680948afc
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  13:   0x2b7ce3 - std::panicking::try::do_call::h287c6909aadcc5f1
                       at src/libstd/rt.rs:49
                       at src/libstd/panicking.rs:297
  14:   0x2bbc57 - __rust_maybe_catch_panic
                       at src/libpanic_unwind/lib.rs:92
  15:   0x2b867b - std::rt::lang_start_internal::h747c58e2c78f4f4f
                       at src/libstd/panicking.rs:276
                       at src/libstd/panic.rs:388
                       at src/libstd/rt.rs:48
  16:    0x6e79f - std::rt::lang_start::h6f00fa225d4b8bc3
                       at /rustc/2aa4c46cfdd726e97360c2734835aa3515e8c858/src/libstd/rt.rs:64
  17:    0xa0873 - main

my router has working ocserv(another vpn server) and lsmod shows tun driver is used twice during ocserv connected . so I think tun device is working

another problem is that running without -f , boringtun reports started successfully but actually it crashed.

comicfans commented 5 years ago

child process failed at tun_linux.rs line 88, IFF_MULTI_QUEUE is only supported by kernel 3.8. by remove it and run with thread number = 1, boringtun finally bring up wg0. it prints

Poll error Interrupted system call
Poll error Interrupted system call

but not crashed

running with threads=1 (even with IFF_MULTI_QUEUE specified) is working too on my 2.6 kernel . maybe earlier tun driver ignore unknown flags, not sure about that. anyway no matter IFF_MULTI_QUEUE specified , threads >1 always failed.

vkrasnov commented 5 years ago

Thank you. What happens if you run with '—disable-multi-queue'?

comicfans commented 5 years ago

unmodified boringtun(except musl SIOCGIFMTU) can bring up wg0 if run with --disable-multi-queue (without specified thread number)

vkrasnov commented 5 years ago

So if I fix #47 and add information to README, would that suffice?

comicfans commented 5 years ago

I think it's OK, but detect kernel version and setup these automatically will be better.

paolieri commented 5 years ago

This would actually be very helpful! Most routers are ARM with 2.6 kernels.

cloudflare / boringtun

add arm linux soft float build target ? #49