search

NHAS / wag

Simple Wireguard 2FA
BSD 3-Clause "New" or "Revised" License
499 stars 27 forks source link

XdpWagFirewall error on sudo ./wag start #37

Closed bluecraank closed 1 year ago

bluecraank commented 1 year ago

I did:

sudo -u wag git clone https://github.com/NHAS/wag.git
cd /home/wag/wag
sudo -u make OR sudo make both not working
sudo ./wag start

Make output

BPF_CLANG=clang BPF_CFLAGS='-O2 -g -Wall -Werror' go generate ./...
go: downloading golang.zx2c4.com/wireguard/wgctrl v0.0.0-20230215201556-9c5414ab4bde
go: downloading golang.zx2c4.com/wireguard v0.0.0-20230304142546-b6a68cf211aa
go: downloading github.com/NHAS/webauthn v0.0.0-20230305085302-c94263588cef
go: downloading github.com/mattn/go-sqlite3 v1.14.16
go: downloading golang.org/x/crypto v0.7.0
go: downloading github.com/cilium/ebpf v0.10.0
go: downloading github.com/coreos/go-iptables v0.6.0
go: downloading github.com/mdlayher/netlink v1.7.1
go: downloading golang.org/x/sys v0.6.0
go: downloading github.com/boombuler/barcode v1.0.1
go: downloading github.com/pquerna/otp v1.4.0
go: downloading github.com/zitadel/oidc v1.13.0
go: downloading github.com/josharian/native v1.1.0
go: downloading github.com/mdlayher/socket v0.4.0
go: downloading golang.org/x/net v0.8.0
go: downloading github.com/golang-jwt/jwt/v4 v4.5.0
go: downloading github.com/golang-jwt/jwt v3.2.2+incompatible
go: downloading github.com/google/go-tpm v0.3.3
go: downloading github.com/google/uuid v1.3.0
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading golang.org/x/oauth2 v0.6.0
go: downloading gopkg.in/square/go-jose.v2 v2.6.0
go: downloading github.com/gorilla/securecookie v1.1.1
go: downloading golang.org/x/text v0.8.0
go: downloading golang.org/x/sync v0.1.0
go: downloading github.com/mdlayher/genetlink v1.3.1
go: downloading github.com/go-webauthn/revoke v0.1.9
go: downloading github.com/fxamacker/cbor/v2 v2.4.0
go: downloading github.com/gorilla/schema v1.2.0
go: downloading github.com/x448/float16 v0.8.4
Compiled /home/wag/wag/internal/router/bpf_bpfel.o
Stripped /home/wag/wag/internal/router/bpf_bpfel.o
Wrote /home/wag/wag/internal/router/bpf_bpfel.go
Compiled /home/wag/wag/internal/router/bpf_bpfeb.o
Stripped /home/wag/wag/internal/router/bpf_bpfeb.o
Wrote /home/wag/wag/internal/router/bpf_bpfeb.go
cd ui/src; npm install; gulp build
npm WARN deprecated source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated source-map-resolve@0.5.3: See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated chokidar@2.1.8: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
npm WARN deprecated popper.js@1.16.1: You can find the new Popper v2 at @popperjs/core, this package is dedicated to the legacy v1

added 491 packages, and audited 492 packages in 15s

29 packages are looking for funding
  run `npm fund` for details

7 high severity vulnerabilities

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
[10:42:20] Using gulpfile ~/wag/ui/src/gulpfile.js
[10:42:20] Starting 'build'...
[10:42:20] Starting 'clean'...
[10:42:20] Finished 'clean' after 26 ms
[10:42:20] Starting 'modules'...
[10:42:20] Finished 'modules' after 145 ms
[10:42:20] Starting 'js'...
[10:42:20] Starting 'scss'...
[10:42:20] Finished 'js' after 364 ms
Deprecation Warning: Using / for division outside of calc() is deprecated and will be removed in Dart Sass 2.0.0.

Recommendation: math.div($spacer, 2) or calc($spacer / 2)

More info and automated migrator: https://sass-lang.com/d/slash-div

    ╷
302 │ $headings-margin-bottom:      $spacer / 2 !default;
    │                               ^^^^^^^^^^^
    ╵
    vendor/bootstrap/scss/_variables.scss 302:31  @import
    vendor/bootstrap/scss/bootstrap.scss 9:9      @import
    scss/sb-admin-2.scss 5:9                      root stylesheet

Deprecation Warning: Using / for division outside of calc() is deprecated and will be removed in Dart Sass 2.0.0.

Recommendation: math.div($input-padding-y, 2) or calc($input-padding-y / 2)

More info and automated migrator: https://sass-lang.com/d/slash-div

    ╷
498 │ $input-height-inner-quarter:            add($input-line-height * .25em, $input-padding-y / 2) !default;
    │                                                                         ^^^^^^^^^^^^^^^^^^^^
    ╵
    vendor/bootstrap/scss/_variables.scss 498:73  @import
    vendor/bootstrap/scss/bootstrap.scss 9:9      @import
    scss/sb-admin-2.scss 5:9                      root stylesheet

Deprecation Warning: Using / for division outside of calc() is deprecated and will be removed in Dart Sass 2.0.0.

Recommendation: math.div($custom-control-indicator-size, 2) or calc($custom-control-indicator-size / 2)

More info and automated migrator: https://sass-lang.com/d/slash-div

    ╷
568 │ $custom-switch-indicator-border-radius:         $custom-control-indicator-size / 2 !default;
    │                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
    vendor/bootstrap/scss/_variables.scss 568:49  @import
    vendor/bootstrap/scss/bootstrap.scss 9:9      @import
    scss/sb-admin-2.scss 5:9                      root stylesheet

Deprecation Warning: Using / for division outside of calc() is deprecated and will be removed in Dart Sass 2.0.0.

Recommendation: math.div($spacer, 2) or calc($spacer / 2)

More info and automated migrator: https://sass-lang.com/d/slash-div

    ╷
713 │ $nav-divider-margin-y:              $spacer / 2 !default;
    │                                     ^^^^^^^^^^^
    ╵
    vendor/bootstrap/scss/_variables.scss 713:37  @import
    vendor/bootstrap/scss/bootstrap.scss 9:9      @import
    scss/sb-admin-2.scss 5:9                      root stylesheet

Deprecation Warning: Using / for division outside of calc() is deprecated and will be removed in Dart Sass 2.0.0.

Recommendation: math.div($spacer, 2) or calc($spacer / 2)

More info and automated migrator: https://sass-lang.com/d/slash-div

    ╷
718 │ $navbar-padding-y:                  $spacer / 2 !default;
    │                                     ^^^^^^^^^^^
    ╵
    vendor/bootstrap/scss/_variables.scss 718:37  @import
    vendor/bootstrap/scss/bootstrap.scss 9:9      @import
    scss/sb-admin-2.scss 5:9                      root stylesheet

Warning: 64 repetitive deprecation warnings omitted.

[10:42:23] gulp-autoprefixer:
  autoprefixer: /home/wag/wag/ui/src/scss/sb-admin-2.css:3269:3: Replace color-adjust to print-color-adjust. The color-adjust shorthand is currently deprecated.
[10:42:23] Finished 'scss' after 3.24 s
[10:42:23] Starting 'css'...
[10:42:24] Finished 'css' after 676 ms
[10:42:24] Finished 'build' after 4.1 s
go build -ldflags="-X 'github.com/NHAS/wag/internal/config.Version=v5.1.1'"

Versions

WAG:
newest (5.1.1)

GO:
go version go1.20.2 linux/amd64

Clang:
Ubuntu clang version 14.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64

Error

2023/03/10 10:43:38 can do migrations, backing up database to  devices.db.20230310104338.bak
2023/03/10 10:43:38 Running migration:  202206251953_inital.sql
2023/03/10 10:43:38 Running migration:  202210171955_rename_totp_devices_table.sql
2023/03/10 10:43:38 Running migration:  20221130174858_registration_update_device.sql
2023/03/10 10:43:38 Running migration:  20221130203322_multi_device.sql
2023/03/10 10:43:38 Running migration:  20221217161441_add_mfa_type.sql
2023/03/10 10:43:38 Running migration:  20221224115527_add_registration_groups.sql
2023/03/10 10:43:38 Running migration:  20230123215232_add_ui_users.sql
2023/03/10 10:43:38 Running migration:  20230211004046_preshared_key.sql
; int xdp_wag_firewall(struct xdp_md *ctx)
0: (b7) r2 = 0
; struct ip ip_info = {0};
1: (63) *(u32 *)(r10 -28) = r2
last_idx 1 first_idx 0
regs=4 stack=0 before 0: (b7) r2 = 0
2: (63) *(u32 *)(r10 -32) = r2
3: (63) *(u32 *)(r10 -36) = r2
4: (b7) r6 = 1
; void *data_end = (void *)(long)ctx->data_end;
5: (61) r2 = *(u32 *)(r1 +4)
; void *data = (void *)(long)ctx->data;
6: (61) r1 = *(u32 *)(r1 +0)
; if ((void *)(ip + 1) > data_end)
7: (bf) r3 = r1
8: (07) r3 += 20
; if ((void *)(ip + 1) > data_end)
9: (2d) if r3 > r2 goto pc+244
 R1_w=pkt(id=0,off=0,r=20,imm=0) R2_w=pkt_end(id=0,off=0,imm=0) R3_w=pkt(id=0,off=20,r=20,imm=0) R6_w=inv1 R10=fp0 fp-32=00000000 fp-40=0000????
; if (ip->version != 4)
10: (71) r3 = *(u8 *)(r1 +0)
; if (ip->version != 4)
11: (57) r3 &= 240
; if (ip->version != 4)
12: (55) if r3 != 0x40 goto pc+241
 R1_w=pkt(id=0,off=0,r=20,imm=0) R2_w=pkt_end(id=0,off=0,imm=0) R3_w=inv64 R6_w=inv1 R10=fp0 fp-32=00000000 fp-40=0000????
; ip_info->proto = ip->protocol;
13: (71) r3 = *(u8 *)(r1 +9)
14: (b7) r8 = 0
; ip_info->src_port = 0;
15: (6b) *(u16 *)(r10 -36) = r8
last_idx 15 first_idx 0
regs=100 stack=0 before 14: (b7) r8 = 0
; ip_info->proto = ip->protocol;
16: (63) *(u32 *)(r10 -24) = r3
; switch (ip->protocol)
17: (15) if r3 == 0x1 goto pc+11
 R1=pkt(id=0,off=0,r=20,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=inv1 R8=invP0 R10=fp0 fp-24=????mmmm fp-32=00000000 fp-40=0000????
18: (15) if r3 == 0x6 goto pc+18
 R1=pkt(id=0,off=0,r=20,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=inv1 R8=invP0 R10=fp0 fp-24=????mmmm fp-32=00000000 fp-40=0000????
19: (55) if r3 != 0x11 goto pc+31
 R1=pkt(id=0,off=0,r=20,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=inv17 R6=inv1 R8=invP0 R10=fp0 fp-24=????mmmm fp-32=00000000 fp-40=0000????
; struct udphdr *udph = (data + (ip->ihl * 4));
20: (71) r3 = *(u8 *)(r1 +0)
; struct udphdr *udph = (data + (ip->ihl * 4));
21: (67) r3 <<= 2
22: (57) r3 &= 60
; struct udphdr *udph = (data + (ip->ihl * 4));
23: (bf) r4 = r1
24: (0f) r4 += r3
last_idx 24 first_idx 17
regs=8 stack=0 before 23: (bf) r4 = r1
regs=8 stack=0 before 22: (57) r3 &= 60
regs=8 stack=0 before 21: (67) r3 <<= 2
regs=8 stack=0 before 20: (71) r3 = *(u8 *)(r1 +0)
; if (udph + 1 > (struct udphdr *)data_end)
25: (bf) r5 = r4
26: (07) r5 += 8
; if (udph + 1 > (struct udphdr *)data_end)
27: (2d) if r5 > r2 goto pc+226
 R1=pkt(id=0,off=0,r=20,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=invP(id=0,umax_value=60,var_off=(0x0; 0x3c)) R4=pkt(id=1,off=0,r=8,umax_value=60,var_off=(0x0; 0x3c)) R5=pkt(id=1,off=8,r=8,umax_value=60,var_off=(0x0; 0x3c)) R6=inv1 R8=invP0 R10=fp0 fp-24=????mmmm fp-32=00000000 fp-40=0000????
28: (05) goto pc+16
; struct udphdr *udph = (data + (ip->ihl * 4));
45: (bf) r2 = r1
46: (0f) r2 += r3
;
47: (69) r8 = *(u16 *)(r2 +2)
invalid access to packet, off=2 size=2, R2(id=2,off=2,r=0)
R2 offset is outside of the packet
processed 32 insns (limit 1000000) max_states_per_insn 0 total_states 2 peak_states 2 mark_read 22023/03/10 10:43:38 Removing Firewall rules...
2023/03/10 10:43:38 unable to start router: loading objects: field XdpWagFirewall: program xdp_wag_firewall: load program: permission denied: invalid access to packet, off=2 size=2, R2(id=2,off=2,r=0): R2 offset is outside of the packet (68 line(s) omitted)
NHAS commented 1 year ago

Hi there, thanks for your bug report.

Could you give me the kernel version you're running wag on? I cant replicate this on my dev machine (kernel 6.1.15-1 lts)

I would also recommend trying to build with the docker image, make docker after following the release build instructions down the end of the readme.

Does the binary release work? https://github.com/NHAS/wag/releases/download/v5.1.1/wag

If it doesnt, could you tell me what version stops working?

bluecraank commented 1 year ago

Yes binary is working. Have to check other things on monday

NHAS commented 1 year ago

Yep fair enough. I suspect your version of clang or the header files are slightly different to mine.

What version of ubuntu are you using 22.04.2 LTS?

NHAS commented 1 year ago

Sweet, found what the problem is. Apparently in later version of the kernel they made the verifier less strict.

So in the C portions of the code it was accessing packets like this: 

struct udphdr *udph = (data + ip->ihl * 4);

However I was never verifying that the ip header length that I was given was still inside the packet. In the older versions of the kernel this is pulled up as an issue (and honestly I should be checking it anyway)

So I've moved it to this:

    __u64 ip_header_length = (ip->ihl * 4);
    if (ip_header_length > MAX_PACKET_OFF)
    {
        return 0;
    }

    if ((void *)(data + ip_header_length) > data_end)
    {
        return 0;
    }

   ...

     struct udphdr *udph = (data + ip_header_length);

  ...

Which works on the ubuntu 22.04.2 LTS VM I was testing it on. Im going to do a small release for this. Thanks for finding i!