search

rustybird / corridor

Tor traffic whitelisting gateway
ISC License
67 stars 6 forks source link

corridor causes Tor assertion failure on Qubes-Fedora-26-minimal standalone proxyVM #41

Closed entr0py closed 6 years ago

entr0py commented 6 years ago

installed per README#Qubes (+ iptables package) in a fedora-26-minimal standalone proxyVM.

tor.service is error-free when systemctl disable corridor.target

journalctl:

Dec 06 02:27:07 corridor Tor[851]: Bootstrapped 100%: Done
Dec 06 02:27:08 corridor corridor-data[388]: corridor_relays updated.
Dec 06 02:28:10 corridor Tor[851]: tor_assertion_failed_(): Bug: src/or/cpuworker.c:499: cpuworker_queue_work: Assertion threadpool failed; aborting. (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug: Assertion threadpool failed in cpuworker_queue_work at src/or/cpuworker.c:499. Stack trace: (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(log_backtrace+0x43) [0x559082b81c13] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(tor_assertion_failed_+0x91) [0x559082b9aa51] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(cpuworker_queue_work+0x6f) [0x559082b3d8ef] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(consdiffmgr_add_consensus+0x2fb) [0x559082b2f96b] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(networkstatus_set_current_consensus+0x7ec) [0x559082a70a8c] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(connection_dir_reached_eof+0x1354) [0x559082b465b4] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(+0x1083d9) [0x559082b1f3d9] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(+0x4d7ee) [0x559082a647ee] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /lib64/libevent-2.0.so.5(event_base_loop+0x7a9) [0x777311fb53f9] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(do_main_loop+0x29d) [0x559082a6578d] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(tor_main+0xe25) [0x559082a685a5] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(main+0x19) [0x559082a61009] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor kernel: audit: type=1701 audit(1512527290.060:105): auid=4294967295 uid=993 gid=989 ses=4294967295 pid=851 comm="tor" exe="/usr/bin/tor" sig=6
Dec 06 02:28:10 corridor audit[851]: ANOM_ABEND auid=4294967295 uid=993 gid=989 ses=4294967295 pid=851 comm="tor" exe="/usr/bin/tor" sig=6
Dec 06 02:28:10 corridor Tor[851]: Bug:     /lib64/libc.so.6(__libc_start_main+0xea) [0x777310e9488a] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor Tor[851]: Bug:     /usr/bin/tor(_start+0x2a) [0x559082a6105a] (on Tor 0.3.1.8 ad5027f7dc790624)
Dec 06 02:28:10 corridor kernel: audit: type=1131 audit(1512527290.064:106): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=tor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Dec 06 02:28:10 corridor audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=tor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Dec 06 02:28:10 corridor systemd[1]: tor.service: Main process exited, code=killed, status=6/ABRT
Dec 06 02:28:10 corridor systemd[1]: tor.service: Unit entered failed state.
Dec 06 02:28:10 corridor systemd[1]: tor.service: Failed with result 'signal'.
Dec 06 02:28:11 corridor corridor-data[388]: 2017/12/06 02:28:11 socat[885] E connect(5, AF=1 "/var/run/tor/control", 22): Connection refused
Dec 06 02:28:11 corridor kernel: audit: type=1130 audit(1512527291.094:107): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=tor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Dec 06 02:28:11 corridor kernel: audit: type=1131 audit(1512527291.094:108): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=tor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

repeats every 10 seconds.

rustybird commented 6 years ago

Thanks! I'm able to reproduce this.

As a workaround, it seems that removing the line SETCONF DirPort="127.0.0.1:9030 NoAdvertise" from /usr/sbin/corridor-data and adding DirPort 127.0.0.1:9030 NoAdvertise to /etc/tor/torrc avoids the tor daemon crash.

entr0py commented 6 years ago

Thanks, the workaround is working for me also.

Ticket 23693: Assertion threadpool failed in cpuworker_queue_work describes the issue I was experiencing. It was reported against 0.3.1.7 but I don't think the fix made it into 0.3.1.8. I will test again with 0.3.1.9. It may not be a corridor issue at all. Strangely though, the crash only happened for me when corridor was running.

rustybird commented 6 years ago

Yup, 0.3.1.9 includes the Make changes in server_mode() affect workers; fix a crash. commit, which should fix this bug.

Strangely though, the crash only happened for me when corridor was running.

That makes sense: tor starts in client mode, then corridor-data dynamically reconfigures it to server mode (using the SETCONF command).

I think it's safe to close this bug. Feel free to reopen if the problem still occurs in 0.3.1.9 or any other version that merged the bugfix commit.

rustybird commented 6 years ago

Ugh, still present in 0.3.1.9

rustybird commented 6 years ago

Fixed in tor 0.3.4.1 alpha, 0.3.3.6 stable, and presumably the upcoming 0.3.2.11 and 0.3.1.11 releases - see ticket 23693