search

opnsense / update

OPNsense update tools
https://opnsense.org/
BSD 2-Clause "Simplified" License
128 stars 78 forks source link

opnsense-update does not notice a connection to the mirror isn't working #90

Closed hboetes closed 3 months ago

hboetes commented 8 months ago

It would be nice if opnsense-update would notice no data is coming from the mirror, instead of simply producing dots ad infinitum.

fichtner commented 8 months ago

We‘re using FreeBSD base fetch utility with sensible timeouts. Apparently the tool cannot detect a timeout in certain instances and these are likely related to network deployment circumstances.

hboetes commented 8 months ago

True, our firewall is behind a very restrictive firewall, nothing is allowed except for access to our vpn server via the vpn port.

# ping xs4all.nl
ping: Unknown host
root@nighthawkfw:~ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
^C

So to apply updates, I have to set the http_proxy env var.

Sometimes I forget that, and it looks like updates are being applied since the dots are increasing. But they are not.

Hence, my kind request.

fichtner commented 8 months ago

So what does fetch even do in these scenarios? Download zero bytes but keeps trying? Why? Because I can’t explain why it will not time out… it appears like it’s being tricked into connecting to an artificial sink which it can’t distinguish from a real connection because from the client side it looks healthy / not terminated immediately.

hboetes commented 8 months ago

I added an entry in dns for pkg.opnsense.org and asked them to at least allow access to that host, to no avail.

But the resolving etc is done by the proxy. In other news, fetch from the CLI fails directly.

# fetch https://pkg.opnsense.org/releases/24.1/OPNsense-24.1-checksums-amd64.sha256
fetch: https://pkg.opnsense.org/releases/24.1/OPNsense-24.1-checksums-amd64.sha256: Connection refused
# host pkg.opnsense.org
pkg.opnsense.org has address 89.149.222.99
# ping opnsense.org
ping: Unknown host
# fetch https://opnsense.org/wp-content/themes/OPNsense/assets/img/opnsense.png
fetch: https://opnsense.org/wp-content/themes/OPNsense/assets/img/opnsense.png: Host does not resolve
# setenv http_proxy "http://10.10.8.1:3142/"
# fetch https://opnsense.org/wp-content/themes/OPNsense/assets/img/opnsense.png
opnsense.png                                          6421  B 3906 kBps    00s
# fetch https://pkg.opnsense.org/releases/24.1/OPNsense-24.1-checksums-amd64.sha256
OPNsense-24.1-checksums-amd64.sha256                   436  B  482 kBps    00s
fichtner commented 8 months ago

The interesting part is the fetch fails and aborts properly. What happens when you use opnsense-fetch instead?

hboetes commented 8 months ago

Without the proxy set opnsense-fetch returns after about a second, returning a dot.

# opnsense-fetch https://pkg.opnsense.org/releases/24.1/OPNsense-24.1-checksums-amd64.sha256
# ls OPNsense-24.1-checksums-amd64.sha256 
OPNsense-24.1-checksums-amd64.sha256
# rm OPNsense-24.1-checksums-amd64.sha256
# setenv http_proxy 
# echo $http_proxy

# opnsense-fetch https://pkg.opnsense.org/releases/24.1/OPNsense-24.1-checksums-amd64.sha256
.# ls OPNsense-24.1-checksums-amd64.sha256
ls: OPNsense-24.1-checksums-amd64.sha256: No such file or directory
fichtner commented 8 months ago

Ok that works how it should. When it hangs from the GUI it’s probably loading a base or kernel set or the .sig signature file for it. Can you seen it keep hanging with those?

hboetes commented 8 months ago

Also works like it should

# opnsense-fetch https://pkg.opnsense.org/FreeBSD:13:amd64/24.1/sets/kernel-24.1.2-amd64.txz.sig
.# opnsense-fetch https://pkg.opnsense.org/FreeBSD:13:amd64/24.1/sets/kernel-24.1.2-amd64.txz
.# ls kernel*
ls: No match.
hboetes commented 8 months ago

Ah, from the GUI I missed that part. That won't work, see: https://github.com/opnsense/update/issues/90#issuecomment-2027154823

fichtner commented 6 months ago

I've been able to produce one of those hangs while breaking IPv6 somehow during working on IPv6... dumping a bit of info here

# truss fetch -qT 5 -mo /usr/local/opnsense/changelog/changelog.txz https://pkg.opnsense.org/FreeBSD:13:amd64/24.1/sets/changelog.txz
mmap(0x0,135168,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 51999142821888 (0x2f4afe6fc000)
mprotect(0x4d545d239000,4096,PROT_READ)      = 0 (0x0)
issetugid()                  = 0 (0x0)
sigfastblock(0x1,0x4d545d23b840)         = 0 (0x0)
open("/etc/libmap.conf",O_RDONLY|O_CLOEXEC,037633770030) = 3 (0x3)
fstat(3,{ mode=-rw-r--r-- ,inode=14794,size=35,blksize=4096 }) = 0 (0x0)
read(3,"includedir /usr/local/etc/libmap"...,35) = 35 (0x23)
close(3)                     = 0 (0x0)
open("/usr/local/etc/libmap.d",O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC,0165) ERR#2 'No such file or directory'
open("/var/run/ld-elf.so.hints",O_RDONLY|O_CLOEXEC,033531741015) = 3 (0x3)
read(3,"Ehnt\^A\0\0\0\M^@\0\0\0\M^@\0\0"...,128) = 128 (0x80)
fstat(3,{ mode=-r--r--r-- ,inode=262538,size=256,blksize=4096 }) = 0 (0x0)
pread(3,"/lib:/usr/lib:/usr/lib/compat:/u"...,128,0x80) = 128 (0x80)
close(3)                     = 0 (0x0)
open("/lib/libfetch.so.6",O_RDONLY|O_CLOEXEC|O_VERIFY,00) ERR#2 'No such file or directory'
open("/usr/lib/libfetch.so.6",O_RDONLY|O_CLOEXEC|O_VERIFY,00) = 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=11036,size=77560,blksize=77824 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 51999174414336 (0x2f4b0051d000)
mmap(0x0,90112,PROT_NONE,MAP_GUARD,-1,0x0)   = 51999150960640 (0x2f4afeebf000)
mmap(0x2f4afeebf000,32768,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) = 51999150960640 (0x2f4afeebf000)
mmap(0x2f4afeec7000,45056,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x7000) = 51999150993408 (0x2f4afeec7000)
mmap(0x2f4afeed2000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x11000) = 51999151038464 (0x2f4afeed2000)
mmap(0x2f4afeed3000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x11000) = 51999151042560 (0x2f4afeed3000)
munmap(0x2f4b0051d000,4096)          = 0 (0x0)
close(3)                     = 0 (0x0)
open("/lib/libc.so.7",O_RDONLY|O_CLOEXEC|O_VERIFY,012320443000) = 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=14886,size=1867008,blksize=131072 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 51999172022272 (0x2f4b002d5000)
mmap(0x0,4112384,PROT_NONE,MAP_GUARD,-1,0x0)     = 51999157620736 (0x2f4aff519000)
mmap(0x2f4aff519000,512000,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) = 51999157620736 (0x2f4aff519000)
mmap(0x2f4aff596000,1302528,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x7c000) = 51999158132736 (0x2f4aff596000)
mmap(0x2f4aff6d4000,36864,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x1b9000) = 51999159435264 (0x2f4aff6d4000)
mmap(0x2f4aff6dd000,24576,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x1c1000) = 51999159472128 (0x2f4aff6dd000)
mmap(0x2f4aff6e3000,2236416,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,-1,0x0) = 51999159496704 (0x2f4aff6e3000)
munmap(0x2f4b002d5000,4096)          = 0 (0x0)
close(3)                     = 0 (0x0)
open("/lib/libssl.so.111",O_RDONLY|O_CLOEXEC|O_VERIFY,010524047400) ERR#2 'No such file or directory'
open("/usr/lib/libssl.so.111",O_RDONLY|O_CLOEXEC|O_VERIFY,010524047400) = 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=11668,size=611136,blksize=131072 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 51999170121728 (0x2f4b00105000)
mmap(0x0,622592,PROT_NONE,MAP_GUARD,-1,0x0)  = 51999185858560 (0x2f4b01007000)
mmap(0x2f4b01007000,233472,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) = 51999185858560 (0x2f4b01007000)
mmap(0x2f4b01040000,331776,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x38000) = 51999186092032 (0x2f4b01040000)
mmap(0x2f4b01091000,36864,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x88000) = 51999186423808 (0x2f4b01091000)
mmap(0x2f4b0109a000,20480,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x90000) = 51999186460672 (0x2f4b0109a000)
munmap(0x2f4b00105000,4096)          = 0 (0x0)
close(3)                     = 0 (0x0)
open("/lib/libcrypto.so.111",O_RDONLY|O_CLOEXEC|O_VERIFY,014232266000) = 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=14933,size=3063136,blksize=131072 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 51999199064064 (0x2f4b01c9f000)
mmap(0x0,3088384,PROT_NONE,MAP_GUARD,-1,0x0)     = 51999163969536 (0x2f4affb27000)
mmap(0x2f4affb27000,1142784,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) = 51999163969536 (0x2f4affb27000)
mmap(0x2f4affc3f000,1728512,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x117000) = 51999165116416 (0x2f4affc3f000)
mmap(0x2f4affde5000,172032,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x2bc000) = 51999166844928 (0x2f4affde5000)
mmap(0x2f4affe0f000,28672,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x2e5000) = 51999167016960 (0x2f4affe0f000)
mmap(0x2f4affe16000,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,-1,0x0) = 51999167045632 (0x2f4affe16000)
munmap(0x2f4b01c9f000,4096)          = 0 (0x0)
close(3)                     = 0 (0x0)
open("/lib/libthr.so.3",O_RDONLY|O_CLOEXEC|O_VERIFY,00) = 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=14890,size=125328,blksize=125440 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 51999199064064 (0x2f4b01c9f000)
mmap(0x0,184320,PROT_NONE,MAP_GUARD,-1,0x0)  = 51999175663616 (0x2f4b0064e000)
mmap(0x2f4b0064e000,53248,PROT_READ,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) = 51999175663616 (0x2f4b0064e000)
mmap(0x2f4b0065b000,73728,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0xc000) = 51999175716864 (0x2f4b0065b000)
mmap(0x2f4b0066d000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x1d000) = 51999175790592 (0x2f4b0066d000)
mmap(0x2f4b0066e000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x1d000) = 51999175794688 (0x2f4b0066e000)
mmap(0x2f4b00670000,45056,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,-1,0x0) = 51999175802880 (0x2f4b00670000)
munmap(0x2f4b01c9f000,4096)          = 0 (0x0)
close(3)                     = 0 (0x0)
mprotect(0x2f4aff6d4000,32768,PROT_READ)     = 0 (0x0)
mprotect(0x2f4b01091000,32768,PROT_READ)     = 0 (0x0)
munmap(0x2f4afe71a000,12288)             = 0 (0x0)
mmap(0x0,262144,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 51999189516288 (0x2f4b01384000)
mprotect(0x2f4affde5000,167936,PROT_READ)    = 0 (0x0)
sysarch(AMD64_SET_FSBASE,0x2f4afdb9efc0)     = 0 (0x0)
mprotect(0x2f4aff6d4000,32768,PROT_READ|PROT_WRITE) = 0 (0x0)
mprotect(0x2f4aff6d4000,32768,PROT_READ)     = 0 (0x0)
readlink("/etc/malloc.conf",0x2f4afdb9e6b0,1024) ERR#2 'No such file or directory'
issetugid()                  = 0 (0x0)
mmap(0x0,2097152,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(21),-1,0x0) = 51999192121344 (0x2f4b01600000)
mmap(0x0,2097152,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) = 51999208058880 (0x2f4b02533000)
mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(21),-1,0x0) = 51999215190016 (0x2f4b02c00000)
thr_self(0x2f4b02545000)             = 0 (0x0)
mmap(0x2f4addba0000,4096,PROT_NONE,MAP_ANON,-1,0x0) = 51999227297792 (0x2f4b0378c000)
rtprio_thread(RTP_LOOKUP,100326,0x2f4afdb9eaf8)  = 0 (0x0)
sysarch(AMD64_SET_FSBASE,0x2f4afdb9eaf8)     = 0 (0x0)
sigaction(SIGTHR,{ 0x2f4b00667130 SA_SIGINFO ss_t },0x0) = 0 (0x0)
sigprocmask(SIG_UNBLOCK,{ },0x0)         = 0 (0x0)
_umtx_op(0x2f4afdb9eaf0,UMTX_OP_WAKE,0x1,0x0,0x0) = 0 (0x0)
mprotect(0x0,0,PROT_NONE)            = 0 (0x0)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigfastblock(0x3,0x0)                = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)         = 0 (0x0)
sigfastblock(0x1,0x2f4b02545038)         = 0 (0x0)
getcontext(0x2f4afdb9e5e0)           = 0 (0x0)
sysarch(AMD64_GET_XFPUSTATE,0x2f4afdb9e5a8)  = 0 (0x0)
mprotect(0x2f42dd682000,4096,PROT_READ)      = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGALRM,{ 0x2f4b00666fe0 SA_SIGINFO ss_t },{ SIG_DFL 0x0 ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)         = 0 (0x0)
sigprocmask(SIG_SETMASK,{ SIGHUP|SIGINT|SIGQUIT|SIGILL|SIGTRAP|SIGABRT|SIGEMT|SIGFPE|SIGKILL|SIGBUS|SIGSEGV|SIGSYS|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
sigaction(SIGINT,{ 0x2f4b00666fe0 SA_RESETHAND|SA_SIGINFO ss_t },{ SIG_DFL SA_RESTART ss_t }) = 0 (0x0)
sigprocmask(SIG_SETMASK,{ },0x0)         = 0 (0x0)
fstatat(AT_FDCWD,"/usr/local/opnsense/changelog/changelog.txz",{ mode=-rw-r--r-- ,inode=239686,size=135200,blksize=131072 },0x0) = 0 (0x0)
ioctl(2,TIOCGETA,0x2f4afdb9f960)         = 0 (0x0)
fstatat(AT_FDCWD,"/usr/local/opnsense/changelog/changelog.txz",{ mode=-rw-r--r-- ,inode=239686,size=135200,blksize=131072 },0x0) = 0 (0x0)
setitimer(ITIMER_REAL,{ 0.000000, 5.000000 },{ 0.000000, 0.000000 }) = 0 (0x0)
fstatat(AT_FDCWD,"/etc/nsswitch.conf",{ mode=-rw-r--r-- ,inode=14977,size=262,blksize=4096 },0x0) = 0 (0x0)
open("/etc/nsswitch.conf",O_RDONLY|O_CLOEXEC,0666) = 3 (0x3)
fstat(3,{ mode=-rw-r--r-- ,inode=14977,size=262,blksize=4096 }) = 0 (0x0)
read(3,"#\n# nsswitch.conf(5) - name ser"...,4096) = 262 (0x106)
read(3,0x2f4b02562000,4096)          = 0 (0x0)
close(3)                     = 0 (0x0)
open("/etc/services",O_RDONLY|O_CLOEXEC,0666)    = 3 (0x3)
fstat(3,{ mode=-rw-r--r-- ,inode=14820,size=72588,blksize=72704 }) = 0 (0x0)
lseek(3,0x0,SEEK_CUR)                = 0 (0x0)
lseek(3,0x0,SEEK_SET)                = 0 (0x0)
read(3,"#\n# Network services, Internet "...,72704) = 72588 (0x11b8c)
close(3)                     = 0 (0x0)
__sysctl("net.routetable.0.0.5.0",6,0x0,0x2f4afdb9edd8,0x0,0) = 0 (0x0)
__sysctl("net.routetable.0.0.5.0",6,0x2f4b02578000,0x2f4afdb9edd8,0x0,0) = 0 (0x0)
socket(PF_INET6,SOCK_DGRAM|SOCK_CLOEXEC,0)   = 3 (0x3)
ioctl(3,SIOCGIFINFO_IN6,0x2f4afdb9ee50)      = 0 (0x0)
close(3)                     = 0 (0x0)
open("/etc/hosts",O_RDONLY|O_CLOEXEC,0666)   = 3 (0x3)
fstat(3,{ mode=-rw-r--r-- ,inode=14959,size=198,blksize=4096 }) = 0 (0x0)
read(3,"127.0.0.1\tlocalhost\tlocalhost."...,4096) = 198 (0xc6)
read(3,0x2f4b02562000,4096)          = 0 (0x0)
close(3)                     = 0 (0x0)
getpid()                     = 83741 (0x1471d)
issetugid()                  = 0 (0x0)
open("/etc/resolv.conf",O_RDONLY|O_CLOEXEC,0666) = 3 (0x3)
fstat(3,{ mode=-rw-r--r-- ,inode=65552,size=193,blksize=4096 }) = 0 (0x0)
fstat(3,{ mode=-rw-r--r-- ,inode=65552,size=193,blksize=4096 }) = 0 (0x0)
read(3,"domain homeoffice.local\nnameser"...,4096) = 193 (0xc1)
read(3,0x2f4b02562000,4096)          = 0 (0x0)
close(3)                     = 0 (0x0)
issetugid()                  = 0 (0x0)
socket(PF_INET6,SOCK_DGRAM|SOCK_CLOEXEC,0)   = 3 (0x3)
connect(3,{ AF_INET6 [2003:cd:8712:61f8:f690:eaff:fe00:d30c]:53 },28) = 0 (0x0)
sendto(3,"\M-N\M^Y\^A\0\0\^A\0\0\0\0\0\0"...,34,0,NULL,0) = 34 (0x22)
poll({ 3/POLLRDNORM },1,5000)            ERR#4 'Interrupted system call'
SIGNAL 14 (SIGALRM) code=SI_KERNEL
sigprocmask(SIG_SETMASK,{ SIGALRM },0x0)     = 0 (0x0)
sigreturn(0x2f4afdb9d140)            EJUSTRETURN
poll({ 3/POLLRDNORM },1,3)           = 0 (0x0)
socket(PF_INET6,SOCK_DGRAM|SOCK_CLOEXEC,0)   = 4 (0x4)
connect(4,{ AF_INET6 [fd00::de39:6fff:fed8:ee9a]:53 },28) = 0 (0x0)
sendto(4,"\M-N\M^Y\^A\0\0\^A\0\0\0\0\0\0"...,34,0,NULL,0) = 34 (0x22)
poll({ 4/POLLRDNORM },1,1000)            = 1 (0x1)
recvfrom(4,"\M-N\M^Y\M^A\M^@\0\^A\0\^A\0\0\0"...,65536,0,{ AF_INET6 [fd00::de39:6fff:fed8:ee9a]:53 },0x2f4afdb9dbcc) = 50 (0x32)
close(3)                     = 0 (0x0)
close(4)                     = 0 (0x0)
socket(PF_INET6,SOCK_DGRAM|SOCK_CLOEXEC,0)   = 3 (0x3)
connect(3,{ AF_INET6 [2003:cd:8712:61f8:f690:eaff:fe00:d30c]:53 },28) = 0 (0x0)
sendto(3,"\M-l\M-W\^A\0\0\^A\0\0\0\0\0\0"...,34,0,NULL,0) = 34 (0x22)
poll({ 3/POLLRDNORM },1,5000)            = 0 (0x0)
socket(PF_INET6,SOCK_DGRAM|SOCK_CLOEXEC,0)   = 4 (0x4)
connect(4,{ AF_INET6 [fd00::de39:6fff:fed8:ee9a]:53 },28) = 0 (0x0)
sendto(4,"\M-l\M-W\^A\0\0\^A\0\0\0\0\0\0"...,34,0,NULL,0) = 34 (0x22)
poll({ 4/POLLRDNORM },1,1000)            = 1 (0x1)
recvfrom(4,"\M-l\M-W\M^A\M^@\0\^A\0\^A\0\0\0"...,65536,0,{ AF_INET6 [fd00::de39:6fff:fed8:ee9a]:53 },0x2f4afdb9dbcc) = 62 (0x3e)
close(3)                     = 0 (0x0)
close(4)                     = 0 (0x0)
__sysctl("net.inet6.ip6.addrctlpolicy",4,0x0,0x2f4afdb9ee20,0x0,0) = 0 (0x0)
__sysctl("net.inet6.ip6.addrctlpolicy",4,0x2f4b025a1000,0x2f4afdb9ee20,0x0,0) = 0 (0x0)
socket(PF_INET6,SOCK_DGRAM|SOCK_CLOEXEC,IPPROTO_UDP) = 3 (0x3)
connect(3,{ AF_INET6 [2001:1af8:5300:a010:1::1]:1 },28) = 0 (0x0)
getsockname(3,{ AF_INET6 [2003:cd:8712:61f9:f690:eaff:fe00:d30d]:31027 },0x2f4afdb9eccc) = 0 (0x0)
ioctl(3,SIOCGIFAFLAG_IN6,0x2f4afdb9ed00)     ERR#6 'Device not configured'
close(3)                     = 0 (0x0)
socket(PF_INET,SOCK_DGRAM|SOCK_CLOEXEC,IPPROTO_UDP) = 3 (0x3)
connect(3,{ AF_INET 89.149.222.99:1 },16)    = 0 (0x0)
getsockname(3,{ AF_INET 192.168.178.55:12930 },0x2f4afdb9eccc) = 0 (0x0)
close(3)                     = 0 (0x0)
socket(PF_INET6,SOCK_STREAM,0)           = 3 (0x3)
connect(3,{ AF_INET6 [2001:1af8:5300:a010:1::1]:443 },28) = 0 (0x0)
fcntl(3,F_SETFD,FD_CLOEXEC)          = 0 (0x0)
setsockopt(3,SOL_SOCKET,SO_NOSIGPIPE,0x2f4afdb9f20c,4) = 0 (0x0)
mmap(0x0,139264,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 51999229161472 (0x2f4b03953000)
__sysctl("kern.ostype",2,0x4d545d23b19a,0x2f4afdb9d7c0,0x0,0) = 0 (0x0)
__sysctl("kern.hostname",2,0x4d545d23b29a,0x2f4afdb9d7c0,0x0,0) = 0 (0x0)
__sysctl("kern.osrelease",2,0x4d545d23b39a,0x2f4afdb9d7c0,0x0,0) = 0 (0x0)
__sysctl("kern.version",2,0x4d545d23b49a,0x2f4afdb9d7c0,0x0,0) = 0 (0x0)
__sysctl("hw.machine",2,0x4d545d23b59a,0x2f4afdb9d7c0,0x0,0) = 0 (0x0)
open("/lib/libcrypto.so.111",O_RDONLY|O_CLOEXEC|O_VERIFY,06014026556) = 4 (0x4)
fstat(4,{ mode=-r--r--r-- ,inode=14933,size=3063136,blksize=131072 }) = 0 (0x0)
close(4)                     = 0 (0x0)
openat(AT_FDCWD,"/dev/crypto",O_RDWR,00)     ERR#2 'No such file or directory'
issetugid()                  = 0 (0x0)
open("/etc/ssl/openssl.cnf",O_RDONLY,0666)   = 4 (0x4)
fstat(4,{ mode=-rw-r--r-- ,inode=14833,size=10921,blksize=11264 }) = 0 (0x0)
read(4,"# $FreeBSD$\n#\n# OpenSSL exampl"...,11264) = 10921 (0x2aa9)
read(4,0x2f4b025f4000,11264)             = 0 (0x0)
close(4)                     = 0 (0x0)
open("/usr/lib/libssl.so.111",O_RDONLY|O_CLOEXEC|O_VERIFY,06013267162) = 4 (0x4)
fstat(4,{ mode=-r--r--r-- ,inode=11668,size=611136,blksize=131072 }) = 0 (0x0)
close(4)                     = 0 (0x0)
getpid()                     = 83741 (0x1471d)
getrandom("\M-C\M^K@\M-Z\M^_&\M-*i\M-;\M-49"...,32,0) = 32 (0x20)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
access("/usr/local/etc/ssl/cert.pem",R_OK)   = 0 (0x0)
open("/usr/local/etc/ssl/cert.pem",O_RDONLY,0666) = 4 (0x4)
fstat(4,{ mode=-rw-r--r-- ,inode=262288,size=1581579,blksize=131072 }) = 0 (0x0)
read(4,"##\n##  GlobalSign Root R46\n##"...,131072) = 131072 (0x20000)
read(4,"NsYXNzIDIwggEiMA0G\nCSqGSIb3DQEB"...,131072) = 131072 (0x20000)
read(4,"              2c:24:b8:c5:2c:5c:"...,131072) = 131072 (0x20000)
read(4,"              0b:69:f3:b6:4e:a8:"...,131072) = 131072 (0x20000)
read(4," D-TRUST BR Root CA 1 2020\n##\n"...,131072) = 131072 (0x20000)
read(4,":\n                    63:84:02:"...,131072) = 131072 (0x20000)
read(4,", O = Dhimyotis, CN = Certigna\n"...,131072) = 131072 (0x20000)
read(4,"     CA:TRUE\n            X509v3"...,131072) = 131072 (0x20000)
read(4,"m7LcT4Tps\n-----END CERTIFICATE-"...,131072) = 131072 (0x20000)
read(4,"c Constraints: critical\n       "...,131072) = 131072 (0x20000)
read(4,"7:c9:0d:50:\n         ea:0d:84:7"...,131072) = 131072 (0x20000)
read(4,"4:e7:63:2c:f3:8c:16:53:7e:5c:fb:"...,131072) = 131072 (0x20000)
mmap(0x0,2621440,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_ALIGNED(12),-1,0x0) = 51999244238848 (0x2f4b047b4000)
read(4,"hkiG9w0BAQsF\nAAOCAQEAtXP4A9xZWx"...,131072) = 8715 (0x220b)
read(4,0x2f4b02632380,131072)            = 0 (0x0)
close(4)                     = 0 (0x0)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
getpid()                     = 83741 (0x1471d)
write(3,"\^V\^C\^A\^B\0\^A\0\^A\M-|\^C\^C"...,517) = 517 (0x205)
read(3,"\^V\^C\^C\0z",5)             = 5 (0x5)
read(3,"\^B\0\0v\^C\^Cd\M-*\M-"\M^Cg<"...,122)   = 122 (0x7a)
read(3,"\^T\^C\^C\0\^A",5)           = 5 (0x5)
read(3,"\^A",1)                  = 1 (0x1)
read(3,"\^W\^C\^C\0\^[",5)           = 5 (0x5)
read(3,"5\M-I\M-i\M^@\M-nP=\M-r\M-6L\^X"...,27)  = 27 (0x1b)
read(3,"\^W\^C\^C\^T\M-W",5)             = 5 (0x5)
read(3,"3\M^V\M-Ld\M->?\M^?\M-a\M^B\M-5"...,5335) = 3926 (0xf56)
read(3,"RC\M-x.\M-#b\M-~\M-qPwp6k(\M-u\v"...,1409) = 1409 (0x581)
open("/usr/share/zoneinfo/Etc/UTC",O_RDONLY,00)  = 4 (0x4)
read(4,"TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0"...,54968) = 114 (0x72)
close(4)                     = 0 (0x0)
read(3,"\^W\^C\^C\^A\^Y",5)          = 5 (0x5)
read(3,"2rU"\M-3%\M^_E2\M-_\M-g,w\M-%"...,281)   = 281 (0x119)
read(3,"\^W\^C\^C\0E",5)             = 5 (0x5)
read(3,"\f\M-]\M->W\M-!S\^\l\M^?\M-H\M^_"...,69) = 69 (0x45)
write(3,"\^T\^C\^C\0\^A\^A\^W\^C\^C\0E"...,80)   = 80 (0x50)
setsockopt(3,IPPROTO_TCP,TCP_NOPUSH,0x2f4afdb9f334,4) = 0 (0x0)
open("/etc/services",O_RDONLY|O_CLOEXEC,0666)    = 4 (0x4)
fstat(4,{ mode=-rw-r--r-- ,inode=14820,size=72588,blksize=72704 }) = 0 (0x0)
lseek(4,0x0,SEEK_CUR)                = 0 (0x0)
lseek(4,0x0,SEEK_SET)                = 0 (0x0)
read(4,"#\n# Network services, Internet "...,72704) = 72588 (0x11b8c)
close(4)                     = 0 (0x0)
poll({ 3/POLLOUT|POLLERR },1,5000)       = 1 (0x1)
write(3,"\^W\^C\^C\0G\M^D^{>\M^D\M-S|q"...,76)   = 76 (0x4c)
write(3,"\^W\^C\^C\0\^S\M-7\M-bi\M-(U\^U"...,24) = 24 (0x18)
poll({ 3/POLLOUT|POLLERR },1,5000)       = 1 (0x1)
write(3,"\^W\^C\^C\0'\M-2\M-b\M^V)\M-D"...,44)   = 44 (0x2c)
write(3,"\^W\^C\^C\0\^SZ\^]\M-+\^R\M^HB"...,24)  = 24 (0x18)
poll({ 3/POLLOUT|POLLERR },1,5000)       = 1 (0x1)
write(3,"\^W\^C\^C\0\^\\M-[\^Wt\M-W\M-j"...,33)  = 33 (0x21)
write(3,"\^W\^C\^C\0\^S\M-0\M-H\M^Vm\^_"...,24)  = 24 (0x18)
poll({ 3/POLLOUT|POLLERR },1,4999)       = 1 (0x1)
write(3,"\^W\^C\^C\0/ \M-I\M-]7\M^T\M^D4"...,52) = 52 (0x34)
write(3,"\^W\^C\^C\0\^S\M^P\M-k\M-3\M-y"...,24)  = 24 (0x18)
poll({ 3/POLLOUT|POLLERR },1,5000)       = 1 (0x1)
write(3,"\^W\^C\^C\0"\M^CM\M^U\M-i\M^T"...,39)   = 39 (0x27)
write(3,"\^W\^C\^C\0\^S\M-'\^^\M-1\M-M"...,24)   = 24 (0x18)
poll({ 3/POLLOUT|POLLERR },1,5000)       = 1 (0x1)
write(3,"\^W\^C\^C\0\^S\^C\M-#1\M-hg\M^A]"...,24) = 24 (0x18)
setsockopt(3,IPPROTO_TCP,TCP_NOPUSH,0x2f4afdb9f330,4) = 0 (0x0)
setsockopt(3,IPPROTO_TCP,TCP_NODELAY,0x2f4afdb9f330,4) = 0 (0x0)
read(3,"\^W\^C\^C\^A*",5)            = 5 (0x5)
read(3,"\M-d9@'\M^H\^N\M-X\M^U\M-ez\M^]"...,298) = 298 (0x12a)
read(3,"\^W\^C\^C\^A*",5)            = 5 (0x5)
read(3,"|\M^B|\^D\M-_\M-q\^Y\M^N\M-$P"...,298)   = 298 (0x12a)
read(3,"\^W\^C\^C \^Q",5)            = 5 (0x5)
read(3,"\vg\M-y\M-R\M-Lk\M-a\M-[\M-3B"...,8209)  = 8209 (0x2011)
setitimer(ITIMER_REAL,{ 0.000000, 0.000000 },{ 0.000000, 0.000000 }) = 0 (0x0)
fetch: write(2,"fetch: ",7)              = 7 (0x7)
transfer timed outwrite(2,"transfer timed out",18)       = 18 (0x12)

write(2,"\n",1)                  = 1 (0x1)
fetch: write(2,"fetch: ",7)              = 7 (0x7)
/usr/local/opnsense/changelog/changelog.txz appears to be truncated: 0/135200 byteswrite(2,"/usr/local/opnsense/changelog/ch"...,83) = 83 (0x53)

write(2,"\n",1)                  = 1 (0x1)
write(3,"\^W\^C\^C\0\^SF\M^FH\M-u~\M^WL"...,24)  = 24 (0x18)
madvise(0x2f4b02727000,4096,MADV_FREE)       = 0 (0x0)
close(3)                     = 0 (0x0)
exit(0x1)                   
process exit, rval = 1
fichtner commented 6 months ago

relevant line is:

/usr/local/opnsense/changelog/changelog.txz appears to be truncated: 0/135200 bytes

Apparently this only happens due to the timeout value and a DNS server not being reachable. I don't think it explains these forever-fetches though

hboetes commented 6 months ago

Hmmm weird, why didn't I mention this before? To make the GUI use a proxy, you will have to edit /usr/local/etc/pkg.conf a bit, here are my notes:

vi /usr/local/etc/pkg.conf , add
pkg_env : {
    http_proxy: "http://your-poxy-ip:port/"
}
Hint: tail /tmp/opnsense-fetch.out*
fichtner commented 3 months ago

It appears that fetch(1) -a and -T 30 don't work together in the sense that the 30 seconds is a global timeout. We've just seen something similar in https://github.com/opnsense/core/issues/7825

The original change is fa77626e7a18f but I can't find any more instances of using -a so we should try removing it.