search

openwrt / openwrt

This repository is a mirror of https://git.openwrt.org/openwrt/openwrt.git It is for reference only and is not active for check-ins. We will continue to accept Pull Requests here. They will be merged via staging trees then into openwrt.git.
Other
20.24k stars 10.48k forks source link

BPI-R4: network buffering issues (with stutters) #15747

Closed damnms closed 4 months ago

damnms commented 4 months ago

Describe the bug

I have a banana pi r4 and it worked like a charm for some months. I build the images myself all few weeks with an included configuration, so there are no changes in the configuration and only in the base system. I noticed for about a week that i have some heavy stutters when i watch IPTV from my ISP. I contacted my ISP and he told me, all looks fine on his side and no one else reported any problems. So i was pretty sure its a problem with the R4 and the openwrt version. Unfortunately, i did not save the previous image where all worked :/

I have an older image from 28.03.2024 that is running now with the exact same configuration. no problems. I will try to bisect that but... phew, that will be a huge thing and many hours of work. So i thought i'll go and ask if anyone has an idea how to figure out whats going wrong.

OpenWrt version

r25677-6ddc9fc4b5

OpenWrt release

SNAPSHOT

OpenWrt target/subtarget

mediatek/filogic

Device

Bananapi BPI-R4

Image kind

Self-built image

Steps to reproduce

build the latest version of openwrt for a banana pi r4 and use udpxy to stream to devices.

Actual behaviour

heavy stuttering and glitches

Expected behaviour

a clear video stream

Additional info

No response

Diffconfig

CONFIG_TARGET_mediatek=y
CONFIG_TARGET_mediatek_filogic=y
CONFIG_TARGET_mediatek_filogic_DEVICE_bananapi_bpi-r4=y
# CONFIG_KERNEL_WERROR is not set
CONFIG_OPENSSL_ENGINE=y
CONFIG_OPENSSL_WITH_ASM=y
CONFIG_OPENSSL_WITH_CHACHA_POLY1305=y
CONFIG_OPENSSL_WITH_CMS=y
CONFIG_OPENSSL_WITH_DEPRECATED=y
CONFIG_OPENSSL_WITH_ERROR_MESSAGES=y
CONFIG_OPENSSL_WITH_IDEA=y
CONFIG_OPENSSL_WITH_MDC2=y
CONFIG_OPENSSL_WITH_PSK=y
CONFIG_OPENSSL_WITH_SEED=y
CONFIG_OPENSSL_WITH_SRP=y
CONFIG_OPENSSL_WITH_TLS13=y
CONFIG_OPENSSL_WITH_WHIRLPOOL=y
CONFIG_PACKAGE_adblock=y
CONFIG_PACKAGE_cgi-io=y
CONFIG_PACKAGE_coreutils=y
CONFIG_PACKAGE_coreutils-sort=y
# CONFIG_PACKAGE_dnsmasq is not set
CONFIG_PACKAGE_kmod-crypto-kpp=y
CONFIG_PACKAGE_kmod-crypto-lib-chacha20=y
CONFIG_PACKAGE_kmod-crypto-lib-chacha20poly1305=y
CONFIG_PACKAGE_kmod-crypto-lib-curve25519=y
CONFIG_PACKAGE_kmod-crypto-lib-poly1305=y
CONFIG_PACKAGE_kmod-mt7615-common=y
CONFIG_PACKAGE_kmod-mt7615-firmware=y
CONFIG_PACKAGE_kmod-mt7615e=y
CONFIG_PACKAGE_kmod-mt7915e=y
CONFIG_PACKAGE_kmod-thermal=y
CONFIG_PACKAGE_kmod-udptunnel4=y
CONFIG_PACKAGE_kmod-udptunnel6=y
CONFIG_PACKAGE_kmod-wireguard=y
CONFIG_PACKAGE_libevent2=y
CONFIG_PACKAGE_liblua=y
CONFIG_PACKAGE_liblucihttp=y
CONFIG_PACKAGE_liblucihttp-lua=y
CONFIG_PACKAGE_liblucihttp-ucode=y
CONFIG_PACKAGE_libopenssl=y
CONFIG_PACKAGE_libpcap=y
CONFIG_PACKAGE_libubus-lua=y
CONFIG_PACKAGE_libunbound=y
CONFIG_PACKAGE_libunbound_ipset=y
CONFIG_PACKAGE_libunbound_libevent=y
CONFIG_PACKAGE_libunbound_libpthread=y
CONFIG_PACKAGE_lua=y
CONFIG_PACKAGE_luci=y
CONFIG_PACKAGE_luci-app-adblock=y
CONFIG_PACKAGE_luci-app-firewall=y
CONFIG_PACKAGE_luci-app-opkg=y
CONFIG_PACKAGE_luci-app-unbound=y
CONFIG_PACKAGE_luci-base=y
CONFIG_PACKAGE_luci-compat=y
CONFIG_PACKAGE_luci-lib-base=y
CONFIG_PACKAGE_luci-lib-ip=y
CONFIG_PACKAGE_luci-lib-jsonc=y
CONFIG_PACKAGE_luci-lib-nixio=y
CONFIG_PACKAGE_luci-light=y
CONFIG_PACKAGE_luci-lua-runtime=y
CONFIG_PACKAGE_luci-mod-admin-full=y
CONFIG_PACKAGE_luci-mod-network=y
CONFIG_PACKAGE_luci-mod-status=y
CONFIG_PACKAGE_luci-mod-system=y
CONFIG_PACKAGE_luci-proto-ipv6=y
CONFIG_PACKAGE_luci-proto-ppp=y
CONFIG_PACKAGE_luci-proto-wireguard=y
CONFIG_PACKAGE_luci-theme-bootstrap=y
CONFIG_PACKAGE_luci-theme-material=y
CONFIG_PACKAGE_odhcpd=y
# CONFIG_PACKAGE_odhcpd-ipv6only is not set
CONFIG_PACKAGE_odhcpd_full_ext_cer_id=0
CONFIG_PACKAGE_rpcd=y
CONFIG_PACKAGE_rpcd-mod-file=y
CONFIG_PACKAGE_rpcd-mod-iwinfo=y
CONFIG_PACKAGE_rpcd-mod-luci=y
CONFIG_PACKAGE_rpcd-mod-rrdns=y
CONFIG_PACKAGE_rpcd-mod-ucode=y
CONFIG_PACKAGE_tcpdump-mini=y
CONFIG_PACKAGE_ucode-mod-html=y
CONFIG_PACKAGE_ucode-mod-lua=y
CONFIG_PACKAGE_ucode-mod-math=y
CONFIG_PACKAGE_udpxy=y
CONFIG_PACKAGE_uhttpd=y
CONFIG_PACKAGE_uhttpd-mod-ubus=y
CONFIG_PACKAGE_unbound-control=y
CONFIG_PACKAGE_unbound-daemon=y
CONFIG_PACKAGE_wireguard-tools=y
CONFIG_TARGET_ROOTFS_PARTSIZE=104

Terms

brada4 commented 4 months ago

Helps to open issue in correct place: https://github.com/openwrt/packages/issues Worth noting package has not changed since forever https://github.com/openwrt/packages/commits/master/net/udpxy

damnms commented 4 months ago

i highly doubt its a problem with udpxy, as also netflix on my TV had buffer problems. I believe more its something in driver/kernel. But i first noticed it with udpxy. I will bisect tomorrow, lets see how far i come.

danpawlik commented 4 months ago

@damnms glad that I'm not the only one. I don't have issue with udpxy, but on the dumb_ap which is connected to the BPI-R4. It started after moving to kernel 6.6.33 (or maybe some other commit is related; my last working well build is comming from dd6bbbabd31a44a2a6fd849131b3a03b6ad53946, then I create image base on the commit https://github.com/openwrt/openwrt/commit/05aec66d5326974a636afe9fbf678a86c7f4e5c8 where I started to think, that something can be broken and then I create image base on the commit 487cc3831ce8eb412279573db17da19cec59f3da which definitely got an issue). I already mentioned that on the forum, but nobody was complaining about any issue.

EDIT: One more hint, the issue is not showing after few minutes, but maybe after an hour or two

damnms commented 4 months ago

One more hint, the issue is not showing after few minutes, but maybe after an hour or two

Yeah, thats the PITA i also had. Sometimes (but not that often) problems occured after 10-20 minutes. But luckily, on my side, they usually occured after about 1 minute. But still a PITA to bisect ;) Thanks for the hashes, i will test that.

https://github.com/openwrt/openwrt/commit/dd6bbbabd31a44a2a6fd849131b3a03b6ad53946 is running now for about 20 or 25 minutes without problems!

danpawlik commented 4 months ago

@damnms if all would be working for a while, please change the topic to other.

damnms commented 4 months ago

not sure what you mean with "topic to other"? i bisected now about 10 versions and ... all work :X the latest one, which should be the faulty one, runs now for about 3 minutes without stuttering but i will let it run 20 or 30 minutes.

I use the luci upgrade function and provide only the sysupgrade .itb. maybe this only occurs when flashing directly to SD. how do you upgrade? with the sysupgrade .itb? or do you perform a fresh dd on the sd?

danpawlik commented 4 months ago

I have done via SD and via upgrade - in all situation I got some troubles on dumb_ap after few minutes. I think that the issue is not related to udpxy, but because of some network buffering issues.

damnms commented 4 months ago

thats wierd, i am running now main 156f5e8f48adf12201849ad33d2b85e3d5a90ae7 without problems for about 2 minutes. in about an hour i will flash latest version with dd to the sd and see what happens. the command i use to generate the images: make dirclean && ./scripts/feeds update -a && ./scripts/feeds install -a && make menuconfig && make -j24

edit: first i thought it never upgraded, but that was wrong. running now OpenWrt SNAPSHOT r26709-156f5e8f48 / LuCI Master 24.158.03388~a6f8361

@danpawlik do you also have init7 as ISP?

i flashed now the exact same version that was running perfectly fine with a sysupgrade but with dd and now i have stutters. at least for me, that means those problems only occur somehow when using dd, not with sysupgrade. just to be 100% sure its a software problem i plugged my workstation to lan3 instead of eth2.

danpawlik commented 4 months ago

@damnms no, some local provider :)

I make the image with similar commands, but always clean the older compilation - https://openwrt.org/docs/guide-developer/toolchain/use-buildsystem 

./scripts/feeds update -a && ./scripts/feeds install -a
make menuconfig
make -j$(nproc) kernel_menuconfig
make -j$(nproc) defconfig download clean world

I will do a new image tomorrow, or take an image from the firmware-selector (yesterday I tried it and I got same issue, so I go back to https://github.com/openwrt/openwrt/commit/dd6bbbabd31a44a2a6fd849131b3a03b6ad53946 )

EDIT: @damnms could you unplug power and plug back after sysupgrade just to verify your theory ?

damnms commented 4 months ago

i will unplug and watch football in a hour, thats the ultimative stress test to make sure it stutters... or not ;)

danpawlik commented 4 months ago

@damnms that's why I have stable build that I trust on eMMC and testing on sdcard :)

And because of that I suggested to unplug power, due I was doing that to go to eMMC to make backup then go back to sdcard to restore

damnms commented 4 months ago

i am running now OpenWrt SNAPSHOT r26709-156f5e8f48 / LuCI Master 24.158.03388~a6f8361 without problems since close to an hour, so i assume its somehow gone. no idea what that was... i will close this bug but feel free to reopen

danpawlik commented 4 months ago

I can confirm, that image based on 5e7955171c seems to work normally. Thanks @damnms for creating issue.