Closed robbbaxley closed 6 months ago
I should note that I realized later that none of the Mesh nodes are actually working now, only the primary (Master) node. Not even the XT9's which are running Asus firmware. All nodes are showing 'RED'. I can reach the nodes directly via IP address and this is what I see after loggin-in:
The XT8's:
The XT9's
Further testing: I installed the previous Gnuton Firmware RT-AX95Q_3004_388.5_0-gnuton1_puresqubi on ONLY the primary (master) node which restored connectivity to all remote Mesh nodes, both the XT8's and XT9's. All nodes are hard-wired.
Hi @robbbaxley I got sort of the same setup as you do, but my Meshnodes are all connected without issue after (dirty)flashing them to 3004.388.6_0-gnuton0_beta1. Difference is that you have mixed XT8 and XT9. Which one is your main node? Repeat the procedure, starting with the main node?
@dacootmeister , My main node is an XT8. I will try that but in the past, using Gnuton or Asus firmware I have never had this problem before. My process is to update the remote Mesh nodes first and then update the primary/master node last. This has always worked with no issues at all until this firmware version. I don't know what has possibly changed that would cause this issue but when I have time I will try your method...but I will need to downgrade the remote Mesh nodes first.
Edited to add screenshots:
I loaded the new beta 2, [3004.388.6_0-gnuton0_beta2], and still have the issue as the earlier Beta.
However, now, I am no longer able to downgrade to 3004.388.5_0-gnuton1 like I did for Beta 1 so I am a bit stuck for the moment.
@robbbaxley do you see any crash in the logs related to the downgrade?
@gnuton, here is the log from my downgrade attemt this morning which, strangely, worked (3rd attempt):
Feb 15 06:59:09 rc_service: httpd 8840:notify_rc stop_upgrade
Feb 15 06:59:09 usb: USB is ejected
Feb 15 06:59:09 WEBDAV_Server: daemon is stopped
Feb 15 06:59:09 WEBDAV_Server: arpping daemon is stopped
Feb 15 06:59:09 iTunes: daemon is stopped
Feb 15 06:59:09 FTP_Server: daemon is stopped
Feb 15 06:59:09 Samba_Server: smb daemon is stopped
Feb 15 06:59:09 Timemachine: daemon is stopped
Feb 15 06:59:09 WEBDAV_Server: daemon is stopped
Feb 15 06:59:10 iTunes: daemon is stopped
Feb 15 06:59:10 FTP_Server: daemon is stopped
Feb 15 06:59:10 Samba_Server: smb daemon is stopped
Feb 15 06:59:10 Timemachine: daemon is stopped
Feb 15 06:59:11 rc_service: hotplug 27826:notify_rc restart_nasapps
Feb 15 06:59:11 rc_service: waitting "stop_upgrade" via httpd ...
Feb 15 06:59:11 ntpd: Stopped ntpd
Feb 15 06:59:13 ahs: ===Terminate ahs daemon===
Feb 15 06:59:14 disk_monitor: Finish
Feb 15 06:59:14 Mastiff: Got SIGTERM
Feb 15 06:59:18 rc_service: hotplug 27844:notify_rc restart_nasapps
Feb 15 06:59:18 rc_service: waitting "restart_nasapps" via hotplug ...
Feb 15 06:59:18 FTP_Server: daemon is stopped
Feb 15 06:59:18 Samba_Server: smb daemon is stopped
Feb 15 06:59:18 Timemachine: daemon is stopped
Feb 15 06:59:19 Samba_Server: daemon is started
Feb 15 06:59:19 FTP_Server: daemon is stopped
Feb 15 06:59:19 wsdd2[27885]: Terminated received.
Feb 15 06:59:21 Samba_Server: smb daemon is stopped
Feb 15 06:59:21 Timemachine: daemon is stopped
Feb 15 06:59:21 Samba_Server: daemon is started
Feb 15 06:59:32 rc_service: httpd 8840:notify_rc start_upgrade
Feb 15 06:59:56 WEBDAV_Server: daemon is stopped
Feb 15 06:59:56 FTP_Server: daemon is stopped
Feb 15 06:59:56 wsdd2[27961]: Terminated received.
Feb 15 06:59:57 Samba_Server: smb daemon is stopped
Feb 15 06:59:57 Timemachine: daemon is stopped
Feb 15 06:59:58 kernel: Flushing net_device wl0.2.
Feb 15 06:59:58 kernel: CSIMON: CSIMON[1.1.0] Initialization
Feb 15 06:59:58 kernel: CSIMON: M2M usr already registered ...
Feb 15 06:59:59 kernel: CSIMON: CSIMON[1.1.0] Initialization
Feb 15 06:59:59 kernel: CSIMON: M2M usr already registered ...
Feb 15 07:00:01 kernel: port_generic_stop 560 skip turnning off power on eth0 here
Feb 15 07:00:01 kernel: port_generic_stop 560 skip turnning off power on eth1 here
Feb 15 07:00:01 kernel: port_generic_stop 560 skip turnning off power on eth2 here
Feb 15 07:00:02 kernel: port_generic_stop 560 skip turnning off power on eth3 here
Feb 15 07:00:02 kernel: ubi2: attaching mtd10
Feb 15 07:00:02 kernel: ubi2: scanning is finished
Feb 15 07:00:02 kernel: ubi2: attached mtd10 (name "misc1", size 8 MiB)
Feb 15 07:00:02 kernel: ubi2: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
Feb 15 07:00:02 kernel: ubi2: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
Feb 15 07:00:02 kernel: ubi2: VID header offset: 2048 (aligned 2048), data offset: 4096
Feb 15 07:00:02 kernel: ubi2: good PEBs: 64, bad PEBs: 0, corrupted PEBs: 0
Feb 15 07:00:02 kernel: ubi2: user volume: 1, internal volumes: 1, max. volumes count: 128
Feb 15 07:00:02 kernel: ubi2: max/mean erase counter: 60/27, WL threshold: 4096, image sequence number: 1526576606
Feb 15 07:00:02 kernel: ubi2: available PEBs: 0, total reserved PEBs: 64, PEBs reserved for bad PEB handling: 4
Feb 15 07:00:02 kernel: ubi2: background thread "ubi_bgt2d" started, PID 28132
Feb 15 07:00:03 kernel: UBIFS (ubi2:0): background thread "ubifs_bgt2_0" started, PID 28142
Feb 15 07:00:03 kernel: UBIFS (ubi2:0): UBIFS: mounted UBI device 2, volume 0, name "nvram"
Feb 15 07:00:03 kernel: UBIFS (ubi2:0): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
Feb 15 07:00:03 kernel: UBIFS (ubi2:0): FS size: 5840896 bytes (5 MiB, 46 LEBs), journal size 1396736 bytes (1 MiB, 11 LEBs)
Feb 15 07:00:03 kernel: UBIFS (ubi2:0): reserved for root: 0 bytes (0 KiB)
Feb 15 07:00:03 kernel: UBIFS (ubi2:0): media format: w4/r0 (latest is w4/r0), UUID 9DC75FF7-E8C0-4018-A1ED-1D54D1E5AC77, small LPT model
Feb 15 07:00:03 kernel: UBIFS (ubi2:0): un-mount UBI device 2
Feb 15 07:00:03 kernel: UBIFS (ubi2:0): background thread "ubifs_bgt2_0" stops
Feb 15 07:00:03 kernel: ubi2: detaching mtd10
Feb 15 07:00:03 kernel: ubi2: mtd10 is detached
Feb 15 07:00:03 FTP_Server: daemon is stopped
Feb 15 07:00:03 Samba_Server: smb daemon is stopped
Feb 15 07:00:04 Timemachine: daemon is stopped
Feb 15 07:00:04 FTP_Server: daemon is stopped
Feb 15 07:00:04 Samba_Server: smb daemon is stopped
Feb 15 07:00:04 Timemachine: daemon is stopped
Feb 15 07:00:04 WEBDAV_Server: daemon is stopped
May 5 01:05:23 kernel: klogd started: BusyBox v1.25.1 (2023-12-31 07:18:45 UTC)
May 5 01:05:23 kernel: Linux version 4.1.52 (docker@1bc8445a2919) (gcc version 5.5.0 (Buildroot 2017.11.1) ) #1 SMP PREEMPT Sun Dec 31 07:22:33 UTC 2023
May 5 01:05:23 kernel: Kernel command line: isolcpus=3 root=/dev/ubiblock0_0 ubi.mtd=0 ubi.block=0,0 rootfstype=squashfs console=ttyAMA0 earlyprintk debug irqaffinity=0 pci=pcie_bus_safe
May 5 01:05:23 kernel: Virtual kernel memory layout:
May 5 01:05:23 kernel: vector : 0xffff0000 - 0xffff1000 ( 4 kB)
May 5 01:05:23 kernel: fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
May 5 01:05:23 kernel: vmalloc : 0xe0800000 - 0xff000000 ( 488 MB)
May 5 01:05:23 kernel: lowmem : 0xc0000000 - 0xe0000000 ( 512 MB)
May 5 01:05:23 kernel: pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
May 5 01:05:23 kernel: modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
May 5 01:05:23 kernel: .text : 0xc0018000 - 0xc0648c60 (6340 kB)
May 5 01:05:23 kernel: .init : 0xc0649000 - 0xc067e000 ( 212 kB)
May 5 01:05:23 kernel: .data : 0xc067e000 - 0xc082eda0 (1732 kB)
May 5 01:05:23 kernel: .bss : 0xc082eda0 - 0xc0897490 ( 418 kB)
May 5 01:05:23 kernel: CCI hardware cache coherency enabled
May 5 01:05:23 kernel: pmc_init:PMC using DQM mode
May 5 01:05:23 kernel: Successfully retrieved NVRAM data from dtb
May 5 01:05:23 kernel: cfe-v^A: 1.0.38-163.243
May 5 01:05:23 kernel: Broadcom Logger v0.1
May 5 01:05:23 kernel: skb_free_task created successfully with start budget 256
May 5 01:05:23 kernel: ^[[0;34mBLOG v3.0 Initialized^[[0m
May 5 01:05:23 kernel: BLOG Rule v1.0 Initialized
May 5 01:05:23 kernel: Broadcom IQoS v1.0 initialized
May 5 01:05:23 kernel: Broadcom GBPM v0.1 initialized
May 5 01:05:23 kernel: >>>>> For primary mtd partition rootfs, cferam/vmlinux.lz UBI volume, vmlinux fs mounted as squash fs on UBI <<<<<
May 5 01:05:23 kernel: Secondary mtd partition rootfs_update detected as UBI for cferam/vmlinux source and squash fs on UBI for vmlinux filesystem
May 5 01:05:23 kernel: setup_mtd_parts: misc indx 2 name misc3 nvram configured size 1
May 5 01:05:23 kernel: setup_mtd_parts: name misc3 configured size 0x00100000 offset 0xF600000
May 5 01:05:23 kernel: setup_mtd_parts: misc indx 1 name misc2 nvram configured size 47
May 5 01:05:23 kernel: setup_mtd_parts: name misc2 configured size 0x02f00000 offset 0xC700000
May 5 01:05:23 kernel: setup_mtd_parts: misc indx 0 name misc1 nvram configured size 8
May 5 01:05:23 kernel: setup_mtd_parts: name misc1 configured size 0x00800000 offset 0xBF00000
May 5 01:05:23 kernel: Creating 11 MTD partitions on "brcmnand.0":
May 5 01:05:23 kernel: 0x000006000000-0x00000bf00000 : "rootfs"
May 5 01:05:23 kernel: 0x000000100000-0x000006000000 : "rootfs_update"
May 5 01:05:23 kernel: 0x00000f700000-0x00000ff00000 : "data"
May 5 01:05:23 kernel: 0x000000000000-0x000000100000 : "nvram"
May 5 01:05:23 kernel: 0x000000100000-0x000006000000 : "image_update"
May 5 01:05:23 kernel: 0x000006000000-0x00000bf00000 : "image"
May 5 01:05:23 kernel: 0x000000000000-0x000010000000 : "dummy1"
May 5 01:05:23 kernel: 0x000000000000-0x000010000000 : "dummy2"
May 5 01:05:23 kernel: 0x00000f600000-0x00000f700000 : "misc3"
May 5 01:05:23 kernel: 0x00000c700000-0x00000f600000 : "misc2"
May 5 01:05:23 kernel: 0x00000bf00000-0x00000c700000 : "misc1"
May 5 01:05:23 kernel: brcmboard registered
May 5 01:05:23 kernel: brcmboard: brcm_board_init entry
May 5 01:05:23 kernel: print_rst_status: Last RESET due to Unknown
May 5 01:05:23 kernel: print_rst_status: RESET reason: 0x00000000
May 5 01:05:23 kernel: DYING GASP IRQ Initialized and Enabled
May 5 01:05:23 kernel: map_hw_timer_interrupt,124: interrupt_id 22
May 5 01:05:23 kernel: map_hw_timer_interrupt,124: interrupt_id 23
May 5 01:05:23 kernel: map_hw_timer_interrupt,124: interrupt_id 24
May 5 01:05:23 kernel: map_hw_timer_interrupt,124: interrupt_id 25
May 5 01:05:23 kernel: Allocated EXT_TIMER number 3
May 5 01:05:23 kernel: Broadcom Timer Initialized
May 5 01:05:23 kernel: Registering button 0 (c0874e0c) (bpGpio: 00008009, bpExtIrq:00002000 (2000))
May 5 01:05:23 kernel: extIrqIdx:0, gpioNum:9 ACTIVE LOW
May 5 01:05:23 kernel: Registering button 1 (c0874e80) (bpGpio: 00008008, bpExtIrq:00002001 (2001))
May 5 01:05:23 kernel: extIrqIdx:1, gpioNum:8 ACTIVE LOW
May 5 01:05:23 kernel: bcm_i2c: No port/bus number define i2c interface 0!!!
May 5 01:05:23 kernel: Broadcom BPM Module Char Driver v0.1 Registered<304>^[[0m
May 5 01:05:23 kernel: BPM: tot_mem_size=536870912B (512MB), buf_mem_size <15%> =80530635B (76MB), num of buffers=30690, buf size=2624
May 5 01:05:23 kernel: ^[[0;34m[NTC bpm] bpm_mem_pool_buf_alloc: tot_mem_pool=1279 mem_idx:1279^[[0m
May 5 01:05:23 kernel: ^[[0;34m[NTC bpm] bpm_init_buf_pool: buf_top_idx:30690^[[0m
May 5 01:05:23 kernel: ^[[0;34m[NTC bpm] bpm_init_skb_pool: skbs pool inited total 30690^[[0m
May 5 01:05:23 kernel: init procfs
May 5 01:05:23 kernel: Registering SWP/SWPB emulation handler
May 5 01:05:23 kernel: ubi0: attaching mtd0
May 5 01:05:23 kernel: ubi0: scanning is finished
May 5 01:05:23 kernel: gluebi (pid 1): gluebi_resized: got update notification for unknown UBI device 0 volume 0
May 5 01:05:23 kernel: ubi0: volume 0 ("rootfs_ubifs") re-sized from 381 to 711 LEBs
May 5 01:05:23 kernel: ubi0: attached mtd0 (name "rootfs", size 95 MiB)
May 5 01:05:23 kernel: ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
May 5 01:05:23 kernel: ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
May 5 01:05:23 kernel: ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
May 5 01:05:23 kernel: ubi0: good PEBs: 760, bad PEBs: 0, corrupted PEBs: 0
May 5 01:05:23 kernel: ubi0: user volume: 4, internal volumes: 1, max. volumes count: 128
May 5 01:05:23 kernel: ubi0: max/mean erase counter: 1/0, WL threshold: 4096, image sequence number: 199487054
May 5 01:05:23 kernel: ubi0: available PEBs: 0, total reserved PEBs: 760, PEBs reserved for bad PEB handling: 15
May 5 01:05:23 kernel: ubi0: background thread "ubi_bgt0d" started, PID 70
May 5 01:05:23 kernel: ubi1: attaching mtd2
May 5 01:05:23 kernel: ubi1: scanning is finished
May 5 01:05:23 kernel: ubi1: attached mtd2 (name "data", size 8 MiB)
May 5 01:05:23 kernel: ubi1: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
May 5 01:05:23 kernel: ubi1: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
May 5 01:05:23 kernel: ubi1: VID header offset: 2048 (aligned 2048), data offset: 4096
May 5 01:05:23 kernel: ubi1: good PEBs: 64, bad PEBs: 0, corrupted PEBs: 0
May 5 01:05:23 kernel: ubi1: user volume: 1, internal volumes: 1, max. volumes count: 128
May 5 01:05:23 kernel: ubi1: max/mean erase counter: 435/294, WL threshold: 4096, image sequence number: 797515133
May 5 01:05:23 kernel: ubi1: available PEBs: 0, total reserved PEBs: 64, PEBs reserved for bad PEB handling: 4
May 5 01:05:23 kernel: ubi1: background thread "ubi_bgt1d" started, PID 165
May 5 01:05:23 kernel: UBIFS (ubi1:0): background thread "ubifs_bgt1_0" started, PID 174
May 5 01:05:23 kernel: UBIFS (ubi1:0): UBIFS: mounted UBI device 1, volume 0, name "data"
May 5 01:05:23 kernel: UBIFS (ubi1:0): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
May 5 01:05:23 kernel: UBIFS (ubi1:0): FS size: 5840896 bytes (5 MiB, 46 LEBs), journal size 1015809 bytes (0 MiB, 6 LEBs)
May 5 01:05:23 kernel: UBIFS (ubi1:0): reserved for root: 275879 bytes (269 KiB)
May 5 01:05:23 kernel: UBIFS (ubi1:0): media format: w4/r0 (latest is w4/r0), UUID A34ADCF6-E3FC-47BA-BC71-F7E5B84A6EEE, small LPT model
May 5 01:05:23 kernel: wlcsm: module license 'Proprietary' taints kernel.
May 5 01:05:23 kernel: Disabling lock debugging due to kernel taint
May 5 01:05:23 kernel: BCMLIBS loaded...
May 5 01:05:23 kernel: License chip name: 6755
May 5 01:05:23 kernel: brcmchipinfo: brcm_chipinfo_init entry
May 5 01:05:23 kernel: Broadcom Ingress QoS Module Char Driver v1.0 Registered<303>^[[0m
May 5 01:05:23 kernel: Broadcom Ingress QoS v1.0 initialized
May 5 01:05:23 kernel: ^[[0;36;44mBroadcom Packet Flow Cache ^[[0m
May 5 01:05:23 kernel: FCACHE_CONFIG_MAX_FLOW_ENTRIES<16384>
May 5 01:05:23 kernel: FCACHE_CONFIG_MAX_FDB_ENTRIES<1024>
May 5 01:05:23 kernel: FCACHE_CONFIG_MAX_HOST_DEV_ENTRIES<64>
May 5 01:05:23 kernel: FCACHE_CONFIG_MAX_HOST_MAC_ENTRIES<32>
May 5 01:05:23 kernel: npe_max_entries<32768>
May 5 01:05:23 kernel: Bind blog_notify_evt_enqueue_fn[
I have no idea why the downgrade worked this morning on my 3rd attempt but glad it did. Once I was able to successfully load the downgraded firmware, I attempted to upgrade the remote/slave XT8 nodes which did work and, so far, appear to be functioning normally.
I have not checked this issue yet, but usually the errors like the one in ths github issue, are the ones you can fix by cleaning your NVRAM. There may be something bad there, Another way to debug the problem is to press F12 and check the network call or the js console. if there are errors you get a red box
@gnuton , quick question for you (or anyone else who knows the answer)...how do you clear the NVRAM on the XT8? I thought I knew how, but...no, I don't remember LOL
Reset to factory. Nvram contains the settings
I believe resetting by holding the wps button during boot clears the NVRAM on most Asus routers? But if not, you can ssh in and run a command for it NB. There's a few variations of the command between versions, you might need to research which is best for your router before running.
@robbbaxley I will close this as duplicate of https://github.com/gnuton/asuswrt-merlin.ng/issues/542 In case I am wrong I will reopen this issue again
@gnuton I concur with you that this issue is likely related to or a duplicate of #542
Router Model Affected Models: Asus ZenWiFi XT8 / RT-AX95Q
Firmware Version Affected RT-AX95Q_3004_388.6_0-gnuton0_beta1_puresqubi
Is this bug present in upstream Merlin releases too? Unknown, but not present in earlier Gnuton releases
Describe the bug Mesh Nodes not showing correctly
To Reproduce Steps to reproduce the behavior:
Expected behavior Mesh nodes to be accessible via Network Map>Clients>View List (
Screenshots