openthread / ot-br-posix

OpenThread Border Router, a Thread border router for POSIX-based platforms.
https://openthread.io/
BSD 3-Clause "New" or "Revised" License
420 stars 232 forks source link

Thread Devices Become Unreachable from RPI Over Time Despite Connectivity via ot-ctl #2226

Open SalimAmry opened 7 months ago

SalimAmry commented 7 months ago

Describe the bug

In my setup, I have a Raspberry Pi (RPI) connected to an OpenThread Border Router (OTBR) via USB, along with a Thread device connected to the OpenThread network. After a period of time, the Thread devices become unreachable from the RPI when attempting to ping them or send/receive UDP packets from/to the thread devices, although they remain reachable from the ot-ctl interface via the border router. I suspect the cause for this is when the otbr-agent is not finding the preferred OMR prefix in Thread network.

otbr-agent Logs

Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.201 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8d97 on infra netif 2
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.201 [I] BorderRouter--: Processing RIO (fdf1:9f5:4ed3:1::/64, 1800 seconds)
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.201 [I] NetDataPublshr: Publishing ExternalRoute fdf1:9f5:4ed3:1::/64
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.201 [I] NetDataPublshr: ExternalRoute fdf1:9f5:4ed3:1::/64 (state:Added) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.320 [I] BorderRouter--: Received Router Solicitation from fe80:0:0:0:ae17:c8ff:fe3d:e9f0 on infra netif 2
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.320 [I] BorderRouter--: Start evaluating routing policy, scheduled in 238 milliseconds
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.327 [I] BorderRouter--: Evaluating routing policy
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.327 [I] BorderRouter--: EvaluateOmrPrefix: No preferred OMR prefix found in Thread network
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.327 [I] BorderRouter--: RouterAdvert: Added RIO for fd99:23ca:8c71:1::/64 (lifetime=1800)
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.327 [I] BorderRouter--: Sent Router Advertisement on infra netif 2
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.327 [I] BorderRouter--: Start evaluating routing policy, scheduled in 226000 milliseconds
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.327 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:feb0:f9f on infra netif 2
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.327 [W] BorderRouter--: Received RA from too many routers, ignore RA from fe80:0:0:0:e65f:1ff:feb0:f9f
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.329 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:feb0:ebe on infra netif 2
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.329 [W] BorderRouter--: Received RA from too many routers, ignore RA from fe80:0:0:0:e65f:1ff:feb0:ebe
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.524 [I] MeshForwarder-: Received IPv6 UDP msg, len:247, chksum:4a2c, ecn:no, from:0x9000, sec:yes, prio:net, rss:-77.625
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.524 [I] MeshForwarder-:     src:[fd2b:2ff3:86e4:8b0f:0:ff:fe00:9000]:61631
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.524 [I] MeshForwarder-:     dst:[fd2b:2ff3:86e4:8b0f:0:ff:fe00:2000]:61631
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.524 [I] NetDiag-------: Diagnostic get answer received
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.524 [I] NetDiag-------: Sent diagnostic answer acknowledgment
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.526 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8ca7 on infra netif 2
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.526 [I] BorderRouter--: Processing RIO (fdaf:2419:d87f:1::/64, 1800 seconds)
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.526 [I] NetDataPublshr: Publishing ExternalRoute fdaf:2419:d87f:1::/64
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.526 [I] NetDataPublshr: ExternalRoute fdaf:2419:d87f:1::/64 (state:Adding in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.530 [I] MeshForwarder-: Sent IPv6 UDP msg, len:54, chksum:d651, ecn:no, to:0x9000, sec:yes, prio:normal
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.530 [I] MeshForwarder-:     src:[fd2b:2ff3:86e4:8b0f:0:ff:fe00:2000]:61631
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.530 [I] MeshForwarder-:     dst:[fd2b:2ff3:86e4:8b0f:0:ff:fe00:9000]:61631
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.554 [I] MeshForwarder-: Sent IPv6 UDP msg, len:61, chksum:134a, ecn:no, to:0x3800, sec:yes, prio:low
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.554 [I] MeshForwarder-:     src:[fd08:3c6a:b51f:379e:e65f:1ff:feb0:f9f]:37051
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.555 [I] MeshForwarder-:     dst:[fd2b:2ff3:86e4:8b0f:0:ff:fe00:3800]:11
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.609 [I] BorderRouter--: Received Router Solicitation from fe80:0:0:0:ae17:c8ff:fe3d:7a28 on infra netif 2
Mar 19 10:00:21 pi otbr-agent[22985]: 03:57:19.609 [I] BorderRouter--: Start evaluating routing policy, scheduled in 2718 milliseconds
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.711 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8c69 on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.711 [W] BorderRouter--: Received RA from too many routers, ignore RA from fe80:0:0:0:e65f:1ff:fed9:8c69
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.718 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:feb0:f30 on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.718 [I] BorderRouter--: Processing RIO (fd6b:790b:2407:1::/64, 1800 seconds)
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.718 [I] NetDataPublshr: Publishing ExternalRoute fd6b:790b:2407:1::/64
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.718 [I] NetDataPublshr: ExternalRoute fd6b:790b:2407:1::/64 (state:Added) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.819 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8d7c on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.819 [I] BorderRouter--: Processing RIO (fd91:b9f:8b72:1::/64, 1800 seconds)
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.819 [I] NetDataPublshr: Publishing ExternalRoute fd91:b9f:8b72:1::/64
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:19.819 [I] NetDataPublshr: ExternalRoute fd91:b9f:8b72:1::/64 (state:Adding) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.095 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8cf3 on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.095 [I] BorderRouter--: Processing RIO (fdb7:aade:ff78:1::/64, 1800 seconds)
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.095 [I] NetDataPublshr: Publishing ExternalRoute fdb7:aade:ff78:1::/64
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.095 [I] NetDataPublshr: ExternalRoute fdb7:aade:ff78:1::/64 (state:Adding in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.300 [I] BorderRouter--: Received Router Solicitation from fe80:0:0:0:ae17:c8ff:fe28:69e0 on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.301 [I] BorderRouter--: Start evaluating routing policy, scheduled in 2026 milliseconds
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.349 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8c7e on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.349 [I] BorderRouter--: Processing RIO (fd2b:8ca3:32c9:1::/64, 1800 seconds)
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.349 [I] NetDataPublshr: Publishing ExternalRoute fd2b:8ca3:32c9:1::/64
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.349 [I] NetDataPublshr: ExternalRoute fd2b:8ca3:32c9:1::/64 (state:Added) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.357 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8d91 on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.357 [I] BorderRouter--: Processing RIO (fdca:6ab7:3407:1::/64, 1800 seconds)
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.357 [I] NetDataPublshr: Publishing ExternalRoute fdca:6ab7:3407:1::/64
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.357 [I] NetDataPublshr: ExternalRoute fdca:6ab7:3407:1::/64 (state:Added) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.365 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8c8a on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.365 [I] BorderRouter--: Processing RIO (fde6:2080:4a58:1::/64, 1800 seconds)
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.365 [I] NetDataPublshr: Publishing ExternalRoute fde6:2080:4a58:1::/64
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.365 [I] NetDataPublshr: ExternalRoute fde6:2080:4a58:1::/64 (state:Added) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.380 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:6322:e7ed:97f3:83a7 on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.380 [I] BorderRouter--: Processing RIO (fdc6:edca:3b88:1::/64, 1800 seconds)
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.380 [I] NetDataPublshr: Publishing ExternalRoute fdc6:edca:3b88:1::/64
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.381 [I] NetDataPublshr: ExternalRoute fdc6:edca:3b88:1::/64 (state:Adding in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.390 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:1fae:224d:5bbc:71d8 on infra netif 2
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.390 [I] BorderRouter--: Processing RIO (fde7:2ec8:58ad:1::/64, 1800 seconds)
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.390 [I] NetDataPublshr: Publishing ExternalRoute fde7:2ec8:58ad:1::/64
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.390 [I] NetDataPublshr: ExternalRoute fde7:2ec8:58ad:1::/64 (state:Added) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.555 [I] MeshForwarder-: Sent IPv6 UDP msg, len:61, chksum:bb8b, ecn:no, to:0x9000, sec:yes, prio:low
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.555 [I] MeshForwarder-:     src:[fd08:3c6a:b51f:379e:e65f:1ff:feb0:f9f]:36985
Mar 19 10:00:22 pi otbr-agent[22985]: 03:57:20.556 [I] MeshForwarder-:     dst:[fd2b:2ff3:86e4:8b0f:0:ff:fe00:9000]:11
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.721 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8d00 on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.721 [I] BorderRouter--: Processing PIO (fdb9:1751:9c4d:b809::/64, 1800 seconds)
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.721 [I] NetDataPublshr: Publishing ExternalRoute fdb9:1751:9c4d:b809::/64
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.721 [I] NetDataPublshr: ExternalRoute fdb9:1751:9c4d:b809::/64 (state:Add in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.721 [I] BorderRouter--: Processing RIO (fdb2:5081:6c0c:1::/64, 1800 seconds)
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.721 [I] NetDataPublshr: Publishing ExternalRoute fdb2:5081:6c0c:1::/64
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.721 [I] NetDataPublshr: ExternalRoute fdb2:5081:6c0c:1::/64 (state:Added) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.723 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8d33 on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.724 [W] BorderRouter--: Received RA from too many routers, ignore RA from fe80:0:0:0:e65f:1ff:fed9:8d33
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.727 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8d6e on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.727 [I] BorderRouter--: Processing RIO (fd9a:2ccb:d5b9:1::/64, 1800 seconds)
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.728 [I] NetDataPublshr: Publishing ExternalRoute fd9a:2ccb:d5b9:1::/64
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.728 [I] NetDataPublshr: ExternalRoute fd9a:2ccb:d5b9:1::/64 (state:Adding in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.729 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:feb0:ffc on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.730 [I] BorderRouter--: Processing RIO (fd55:172f:863f:1::/64, 1800 seconds)
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.730 [I] NetDataPublshr: Publishing ExternalRoute fd55:172f:863f:1::/64
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.730 [I] NetDataPublshr: ExternalRoute fd55:172f:863f:1::/64 (state:Adding in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.895 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8cd1 on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.895 [W] BorderRouter--: Received RA from too many routers, ignore RA from fe80:0:0:0:e65f:1ff:fed9:8cd1
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.902 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:fed9:8cd6 on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.902 [I] BorderRouter--: Processing PIO (fd50:69d5:a01b:ff53::/64, 1800 seconds)
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.902 [I] NetDataPublshr: Publishing ExternalRoute fd50:69d5:a01b:ff53::/64
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.902 [I] NetDataPublshr: ExternalRoute fd50:69d5:a01b:ff53::/64 (state:Add in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.902 [I] BorderRouter--: Processing RIO (fd62:4fe7:13b9:1::/64, 1800 seconds)
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.902 [I] NetDataPublshr: Publishing ExternalRoute fd62:4fe7:13b9:1::/64
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.902 [I] NetDataPublshr: ExternalRoute fd62:4fe7:13b9:1::/64 (state:Added) in netdata - total:0, preferred:0, desired:10
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.928 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:4e68:4049:f8a:3805 on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.928 [W] BorderRouter--: Received RA from too many routers, ignore RA from fe80:0:0:0:4e68:4049:f8a:3805
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.958 [I] BorderRouter--: Received Router Solicitation from fe80:0:0:0:3656:feff:fe7f:bc30 on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:20.958 [I] BorderRouter--: Start evaluating routing policy, scheduled in 1369 milliseconds
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:21.051 [I] BorderRouter--: Received Router Advertisement from fe80:0:0:0:e65f:1ff:feb0:f60 on infra netif 2
Mar 19 10:00:23 pi otbr-agent[22985]: 03:57:21.051 [W] BorderRouter--: Received RA from too many routers, ignore RA from fe80:0:0:0:e65f:1ff:feb0:f60

Ping Snapshots

Border router mIP is fd2b:2ff3:86e4:8b0f:6172:566b:4bb0:1361 Thread device mIP is fd2b:2ff3:86e4:8b0f:bf95:5aee:84de:45fc

Here are the ping that were performed from the RPI and ot-ctl to the Thread device:

pi@pi:~ $ ping fd2b:2ff3:86e4:8b0f:bf95:5aee:84de:45fc
PING fd2b:2ff3:86e4:8b0f:bf95:5aee:84de:45fc(fd2b:2ff3:86e4:8b0f:bf95:5aee:84de:45fc) 56 data bytes

--- fd2b:2ff3:86e4:8b0f:bf95:5aee:84de:45fc ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6243ms

pi@pi:~ $
pi@pi:~ $ sudo ot-ctl ping fd2b:2ff3:86e4:8b0f:bf95:5aee:84de:45fc
16 bytes from fd2b:2ff3:86e4:8b0f:bf95:5aee:84de:45fc: icmp_seq=1 hlim=64 time=11ms
1 packets transmitted, 1 packets received. Packet loss = 0.0%. Round-trip min/avg/max = 11/11.0/11 ms.
Done

Ping to the border router works fine

pi@pi:~ $ ping fd2b:2ff3:86e4:8b0f:6172:566b:4bb0:1361
PING fd2b:2ff3:86e4:8b0f:6172:566b:4bb0:1361(fd2b:2ff3:86e4:8b0f:6172:566b:4bb0:1361) 56 data bytes
64 bytes from fd2b:2ff3:86e4:8b0f:6172:566b:4bb0:1361: icmp_seq=1 ttl=64 time=0.132 ms
64 bytes from fd2b:2ff3:86e4:8b0f:6172:566b:4bb0:1361: icmp_seq=2 ttl=64 time=0.104 ms
64 bytes from fd2b:2ff3:86e4:8b0f:6172:566b:4bb0:1361: icmp_seq=3 ttl=64 time=0.120 ms

--- fd2b:2ff3:86e4:8b0f:6172:566b:4bb0:1361 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2107ms
rtt min/avg/max/mdev = 0.104/0.118/0.132/0.011 ms

Environment

abtink commented 7 months ago

Some observations from the logs:

You seem to be running an older commit (from 2022). Suggest upgrading to new version. Some specific changes to support larger number of infra-if routers:

SalimAmry commented 7 months ago

Thank you for your feedback.

In our setup, we have one RCP based border router and 3 other OpenThread devices, which are configured to function as routers within the network. To manage and monitor the state of our OpenThread Border Router (OTBR), we've implemented a script that actively sends messages to these OpenThread devices via UDP. These devices are programmed to respond to these messages, allowing us to ensure continuous communication and validate network stability.

The older commit version we're using might indeed be contributing to the issues. We plan to upgrade to a newer version, keeping in mind the enhancements mentioned, especially regarding netdata space savings and flexible data structures. Upgrading will take some time to observe if the issue reoccurs due to its intermittent nature.