search

skycoin / skywire

Skywire Node implementation
73 stars 45 forks source link

cli RPC methods not found or not working when using dmsghttp-config (deployment outage?) #1762

Open 0pcom opened 3 months ago

0pcom commented 3 months ago

i can confirm the dmsghttp-config used is the one on develop branch

$ skywire cli visor info
[2024-03-12T10:34:29.173333014-05:00] FATAL [skywire-cli]: Failed to connect to visor RPC or RPC method not found; is skywire running?: overview
$ skywire cli visor pk
[2024-03-12T10:34:30.452008919-05:00] FATAL [skywire-cli]: Failed to connect to visor RPC or RPC method not found; is skywire running?: no transport manager available

[2024-03-12T10:29:59.379174594-05:00] DEBUG [dmsgC]: Serving session. remote_pk=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
[2024-03-12T10:30:16.722460819-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:16.722679307-05:00] WARN retrier [utclient]: Retrying... current_backoff=10s error="uptime tracker httpauth: Get "dmsg://022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:16.722936004-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:16.723042233-05:00] WARN [address_resolver]: Failed to connect to address resolver. STCPR/SUDPH services are temporarily unavailable. Retrying... error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:17.173935316-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:17.174073975-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:17.174121543-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:36.730892483-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:36.731032984-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:36.730949386-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:36.731259761-05:00] WARN retrier [utclient]: Retrying... current_backoff=20s error="uptime tracker httpauth: Get "dmsg://022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:37.175133513-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:37.175299642-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:37.175375817-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:56.731852097-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:56.731901726-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:56.732009726-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:56.732165004-05:00] WARN retrier [utclient]: Retrying... current_backoff=40s error="uptime tracker httpauth: Get "dmsg://022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:57.176567062-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:30:57.176705024-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:30:57.176768541-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:16.732567228-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:31:16.732711435-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:16.732612982-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:31:17.177616602-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:31:17.177755137-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:17.177807733-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:36.738483978-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:31:36.738493967-05:00] WARN retrier [utclient]: Retrying... current_backoff=1m20s error="uptime tracker httpauth: Get "dmsg://022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:36.738629695-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:37.178243726-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:31:37.178420493-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:37.178466813-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:56.739498171-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:31:56.739505864-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:31:56.739723271-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:57.179228296-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:31:57.179374108-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:31:57.179420014-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:16.740676232-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:32:16.740840038-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:17.180113268-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:32:17.180279383-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:17.180327502-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:36.749666912-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:32:36.749794133-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:37.180850716-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:32:37.181022568-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:37.181072552-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:56.738935202-05:00] WARN retrier [utclient]: Retrying... current_backoff=2m40s error="uptime tracker httpauth: Get "dmsg://022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:56.750069147-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:32:56.750232186-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:57.18218664-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:32:57.182324485-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:32:57.18237059-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:33:16.739823734-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=022c424caa6239ba7d1d9d8f7dab56cd5ec6ae2ea9ad97bb94ad4b48f62a540d3f:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:33:16.751021867-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:33:16.751181855-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:33:17.183606117-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:33:17.183695493-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:33:17.183723537-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:33:36.75493078-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:33:36.755105677-05:00] WARN retrier [address_resolver]: Retrying... current_backoff=1s error="Get "dmsg://03234b2ee4128d1f78c180d06911102906c80795dfe41bd6253f2619c8b6252a02:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:33:37.18435463-05:00] DEBUG ClientSession.DialStream [dmsg_http:dmsgDC]: Stream closed on failure. close_error=<nil> dst_addr=02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80 error="i/o deadline reached" session=03717576ada5b1744e395c66c2bb11cea73b0e23d0dcd54422139b1a7f12e962c4
[2024-03-12T10:33:37.184758507-05:00] ERROR [transport_manager]: Failed to connect to transport discovery, retrying... error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
[2024-03-12T10:33:37.18486117-05:00] WARN retrier [transport_manager]: Retrying... current_backoff=1s error="transport discovery httpauth: Get "dmsg://02b307aee5c8ce1666c63891f8af25ad2f0a47a243914c963942b3ba35b9d095ae:80/security/nonces/03c758a0daabab4b4bbd524cf46306212818b051b9d92949bd8acdf6226aad32e6": i/o deadline reached"
mrpalide commented 3 months ago

Temporary fixed, but needs more investigation on reason or add/implement scenario to handle this issue.