search

skycoin / skywire

Skywire Node implementation
73 stars 45 forks source link

Multi-hop routes not working automatically ; erroneous auto-transport creation #1793

Open 0pcom opened 3 months ago

0pcom commented 3 months ago

Regardless of the minimum hops set, the visor will attempt to make a direct transport to another public key.

A visor was started with minimum hops = 1

transports were created to several heavily transported visors (skywire-cli rtree -s | head -n100) .

A proxy server with potential multihop routes was selected and connected to.

$ skywire cli rtfind 03d8ee5d4c2c29a3d028b3a210281a52b1693120bc71e9a52ae58e2da2803284b0
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 03f1a1b354e412340ca1e8c3e59e738f1879d52213d973ad1a102c2f6e192033d8 @ 6b188449-1ece-0468-9c11-ea0494ce6444 03f1a1b354e412340ca1e8c3e59e738f1879d52213d973ad1a102c2f6e192033d8 -> 0313a76e2c331669a0cb1a3b749930881f9881cca89b59ee52365d1c15141d9d83 @ 8376c4b3-acbc-0170-9094-2d75ae39dd88 0313a76e2c331669a0cb1a3b749930881f9881cca89b59ee52365d1c15141d9d83 -> 03d8ee5d4c2c29a3d028b3a210281a52b1693120bc71e9a52ae58e2da2803284b0 @ 162bc304-8f29-0dbd-b348-24fc86126078]
 reverse: [03d8ee5d4c2c29a3d028b3a210281a52b1693120bc71e9a52ae58e2da2803284b0 -> 029fda16bb7998c1a83ad8cc5e685cf73e41d6d6c9a3a4c74d6e1668d0dc321062 @ a883750c-abbc-01aa-9768-37d3c987d09b 029fda16bb7998c1a83ad8cc5e685cf73e41d6d6c9a3a4c74d6e1668d0dc321062 -> 0360889c923eb2d898044f1e6f8e3012cc919bda20c8b81a9a9d86f802ac850cda @ 70cbd927-fdac-025d-a79f-daf7b8cd4431 0360889c923eb2d898044f1e6f8e3012cc919bda20c8b81a9a9d86f802ac850cda -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 2ea1316b-2a31-0a45-b9c4-4273c99ef371]

when the proxy server was started, a direct transport was established automatically to that visor which should not happen in this instance

When the transport was deleted, the connection was broken - despite the possibility of alternative routes and despite having more than 1 hop configured

0pcom commented 3 months ago

A test of manual route creation shows that, despite the existence of a route, when starting the proxy another, a transport and another direct route is created ; the proxy actually uses that one to connect instead of the existing route!

perhaps I did not configure the route correctly @mrpalide ?

[user@linux ~]$ skywire cli route rm 1
OK
[user@linux ~]$ skywire cli route rm 8
OK
[user@linux ~]$ skywire cli route
id     type     local-port     remote-port     remote-pk     next-route-id     next-transport-id     expire-at
[user@linux ~]$ skywire cli route add c -i 1 -j 2 -k 743f7683-78da-0401-a334-c74fc0c564ed -l 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -m 49154 -p 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -q 4 --keep-alive 300s
skywire cli route add b -i 2 -j 3 -k 05352249-ca49-0001-aefc-1d520078ecf4 --keep-alive 300s
skywire cli route add a -i 3 -j 3 -k 05352249-ca49-0001-aefc-1d520078ecf4 -l 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -m 49154 -p 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 -q 4 --keep-alive 300s
Routing Rule Key: 1
Routing Rule Key: 2
unknown shorthand flag: 'j' in -j
[user@linux ~]$ skywire cli route add a -i 3  -k 05352249-ca49-0001-aefc-1d520078ecf4 -l 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -m 49154 -p 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 -q 4 --keep-alive 300s
unknown shorthand flag: 'k' in -k
[user@linux ~]$ skywire cli route add a -i 3   -l 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -m 49154 -p 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 -q 4 --keep-alive 300s
Routing Rule Key: 3
[user@linux ~]$ skywire-cli proxy start -k 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9
Starting......
Running!

perhaps the issue is apparent here, some routes have the same route id! Here's the routing table:

$ skywire-cli route
id     type                    local-port     remote-port     remote-pk                                                              next-route-id     next-transport-id                        expire-at
2      Forward                 49154          4               02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105     2                 743f7683-78da-0401-a334-c74fc0c564ed     5m0s
3      IntermediaryForward     -              -               -                                                                      3                 05352249-ca49-0001-aefc-1d520078ecf4     5m0s
3      Consume                 49154          4               02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9     -                 -                                        5m0s
3      Forward                 49157          3               02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9     3                 c36129a1-02ee-01ee-bbd5-1df55e0220a4     30s
10     Consume                 3              49157           0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c     -                 -                                        30s

Visor debug logging

[2024-04-04T09:28:38.078997535-05:00] DEBUG [router]: ROUTING TABLE CONTENTS: map[1:FWD(keyRtID:1, nxtRtID:2, nxtTpID:743f7683-78da-0401-a334-c74fc0c564ed, rAddr:02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105:4, lAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49154)]
[2024-04-04T09:28:38.213824181-05:00] DEBUG [router]: ROUTING TABLE CONTENTS: map[1:FWD(keyRtID:1, nxtRtID:2, nxtTpID:743f7683-78da-0401-a334-c74fc0c564ed, rAddr:02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105:4, lAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49154) 2:INTER(keyRtID:2, nxtRtID:3, nxtTpID:05352249-ca49-0001-aefc-1d520078ecf4)]
[2024-04-04T09:29:34.80055862-05:00] DEBUG [router]: ROUTING TABLE CONTENTS: map[1:FWD(keyRtID:1, nxtRtID:2, nxtTpID:743f7683-78da-0401-a334-c74fc0c564ed, rAddr:02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105:4, lAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49154) 2:INTER(keyRtID:2, nxtRtID:3, nxtTpID:05352249-ca49-0001-aefc-1d520078ecf4) 3:REV(keyRtID:3, rAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:4, lAddr:02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105:49154)]
[2024-04-04T09:29:57.527061841-05:00] INFO [visor]: Changing skysocks-client Settings to map["--addr":":1080" "--srv":"02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9" "app":"skysocks-client"]
[2024-04-04T09:29:57.527105778-05:00] INFO [visor:config]: Flushing config to file. config_version="v1.3.20-rc2-3-g9052899e8" filepath="/opt/skywire/skywire.json"
[2024-04-04T09:29:57.527402983-05:00] INFO [visor:config]: Flushing config to file. config_version="v1.3.20-rc2-3-g9052899e8" filepath="/opt/skywire/skywire.json"
[2024-04-04T09:29:57.527599869-05:00] INFO [visor]: Updated Settings.
[2024-04-04T09:29:57.527812322-05:00] DEBUG [proc:skysocks-client:22a1aac016a64590a43473830a477b68]: No app discovery associated with app. appName="skysocks-client"
[2024-04-04T09:29:57.658191327-05:00] DEBUG [proc_manager]: Accepting proc conn... remote=127.0.0.1:50094
[2024-04-04T09:29:57.658297348-05:00] DEBUG [proc_manager]: Read hello from proc. hello="{"proc_key":"22a1aac016a64590a43473830a477b68"}" remote=127.0.0.1:50094
[2024-04-04T09:29:57.658334528-05:00] DEBUG [proc_manager]: Accepted proc conn. hello="{"proc_key":"22a1aac016a64590a43473830a477b68"}" remote=127.0.0.1:50094
[2024-04-04T09:29:57.658965306-05:00] DEBUG [router]: Established transport exist. Type: dmsg
[2024-04-04T09:29:57.659010898-05:00] DEBUG [router]: Requesting new routes from 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c to 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9
[2024-04-04T09:29:57.658343708-05:00] DEBUG (STDOUT) [proc:skysocks-client:22a1aac016a64590a43473830a477b68]: Version "v1.3.20-rc2-5-g3fc123817" built on "2024-04-03T14:45:35Z" against commit "3fc123817a902be7106ec4b43d00e3ff218b3186"
[2024-04-04T09:29:57.658426307-05:00] DEBUG [proc:skysocks-client:22a1aac016a64590a43473830a477b68]: Associated and serving proc conn.
[2024-04-04T09:29:59.426735138-05:00] DEBUG [router]: Found routes Forward: [[0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 @ c36129a1-02ee-01ee-bbd5-1df55e0220a4] [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 @ 743f7683-78da-0401-a334-c74fc0c564ed 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -> 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 @ 05352249-ca49-0001-aefc-1d520078ecf4] [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 0305918027b75cabffab6f0f7a50c6eee59c3c6f994687ea23a7ea477ec195d753 @ 723bf75e-b398-014a-beb7-f0d5729607da 0305918027b75cabffab6f0f7a50c6eee59c3c6f994687ea23a7ea477ec195d753 -> 02058471ce833042566d7128cf708976a7fec5669f48647041e6e585ba29bd95c3 @ 3ed790c4-b9a8-0af8-81d5-aff79ede8101 02058471ce833042566d7128cf708976a7fec5669f48647041e6e585ba29bd95c3 -> 022d2e9ff0ab2b5ad03851b07a61c32b403570423a2b5a4567e2f4d2aabcd1119d @ d63d79c7-04d7-0e1b-870f-07e3c84fd84d 022d2e9ff0ab2b5ad03851b07a61c32b403570423a2b5a4567e2f4d2aabcd1119d -> 037d6cddd7efa411fbc7e9d7e81d625dbe8ce05c8ffeda06f12916c345be15693b @ 451b2979-a33d-08ff-8cc4-f5022496f62d 037d6cddd7efa411fbc7e9d7e81d625dbe8ce05c8ffeda06f12916c345be15693b -> 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 @ a06cdb5b-de78-02fb-bffb-fbee8ce69da0] [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 0274bd1ed89452f8edc41e3e8dde72b076f235e003e7b17043aad6b6c1c8a02ed0 @ de2fc411-a83c-0b07-b364-d3f85afb7b99 0274bd1ed89452f8edc41e3e8dde72b076f235e003e7b17043aad6b6c1c8a02ed0 -> 0359298e04891c531f2d8b7737eabd3e6c0ae907a3989b1747e8d6c9be23d78c48 @ 4b699ce9-e9cc-09c0-8fcc-8bba9c467972 0359298e04891c531f2d8b7737eabd3e6c0ae907a3989b1747e8d6c9be23d78c48 -> 02c447d38c79766fc96a68f1f7fb951d987d7df13e5f54307f85385584b9c37947 @ 5c97222b-a993-068f-aebf-9923f0914806 02c447d38c79766fc96a68f1f7fb951d987d7df13e5f54307f85385584b9c37947 -> 031eefdaa55b37d792a2c902beccf378448b7baecf34a9e1407253f0c444aa057c @ 79b40064-86f3-06af-8fcf-4aa86ee2ce2a 031eefdaa55b37d792a2c902beccf378448b7baecf34a9e1407253f0c444aa057c -> 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 @ 2cc9b18e-b4da-09a9-9c59-782ac48ddc10]]. Reverse [[02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ c36129a1-02ee-01ee-bbd5-1df55e0220a4] [02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 -> 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 @ 05352249-ca49-0001-aefc-1d520078ecf4 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 743f7683-78da-0401-a334-c74fc0c564ed] [02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 -> 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 @ 05352249-ca49-0001-aefc-1d520078ecf4 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -> 0219453427908bc29cc532906878f9d9693c2692fa2569d92f2f7236d749c88233 @ 573e4f6a-1f06-0ef0-8105-0189ab9cf7d1 0219453427908bc29cc532906878f9d9693c2692fa2569d92f2f7236d749c88233 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 5c202a3e-dea2-0ac8-ae1d-31f0d536ed19] [02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 -> 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 @ 05352249-ca49-0001-aefc-1d520078ecf4 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -> 0217863fffc270544581ec3d69d95fc5e0bf9a61decedf335f421c8fe6a14bced5 @ c195d08d-8f09-0264-b6c3-edb8c395b02f 0217863fffc270544581ec3d69d95fc5e0bf9a61decedf335f421c8fe6a14bced5 -> 0374623449e68528ff4c18a8cba9b42c424c2dd4ee7e2b38b18aa7828b014a72e5 @ b033fd73-1a02-0290-914f-f957bb95371c 0374623449e68528ff4c18a8cba9b42c424c2dd4ee7e2b38b18aa7828b014a72e5 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 7622075a-3b19-0c20-b9f0-fd5da1ce815c] [02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 -> 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 @ 05352249-ca49-0001-aefc-1d520078ecf4 02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105 -> 03827409b68b9d90f33f69204895bd0463710ccdb679177a8d83a64c4538d2270d @ ebc79f2c-b3f5-08d5-be22-cc82951cbba0 03827409b68b9d90f33f69204895bd0463710ccdb679177a8d83a64c4538d2270d -> 027d8dc3b54367df8b81cbefd3ff3fcbbbd9775f448b80e3bb11e97a174c4cbb22 @ d7fa6519-6d92-06d7-b69c-fc823a860a74 027d8dc3b54367df8b81cbefd3ff3fcbbbd9775f448b80e3bb11e97a174c4cbb22 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 181221ec-d15d-0598-bfac-b742cc6e240f]]
[2024-04-04T09:30:00.558192586-05:00] DEBUG [router]: handling setup request: setupPK(0324579f003e6b4048bae2def4365e634d8e0e3054a20fc7af49daf2a179658557)
[2024-04-04T09:30:01.466244616-05:00] DEBUG [router]: ROUTING TABLE CONTENTS: map[1:FWD(keyRtID:1, nxtRtID:2, nxtTpID:743f7683-78da-0401-a334-c74fc0c564ed, rAddr:02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105:4, lAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49154) 2:INTER(keyRtID:2, nxtRtID:3, nxtTpID:05352249-ca49-0001-aefc-1d520078ecf4) 3:REV(keyRtID:3, rAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:4, lAddr:02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105:49154) 9:FWD(keyRtID:9, nxtRtID:3, nxtTpID:c36129a1-02ee-01ee-bbd5-1df55e0220a4, rAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:3, lAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49157)]
[2024-04-04T09:30:01.466325241-05:00] DEBUG [router]: Save new Routing Rule with ID 9 FWD(keyRtID:9, nxtRtID:3, nxtTpID:c36129a1-02ee-01ee-bbd5-1df55e0220a4, rAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:3, lAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49157)
[2024-04-04T09:30:01.466386239-05:00] DEBUG [router]: ROUTING TABLE CONTENTS: map[1:FWD(keyRtID:1, nxtRtID:2, nxtTpID:743f7683-78da-0401-a334-c74fc0c564ed, rAddr:02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105:4, lAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49154) 2:INTER(keyRtID:2, nxtRtID:3, nxtTpID:05352249-ca49-0001-aefc-1d520078ecf4) 3:REV(keyRtID:3, rAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:4, lAddr:02f933f978a025b9ccca71d585f0c75ec79710ccdd665d392988eb906ba69ba105:49154) 9:FWD(keyRtID:9, nxtRtID:3, nxtTpID:c36129a1-02ee-01ee-bbd5-1df55e0220a4, rAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:3, lAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49157) 10:REV(keyRtID:10, rAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49157, lAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:3)]
[2024-04-04T09:30:01.466440831-05:00] DEBUG [router]: Save new Routing Rule with ID 10 REV(keyRtID:10, rAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49157, lAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:3)
[2024-04-04T09:30:01.466496797-05:00] DEBUG [router]: Saving route group rules with desc: rAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49157, lAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:3
[2024-04-04T09:30:01.466670929-05:00] DEBUG [RouteGroup rAddr:0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c:49157, lAddr:02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9:3]: Sent handshake via transport c36129a1-02ee-01ee-bbd5-1df55e0220a4
[2024-04-04T09:30:02.501737577-05:00] DEBUG [router]: Created new routes to 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9 on port 49157
[2024-04-04T09:30:02.502065798-05:00] DEBUG (STDOUT) [proc:skysocks-client:22a1aac016a64590a43473830a477b68]: Connected to 02a49534505bd949af5e14c8ec2ad6df54664f643c7af0a8f866ed7619e826c2c9
[2024-04-04T09:30:02.503473764-05:00] DEBUG (STDOUT) [proc:skysocks-client:22a1aac016a64590a43473830a477b68]: Serving proxy client :1080
[2024-04-04T09:30:02.502303655-05:00] INFO [proc:skysocks-client:22a1aac016a64590a43473830a477b68]: App skysocks-client is Running
[2024-04-04T09:30:14.259203565-05:00] DEBUG (STDOUT) [proc:skysocks-client:22a1aac016a64590a43473830a477b68]: Listening skysocks client on :1080Accepted skysocks client
[2024-04-04T09:30:14.268582418-05:00] DEBUG (STDOUT) [proc:skysocks-client:22a1aac016a64590a43473830a477b68]: Opened session skysocks client
0pcom commented 2 months ago

Another attempt at a test of multihop routes

using transport setup-node I created transports between (10) US visors providing proxy servers of version 1.3.21 currently

skywire cli proxy list -v v1.3.21 -c US | while read _pk ; do skywire svc tps add -t dmsg -1 $_pk -2 $(skywire cli proxy list -v v1.3.21 -c US | grep -v "$_pk" | shuf -n 1) ; done

I ran the above command a few times so that enough transports would exist between those visors

after setting min_hops in the visor's config to 3 i start a visor and made transports to all of the above visors

skywire cli proxy list -v v1.3.21 -c US | while read _pk ; do skywire cli tp add -t dmsg $_pk ; done

then i check for routes

skywire cli proxy list -v v1.3.21 -c US | while read _pk ; do echo $_pk ; skywire cli route find -n2 $_pk ; echo ; done
022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 @ bf97fc52-a273-0016-b063-d7a7518fbf13 025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 -> 022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a @ 01e348e4-fe70-0934-b63c-457285d05e59]
 reverse: [022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 5b38f859-3019-05d2-90b2-98c0436eb042 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 22d0bf88-378f-0af9-8009-060887e0d221]
025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c @ 6e5ec87d-4595-04f1-b6cf-62c9e95e0ee8 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c -> 025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 @ 4e0bb625-ea1a-05eb-ab3d-e7e63be78ea4]
 reverse: [025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 -> 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c @ 4e0bb625-ea1a-05eb-ab3d-e7e63be78ea4 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 6e5ec87d-4595-04f1-b6cf-62c9e95e0ee8]
02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec @ e3127e3f-b939-06e2-82cb-e0a091986a82 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec -> 02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696 @ 8c71315d-c08d-05ac-8f0e-064df06ec798]
 reverse: [02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696 -> 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 @ de23cd99-4e4c-00a7-a53f-4dcf7760354e 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 5b6598fd-4e13-0a72-8745-f1d587427178]
029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 22d0bf88-378f-0af9-8009-060887e0d221 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec @ 6cf783d5-9332-0686-bb2b-62ddd3304fbe]
 reverse: [029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 6cf783d5-9332-0686-bb2b-62ddd3304fbe 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 22d0bf88-378f-0af9-8009-060887e0d221]
03075370d8cc7f2b66d19795ca47f8bfe26ee5cd755029c59616c900319c2f4b63
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 0383713043445e71158edc5a967207ac0e92b474218965052497b26212f4538eaf @ 2a420c66-2670-0da8-9e55-276ddeeabbeb 0383713043445e71158edc5a967207ac0e92b474218965052497b26212f4538eaf -> 03075370d8cc7f2b66d19795ca47f8bfe26ee5cd755029c59616c900319c2f4b63 @ 0aac49d6-b79d-0723-9add-05390e5bd4aa]
 reverse: [03075370d8cc7f2b66d19795ca47f8bfe26ee5cd755029c59616c900319c2f4b63 -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 423c17d5-c0d8-0b51-87ab-d4502cf504c6 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 22d0bf88-378f-0af9-8009-060887e0d221]
03419b3c3f9db0404b44afc532a91eb2c108499f7575d0e7937dd8d076d385b607
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696 @ eb54647a-67ab-097b-83ff-b93072e542a2 02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696 -> 03419b3c3f9db0404b44afc532a91eb2c108499f7575d0e7937dd8d076d385b607 @ cd455d03-7698-0a42-bb22-bf99c24ec828]
 reverse: [03419b3c3f9db0404b44afc532a91eb2c108499f7575d0e7937dd8dskywire cli proxy list -v v1.3.21 -c US | while read _pk ; do echo $_pk ; skywire cli route find -n2 $_pk ; echo ; done
022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 @ bf97fc52-a273-0016-b063-d7a7518fbf13 025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 -> 022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a @ 01e348e4-fe70-0934-b63c-457285d05e59]
 reverse: [022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 5b38f859-3019-05d2-90b2-98c0436eb042 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 22d0bf88-378f-0af9-8009-060887e0d221]
025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c @ 6e5ec87d-4595-04f1-b6cf-62c9e95e0ee8 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c -> 025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 @ 4e0bb625-ea1a-05eb-ab3d-e7e63be78ea4]
 reverse: [025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 -> 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c @ 4e0bb625-ea1a-05eb-ab3d-e7e63be78ea4 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 6e5ec87d-4595-04f1-b6cf-62c9e95e0ee8]
02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec @ e3127e3f-b939-06e2-82cb-e0a091986a82 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec -> 02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696 @ 8c71315d-c08d-05ac-8f0e-064df06ec798]
 reverse: [02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696 -> 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 @ de23cd99-4e4c-00a7-a53f-4dcf7760354e 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 5b6598fd-4e13-0a72-8745-f1d587427178]
029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 22d0bf88-378f-0af9-8009-060887e0d221 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec @ 6cf783d5-9332-0686-bb2b-62ddd3304fbe]
 reverse: [029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 6cf783d5-9332-0686-bb2b-62ddd3304fbe 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 22d0bf88-378f-0af9-8009-060887e0d221]
03075370d8cc7f2b66d19795ca47f8bfe26ee5cd755029c59616c900319c2f4b63
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 0383713043445e71158edc5a967207ac0e92b474218965052497b26212f4538eaf @ 2a420c66-2670-0da8-9e55-276ddeeabbeb 0383713043445e71158edc5a967207ac0e92b474218965052497b26212f4538eaf -> 03075370d8cc7f2b66d19795ca47f8bfe26ee5cd755029c59616c900319c2f4b63 @ 0aac49d6-b79d-0723-9add-05390e5bd4aa]
 reverse: [03075370d8cc7f2b66d19795ca47f8bfe26ee5cd755029c59616c900319c2f4b63 -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 423c17d5-c0d8-0b51-87ab-d4502cf504c6 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ 22d0bf88-378f-0af9-8009-060887e0d221]
076d385b607 -> 02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696 @ cd455d03-7698-0a42-bb22-bf99c24ec828 02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ eb54647a-67ab-097b-83ff-b93072e542a2]
0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 @ 5b6598fd-4e13-0a72-8745-f1d587427178 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 -> 0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c @ 8460b401-7258-0f48-ac0d-9f8929f279d9]
 reverse: [0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c -> 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec @ bfb204ff-c558-046c-9742-e5a540d6f8d8 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ e3127e3f-b939-06e2-82cb-e0a091986a82]
0383713043445e71158edc5a967207ac0e92b474218965052497b26212f4538eaf
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 @ 5b6598fd-4e13-0a72-8745-f1d587427178 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 -> 0383713043445e71158edc5a967207ac0e92b474218965052497b26212f4538eaf @ 57abbfd2-b3a8-04b5-b4dd-da7c18da42b4]
 reverse: [0383713043445e71158edc5a967207ac0e92b474218965052497b26212f4538eaf -> 025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 @ 0bedbfbd-c6f5-092a-8259-96ca71cac18f 025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419 -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ bf97fc52-a273-0016-b063-d7a7518fbf13]
03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec @ e3127e3f-b939-06e2-82cb-e0a091986a82 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec -> 03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 @ 3ffbc6d9-f11c-0c7f-9e07-28ba7a081a43]
 reverse: [03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367 -> 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec @ 3ffbc6d9-f11c-0c7f-9e07-28ba7a081a43 029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ e3127e3f-b939-06e2-82cb-e0a091986a82]
03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0
forward: [0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c -> 022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a @ f8a80bcf-35f1-0d67-bd43-37d39e1771e2 022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a -> 03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 @ 5b38f859-3019-05d2-90b2-98c0436eb042]
 reverse: [03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0 -> 022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a @ 5b38f859-3019-05d2-90b2-98c0436eb042 022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a -> 0323272a60895f56aad82cb767fb5c413807adcf7c9fb0578b1b1c5807c7f29d4c @ f8a80bcf-35f1-0d67-bd43-37d39e1771e2]

the above was with 2 hops, however when I check for a route with 3 hops, none are returned

$ skywire cli proxy list -v v1.3.21 -c US | while read _pk ; do echo $_pk ; skywire cli route find -n3 $_pk ; echo ; done
022e504c96fae546deae42f1109a9effed6ba5e427f01ec864f1fedcab6723b25a
[2024-04-17T07:58:32.124210935-05:00] FATAL [skywire-cli]: transport not found

025b8f1a36bbdc1678811c26035ce1b84e92a3bf75ab439790c82143e81a07b419
[2024-04-17T07:58:32.817154881-05:00] FATAL [skywire-cli]: transport not found

02781a0ac87d137d612bdefc5d68f420125887771e2d5cf6bc710c83ffc3b9d696
[2024-04-17T07:58:33.479610006-05:00] FATAL [skywire-cli]: transport not found

029eb3bd20846d6488132310236e2eab7d3b5dd92fb06050c765ce8f0b916eb8ec
[2024-04-17T07:58:34.150888919-05:00] FATAL [skywire-cli]: transport not found

03075370d8cc7f2b66d19795ca47f8bfe26ee5cd755029c59616c900319c2f4b63
[2024-04-17T07:58:34.80636321-05:00] FATAL [skywire-cli]: transport not found

03419b3c3f9db0404b44afc532a91eb2c108499f7575d0e7937dd8d076d385b607
[2024-04-17T07:58:35.466311774-05:00] FATAL [skywire-cli]: transport not found

0362aae9abdff59026a881fe1991e8262a2047c26842334b23744a942b2f298e0c
[2024-04-17T07:58:36.166703458-05:00] FATAL [skywire-cli]: transport not found

0383713043445e71158edc5a967207ac0e92b474218965052497b26212f4538eaf
[2024-04-17T07:58:36.84586608-05:00] FATAL [skywire-cli]: transport not found

03a3dfc9819cd6cddd97e3b2af553151f2c02263e7da0a5d55857d56a5efc59367
[2024-04-17T07:58:37.525304288-05:00] FATAL [skywire-cli]: transport not found

03edb4ea9cc744bed9c72af65c70c2e3b24f3d946f40d9ac4052538d800e1ea2d0
[2024-04-17T07:58:38.176468899-05:00] FATAL [skywire-cli]: transport not found

there should be sufficient transports in this instance to support a route with 3 hops, but none are returned.