search

Blockstream / greenlight

Build apps using self-custodial lightning nodes in the cloud
https://blockstream.github.io/greenlight/getting-started/
MIT License
117 stars 27 forks source link

Payment fails with `We don't have any channels` #427

Closed roeierez closed 4 months ago

roeierez commented 6 months ago

Here are some relevant logs. It seems that in this case there is no attempt to pay. Perhaps the log line “we don’t have any channels” hint that the pay plugin can’t find active channels? I checked and my channel is normal and active. NodeID: 02de1e98d0f87a1a5d9674f33d997b9c63cb65b27e10319cfa83b1b5ab58913f86

Waiting to re-establish a channel: Deadline reached while waiting for at least one channel to be re-established\n\nStack backtrace:\n   0: <gl_plugin_internal::node::InternalNodeServer<R> as cln_grpc::pb::node_server::Node>::pay::{{closure}}\n   1: <<cln_grpc::pb::node_server::NodeServer<T> as tower_service::Service<http::request::Request<B>>>::call::PaySvc<T> as tonic::server::service::UnaryService<cln_grpc::pb::PayRequest>>::call::{{closure}}\n   2: <cln_grpc::pb::node_server::NodeServer<T> as tower_service::Service<http::request::Request<B>>>::call::{{closure}}\n   3: <gl_plugin::node::rpcwait::RpcWaitService<S> as tower_service::Service<http::request::Request<hyper::body::body::Body>>>::call::{{closure}}\n   4: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll\n   5: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll\n   6: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll\n   7: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll\n   8: <tonic::transport::service::router::RoutesFuture as core::future::future::Future>::poll\n   9: <gl_plugin::node::SignatureContextService<S> as tower_service::Service<http::request::Request<hyper::body::body::Body>>>::call::{{closure}}\n  10: <tonic::transport::server::recover_error::ResponseFuture<F> as core::future::future::Future>::poll\n  11: <tonic::transport::server::SvcFuture<F> as core::future::future::Future>::poll\n  12: <hyper::proto::h2::server::H2Stream<F,B> as core::future::future::Future>::poll\n  13: tokio::runtime::task::core::Core<T,S>::poll\n  14: tokio::runtime::task::harness::Harness<T,S>::poll\n  15: tokio::runtime::scheduler::multi_thread::worker::Context::run_task\n  16: tokio::runtime::scheduler::multi_thread::worker::Context::run\n  17: tokio::runtime::context::set_scheduler\n  18: tokio::runtime::context::runtime::enter_runtime\n  19: tokio::runtime::scheduler::multi_thread::worker::run\n  20: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll\n  21: tokio::runtime::task::core::Core<T,S>::poll\n  22: tokio::runtime::task::harness::Harness<T,S>::poll\n  23: tokio::runtime::blocking::pool::Inner::run\n  24: std::sys_common::backtrace::__rust_begin_short_backtrace\n  25: core::ops::function::FnOnce::call_once{{vtable.shim}}\n  26: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2015:9\n      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/alloc/src/boxed.rs:2015:9\n      std::sys::unix::thread::thread::new::thread_start\n             at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys/unix/thread.rs:108:17\n  27: start_thread\n  28: clone
[BreezSdk] {INFO} (2024-05-08T06:04:41.914220Z) : node-logs: DEBUG   plugin-gl-plugin-internal: We should be ready to send a payment.
[BreezSdk] {INFO} (2024-05-08T06:04:41.915741Z) : node-logs: DEBUG   plugin-gl-plugin-internal: Calling pay on the node
[BreezSdk] {INFO} (2024-05-08T06:04:44.052337Z) : node-logs: DEBUG   plugin-pay: gossmap ignored 615 channel updates
[BreezSdk] {INFO} (2024-05-08T06:04:44.053128Z) : node-logs: DEBUG   plugin-pay: cmd 34 partid 0: After filtering routehints we're left with 1 usable hints
[BreezSdk] {INFO} (2024-05-08T06:04:44.054276Z) : node-logs: DEBUG   plugin-pay: cmd 34 partid 0: Using routehint 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f (16729456x2238734x53272) cltv_delta=144
[BreezSdk] {INFO} (2024-05-08T06:04:44.055275Z) : node-logs: DEBUG   plugin-pay: cmd 34 partid 0: The destination is not directly reachable excluding attempts without routehints
[BreezSdk] {INFO} (2024-05-08T06:04:44.056374Z) : node-logs: INFO    plugin-pay: cmd 34 partid 0: Initial limit on max HTLCs: 15, Destination 02c560e8c5fad3379be37b2e3b61544f3b9fc7ac82234cef269e457022290ed018 has 1 channels, assuming 15 HTLCs per channel
[BreezSdk] {INFO} (2024-05-08T06:04:44.057810Z) : node-logs: INFO    plugin-pay: cmd 34 partid 0: We don't have any channels
[BreezSdk] {INFO} (2024-05-08T06:04:44.060278Z) : node-logs: 
[BreezSdk] {INFO} (2024-05-08T06:04:44.060893Z) : node-logs: INFO    plugin-pay: cmd 34 partid 0: Lower limit of adaptive splitter reached (11000msat < 100000msat), not splitting further.
[BreezSdk] {INFO} (2024-05-08T06:04:44.061945Z) : node-logs: DEBUG   plugin-gl-plugin-internal: pay returned: Err(\n    Status {\n        code: Unknown,\n        message: \"Error calling method Pay: RpcError { code: Some(210), message: \\\"Ran out of routes to try after 1 attempt: see `paystatus`\\\", data: None }\",\n        source: None,\n    },\n)
[BreezSdk] {INFO} (2024-05-08T06:04:44.062783Z) : node-logs: DEBUG   plugin-gl-plugin-internal: Configuring watchtower at https://10.168.0.7:2603/
[BreezSdk] {CONFIG} (2024-05-08T06:04:44.072964Z) : received frame=Headers { stream_id: StreamId(33), flags: (0x5: END_HEADERS | END_STREAM) }
[BreezSdk] {CONFIG} (2024-05-08T06:04:44.103204Z) : Sending request Request { method: POST, uri: /cln.Node/Getinfo, version: HTTP/2.0, headers: {"te": "trailers", "content-type": "application/grpc", "glauthpubkey": "BH9hWxQUT+FUBShx3ODBjKYKcmGk7GDlLMPU4b0YSJslRmIVooJQeIcOkLtCf4PuM/Y+nqAR/fmDQcieNxSsfl8", "glauthsig": "MrF2cq1BcA3vB8IzAcexIiIflH//bC8BpSR77lXNrLA9tq/nTU+MdYUARrQtA+LYq60ppzV0zy2TnICV/4mdgw", "glts": "AAABj1bN4ME"}, body: UnsyncBoxBody }
[9:19](https://breez-tech.slack.com/archives/C033VG4L7LY/p1715149145048789)
cdecker commented 5 months ago

Having seen a couple of instances of this by now, including #443, I think we can say that this error originates in the pay plugin when we can't find ourselves in the network topology, or we can't find any channels of ours in the network topology. It does not mean that the node has forgotten about the channel it has, just that it doesn't appear in the gossip.

Since this is going to be fixed in v24.02gl1 I'm tempted to use that as the fix.

cdecker commented 4 months ago

As we begin to roll out v24.02gl1 we can close this issue as resolved. Feel free to reopen if it manifests again.