fiatjaf
commented
5 years ago Apparently the continuous crashes are related to calls to listsendpays too. But I don't get it. lntxbot doesn't call listsendpays ever.
Closed fiatjaf closed 5 years ago
fiatjaf
commented
5 years ago Apparently the continuous crashes are related to calls to listsendpays too. But I don't get it. lntxbot doesn't call listsendpays ever.
fiatjaf
commented
5 years ago I've deleted the pay plugin from the plugins folder as it was the only thing I could find that could be calling listsendpays, the the node stopped crashing continuously. I don't understand why is that, as lntxbot doesn't call pay ever either, but anyway, now the problem is narrowed down to the node crashing on listinvoices and listsendpays as shown in the stack dumps above. Most other commands work fine, including invoice (and receiving payments) and decodepay, getroute, sendpay etc.
fiatjaf
commented
5 years ago Dear reader, please ignore all of the above and focus on the following part, as it is the error common to all errors:
Sep 22 02:32:04 lntxbot lightningd[1008]: 0x56181c6ed6be send_backtrace
Sep 22 02:32:04 lntxbot lightningd[1008]: common/daemon.c:40
Sep 22 02:32:04 lntxbot lightningd[1008]: 0x56181c6ed751 crashdump
Sep 22 02:32:04 lntxbot lightningd[1008]: common/daemon.c:53
Sep 22 02:32:04 lntxbot lightningd[1008]: 0x7f4305a01f1f ???
Sep 22 02:32:04 lntxbot lightningd[1008]: ???:0
Sep 22 02:32:04 lntxbot lightningd[1008]: 0x7f4305a7e578 ???
Sep 22 02:32:04 lntxbot lightningd[1008]: ???:0
Sep 22 02:32:04 lntxbot lightningd[1008]: 0x56181c729b70 memcpy
Sep 22 02:32:04 lntxbot lightningd[1008]: /usr/include/x86_64-linux-gnu/bits/string_fortified.h:34
Sep 22 02:32:04 lntxbot lightningd[1008]: 0x56181c729b70 tal_dup_
Sep 22 02:32:04 lntxbot lightningd[1008]: ccan/ccan/tal/tal.c:796
Sep 22 02:32:04 lntxbot lightningd[1008]: 0x56181c727e81 tal_strdup_
Sep 22 02:32:04 lntxbot lightningd[1008]: ccan/ccan/tal/str/str.c:18
ZmnSCPxj
commented
5 years ago tal_strdup_ is very reliable code, but naively reads whatever pointer is passed in. The error is in its callers, wallet_stmt2invoice_details and wallet_stmt2payment. This makes me suspect the recent changes regarding database abstraction, simply because that is the newest part of the codebase. In particular we used to use the term stmt to mean a SQL statement currently being queried, and the stmt2whatever functions convert from the SQL statement results to whatever type.
rustyrussell
commented
5 years ago @ZmnSCPxj nailed it.
Looks like a four-char typo in wallet/db.c:
diff --git a/wallet/db.c b/wallet/db.c
index 2e741ab1f..7e6cee8a2 100644
--- a/wallet/db.c
+++ b/wallet/db.c
@@ -567,7 +567,7 @@ const void *db_column_blob(struct db_stmt *stmt, int col)
const unsigned char *db_column_text(struct db_stmt *stmt, int col)
{
- return stmt->db->config->column_blob_fn(stmt, col);
+ return stmt->db->config->column_text_fn(stmt, col);
}
size_t db_count_changes(struct db_stmt *stmt)Does changing blob to text there fix it?
Issue and Steps to Reproduce
I always update my node to
master. Today I realized I was on 0.7.0-something, so I updated withgit pulland recompiled. For some reason I was still on0.7.0after that, but nowlistpaymentsdidn't work, so I guess I was really updated (I don't know when that was removed). However running it crashed the daemon:Then I checked out the release 0.7.1 as I thought
masterwas probably broken. Butlightningdfailed to start there, complaining about a move from database v107 to v104. So I went back tomasteragain and now for some reason it was0.7.2-something. Maybe this story is worthless and just a picture of my own confusion, but nonetheless it's told here.Now
listsendpaysandlistinvoicesboth give me errors (and the daemon crashes):Some other basic commands work well. However the daemon it's crashing and restarting by itself every minute or so for no reason:
Is it crashing due to lack of memory? It cannot be. This is a very small machine, but it has only lightningd and lntxbot running, and both use few memory and there were no big changes, but maybe, who knows, i don't understand these crash dumps.
getinfooutput