Earned fees calculation

[sipsorcery]

The earned fees calculation seems to be double the actual amount. The problem seems to be around here. I'm not sure why fees are counted on both the incoming and outgoing channels? The relay fee is only earned on the outgoing channel.

Commenting out this line results in a calculation that seems closer.

I checked the suez script calculation with this homemade script (could have its own flaw but I did check a daily period by hand and it was correct):

lncli fwdinghistory --start_time -30d --max_events 5000 | grep '"fee"' | sed -nr 's/.*"([0-9]+).*/\1/p' | awk 'BEGIN {total=0;}{total+=$1;}END {print "Total: ",total}'

On my node, even after applying the fix mentioned above, my script and suez still result in an earned fees discrepancy of 16 sats.

[prusnak]

Isn't the fee in fwdinghistory always the fee earned by my node?

If I do the suggested change, the earned_fees is around half of what lncli feereport shows asmonth_fee_sum. If I keep it unchanged, the numbers are almost the same (difference <100 sat).

[sipsorcery]

Isn't the fee in fwdinghistory always the fee earned by my node?

Yes, that's my understanding. But isn't the suez script counting the fee twice for each forward:

(snippet bwlow adjusted for brevity)

for fe in fwd_events:
    c1 = fe["chan_id_in"]
    c2 = fe["chan_id_out"]
    fee = int(fe["fee"])
    if not c1 in fees:
        fees[c1] = 0
    if not c2 in fees:
        fees[c2] = 0
    fees[c1] += fee  # Why record the fee against the incoming channel?
    fees[c2] += fee 

If I do the suggested change, the earned_fees is around half of what lncli feereport shows as month_fee_sum. If I keep it unchanged, the numbers are almost the same (difference <100 sat).

Very strange. In my case it's the opposite. lncli feereport is approx. half of what suez calculates. If I adjust the suez script to not record the forwarding fee against the incoming channel they are within 48 sats.

[prusnak]

My understanding is that one of the channels listed in fwd_events is mine, the other is not mine. I record both, because I don't know which is mine or not at this point. But later I look into the correct records, leaving not mine channels without access.

I might be wrong, though.

[prusnak]

I did a small refactor in 094e9529430a52974d9c598205eefafc47d8c33d, but that does not change functionality at all.

When I run bos forwards --days 30, it shows Earned In and Earned Out with total matching value in my table, so it seems node earns fees for both in and out channels.

bos = https://github.com/alexbosworth/balanceofsatoshis

[prusnak]

Btw, there will be always discrepancies when a channel was closed in the last 30 days - my table does not show closed channels at all, while lncli fee_report and lncli fwdinghistory take these into account.

[sipsorcery]

When I run bos forwards --days 30, it shows Earned In and Earned Out with total matching value in my table, so it seems node earns fees for both in and out channels.

I think bos does some kind of double entry book-keeping in the bos forwards table. The fact the Earned In total exactly matches the Earned Out table supports that likelihood.

I did see Alex Bosworth explain this to someone else, I think it was on the Telegram group. I'll see if I can dig up the explanation.

Btw, there will be always discrepancies when a channel was closed in the last 30 days - my table does not show closed channels at all, while lncli fee_report and lncli fwdinghistory take these into account.

Ah, I didn't even think of that. That could explain the 48 sat discrepancy I am getting. I'll see if I can double check that on my node somehow.

[sipsorcery]

Alex confirmed that the Earned In and Earned Out columns produced by bos forwards are using double entries. The total earned fees is Sum(Earned Out), (which is == Sum(Earned In)).

[prusnak]

I have trouble understanding why all these 3 values are the same on my system:

• month_fee_sum in lncli feereport
• your script from the OP
• earned fees total in suez table

[prusnak]

Argh, I get it now. I had one closed channel that earned approx 1/2 of the total fees, which confused me ...

Thanks

[sipsorcery]

How come lncli feereport shows approx the same value in month_fee_sum as in suez earned fees total?

That I can't explain, unless, as you alluded to, the fees from closed channels in the lncli feereport coincidentally match the value of your active channels.

For one of my nodes:

$ lncli feereport "month_fee_sum": "2039"

$ suez earned fees (sat) 3,998

$ bos forwards --days 30 sum(Earned Out)=2007

$ lncli fwdinghistory --start_time -30d --max_events 5000 | grep '"fee"' | sed -nr 's/."([0-9]+)./\1/p' | a wk 'BEGIN {total=0;}{total+=$1;}END {print "Total: ",total}' Total: 2007

[prusnak]

I believe I fixed the issue in https://github.com/prusnak/suez/commit/a0d43b9057fab19ed19a0549135e0962e34e13d2 (by splitting into local and remote fees). Can you please confirm the numbers are now what you expect?

[sipsorcery]

I believe I fixed the issue in a0d43b9 (by splitting into local and remote fees). Can you please confirm the numbers are now what you expect?

Pretty close.

I have a discrepancy between the local fees (sat) total of 1991 and the remote fees (sat) total of 2007. A channel I closed in the last 30 days did earn a relay fee of 16. The closed channel does show up in bos forwards. Probably not that big a deal.

IMHO the local & remote fee columns do create a bit of confusion. I found the previous earned fees column more intuitive. There is only one relay fee for each lightning payment, which is the relay fee on the outgoing channel, so it's not as if there really is a local and remote component. This is purely personal preference though. The script now looks to be working well enough for me. Thanks for your work!