Open BestQualityVacuum opened 1 year ago
kristapsk
commented
1 year ago Onion messaging will not work as: 1) you are trying to connect to mainnet directory nodes, 2) AFAIK nobody runs directory nodes for testnet3, for onion messaging thhere is only mainnet or signet available currently (although people are free to start running testnet3 directory nodes too). See directory_nodes setting in joinmarket.cfg.
BestQualityVacuum
commented
1 year ago Onion messaging will not work as: 1) you are trying to connect to mainnet directory nodes, 2) AFAIK nobody runs directory nodes for testnet3, for onion messaging thhere is only mainnet or signet available currently (although people are free to start running testnet3 directory nodes too). See
directory_nodessetting injoinmarket.cfg.
Disappointed to read this. Joinmarket is a relatively hard software to setup and use. I would definitely want to try it out with worthless coins first, and play a little bit with it; maybe lose some coins during this process too.
I guess I should acquire some signet coins. Thanks.
kristapsk
commented
1 year ago People are free to spin up testnet directory nodes and open a PR to add them to default joinmarket.cfg. There are no technological limitations.
I guess I should acquire some signet coins.
There are signet faucets available where you can get free signet coins, for example, https://signetfaucet.com/.
AdamISZ
commented
1 year ago Yes 100% use signet (I lost all trust in trying to use testnet3 years ago during segwit times, I would never bother with it since signet is just better, for our use case).
Your problem though, in trying the software out with riskless coins, is that people are usually not incented to bother setting up signet bots for any length of time (they do it, now and again, especially if we want to test some radically new JM setup). So if you want to test it you end up running several bots yourself. In that case, why not use regtest? If you look at docs/TESTING.md it has a suggested way of running on regtest with yield generator bots automatically started up for you, along with a local IRC instance and a local bitcoind node backend.
If you specifically are interested in testing how well onion messaging functions though, you probably may as well go with setting multiple bots up on signet (and yes, there are a couple of signet directory nodes that are left running long term, so that works). It depends what functionality you're looking to test.
AdamISZ
commented
1 year ago I've changed the title, hope that's OK
BestQualityVacuum
commented
1 year ago Just for your information, I've configured it to signet, but it still returns me these "Starting factory" and "Stopping factory" messages. Should I try mainnet?
AdamISZ
commented
1 year ago Just for your information, I've configured it to signet, but it still returns me these "Starting factory" and "Stopping factory" messages. Should I try mainnet?
'Starting factory' is normal. I just tried (see attached:) logofsignetstart.txt
You need to have the signet directory nodes uncommented (and therefore used) in the joinmarket.cfg file, and remove the mainnet d-nodes. You also obviously have to have your signet bitcoind node running and the joinmarket config pointing at it with the right port and credentials. (the [BLOCKCHAIN] section; the comments guide you). There are a lot of network level logging messages as you can see; we leave those on by default when running non-mainnet.
My advice here is generic because I don't know exactly what happened.
BestQualityVacuum
commented
1 year ago @AdamISZ
I do have starting factory messages too, but the difference with your log, is that you don't have a single "Stopping factory", and I have lots (once it starts the factory, it stops it).
Does it work? Like, am I visible to the orderbook? Can I provide you a nickname to check?
AdamISZ
commented
1 year ago Sorry busy but first glance response is: that looks like failure to connect outbound to something over Tor, using the socks proxy. The 'jmdaemon complete' is telling you that at least some of your message channels are set up and working; do you perhaps have both IRC and onion message channels configured? I would (a) check thoroughly that the socks configuration in the config file is correct (can you make outbound Tor connections successfully in any configuration) and (b) isolate the problem by enabling only one message channel (and, even, only one directory node, within the onion message channel), to locate the issue.
BestQualityVacuum
commented
1 year ago Hey, thanks for your quick response.
As you can see below, I do have multiple IRC channels. I also do connect to Tor properly; it is, indeed, listening to localhost:9050, which is used by other applications as well.
This is my joinmarket.cfg. Confirm me to do your second proposal (b).
[DAEMON]
# set to 1 to run the daemon service within this process;
# set to 0 if the daemon is run separately (using script joinmarketd.py)
no_daemon = 1
# Port on which daemon serves; note that communication still
# occurs over this port even if no_daemon = 1
daemon_port = 27183
# Currently, running the daemon on a remote host is
# *NOT* supported, so don't change this variable
daemon_host = localhost
# by default the client-daemon connection is plaintext, set to 'true' to use TLS;
# for this, you need to have a valid (self-signed) certificate installed
use_ssl = false
[BLOCKCHAIN]
# options: bitcoin-rpc, regtest, bitcoin-rpc-no-history, no-blockchain
# When using bitcoin-rpc-no-history remember to increase the gap limit to scan for more addresses, try -g 5000
# Use 'no-blockchain' to run the ob-watcher.py script in scripts/obwatch without current access
# to Bitcoin Core; note that use of this option for any other purpose is currently unsupported.
blockchain_source = bitcoin-rpc
# options: signet, testnet, mainnet
# Note: for regtest, use network = testnet
network = signet
rpc_host = localhost
# default ports are 8332 for mainnet, 18443 for regtest, 18332 for testnet, 38332 for signet
rpc_port =
# Use either rpc_user / rpc_password pair or rpc_cookie_file.
#rpc_user = bitcoin
#rpc_password = password
rpc_cookie_file = /data/bitcoin/signet/.cookie
# rpc_wallet_file is Bitcoin Core wallet which is used for address and
# transaction monitoring (it is watchonly, no private keys are stored there).
# It must be created manually if does not exist, see docs/USAGE.md for more
# information.
rpc_wallet_file = jm_wallet
[MESSAGING:onion]
# onion based message channels must have the exact type 'onion'
# (while the section name above can be MESSAGING:whatever), and there must
# be only ONE such message channel configured (note the directory servers
# can be multiple, below):
type = onion
socks5_host = localhost
socks5_port = 9050
# the tor control configuration.
# for most people running the tor daemon
# on Linux, no changes are required here:
tor_control_host = localhost
# or, to use a UNIX socket
# tor_control_host = unix:/var/run/tor/control
# note: port needs to be provided (but is ignored for UNIX socket)
tor_control_port = 9051
# the host/port actually serving the hidden service
# (note the *virtual port*, that the client uses,
# is hardcoded to as per below 'directory node configuration'.
onion_serving_host = 127.0.0.1
onion_serving_port = 8090
# directory node configuration
#
# This is mandatory for directory nodes (who must also set their
# own *.onion:port as the only directory in directory_nodes, below),
# but NOT TO BE USED by non-directory nodes (which is you, unless
# you know otherwise!), as it will greatly degrade your privacy.
# (note the default is no value, don't replace it with "").
hidden_service_dir =
#
# This is a comma separated list (comma can be omitted if only one item).
# Each item has format host:port ; both are required, though port will
# be 5222 if created in this code.
# for MAINNET:
#directory_nodes = 3kxw6lf5vf6y26emzwgibzhrzhmhqiw6ekrek3nqfjjmhwznb2moonad.onion:5222,jmdirjmioywe2s5jad7ts6kgcqg66rj6wujj6q77n6wbdrgocqwexzid.onion:5222,bqlpq6ak24mwvuixixitift4yu42nxchlilrcqwk2ugn45tdclg42qid.onion:5222
# for SIGNET (testing network):
directory_nodes = rr6f6qtleiiwic45bby4zwmiwjrj3jsbmcvutwpqxjziaydjydkk5iad.onion:5222,k74oyetjqgcamsyhlym2vgbjtvhcrbxr4iowd4nv4zk5sehw4v665jad.onion:5222,y2ruswmdbsfl4hhwwiqz4m3sx6si5fr6l3pf62d4pms2b53wmagq3eqd.onion:5222
# This setting is ONLY for developer regtest setups,
# running multiple bots at once. Don't alter it otherwise
regtest_count = 0,0
## IRC SERVER 1: Darkscience IRC (Tor, IP)
################################################################################
[MESSAGING:server1]
# by default the legacy format without a `type` field is
# understood to be IRC, but you can, optionally, add it:
# type = irc
channel = joinmarket-pit
port = 6697
usessl = true
# For traditional IP:
#host = irc.darkscience.net
#socks5 = false
# For Tor (recommended as clearnet alternative):
host = darkirc6tqgpnwd3blln3yfv5ckl47eg7llfxkmtovrv7c7iwohhb6ad.onion
socks5 = true
socks5_host = localhost
socks5_port = 9050
## IRC SERVER 2: ILITA IRC (optional IRC alternate, Tor only)
################################################################################
[MESSAGING:server2]
channel = joinmarket-pit
port = 6667
usessl = false
socks5 = true
socks5_host = localhost
host = ilitafrzzgxymv6umx2ux7kbz3imyeko6cnqkvy4nisjjj4qpqkrptid.onion
socks5_port = 9050
## IRC SERVER 3: (backup) hackint IRC (Tor, IP)
################################################################################
#[MESSAGING:server3]
# channel = joinmarket-pit
# For traditional IP:
## host = irc.hackint.org
## port = 6697
## usessl = true
## socks5 = false
# For Tor (default):
#host = ncwkrwxpq2ikcngxq3dy2xctuheniggtqeibvgofixpzvrwpa77tozqd.onion
#port = 6667
#usessl = false
#socks5 = true
#socks5_host = localhost
#socks5_port = 9050
[LOGGING]
# Set the log level for the output to the terminal/console
# Possible choices: DEBUG / INFO / WARNING / ERROR
# Log level for the files in the logs-folder will always be DEBUG
console_log_level = INFO
# Use color-coded log messages to help distinguish log levels?:
color = true
[TIMEOUT]
maker_timeout_sec = 60
unconfirm_timeout_sec = 180
confirm_timeout_hours = 6
[POLICY]
# Use segwit style wallets and transactions
# Only set to false for old wallets, Joinmarket is now segwit only.
segwit = true
# Use native segwit (bech32) wallet. If set to false, p2sh-p2wkh
# will be used when generating the addresses for this wallet.
# Notes: 1. The default joinmarket pit is native segwit.
# 2. You cannot change the type of a pre-existing wallet.
native = true
# for dust sweeping, try merge_algorithm = gradual
# for more rapid dust sweeping, try merge_algorithm = greedy
# for most rapid dust sweeping, try merge_algorithm = greediest
# but don't forget to bump your miner fees!
merge_algorithm = default
# The fee estimate is based on a projection of how many sats/kilo-vbyte
# are needed to get in one of the next N blocks. N is set here as
# the value of 'tx_fees'. This cost estimate is high if you set
# N=1, so we choose 3 for a more reasonable figure, as our default.
# You can also set your own fee/kilo-vbyte: any number higher than 1 thousand
# will be interpreted as the fee in sats/kilo-vbyte that you wish to use.
#
# Example: N=30000 will use 30 thousand sats/kilo-vbyte (30 sats/vB) as a fee,
# while N=5 will use the 5 block estimate from your selected blockchain source.
tx_fees = 3
# Transaction fee rate variance factor, 0.2 means 20% variation around
# any manually chosen values, so if you set tx_fees=10000 and
# tx_fees_factor=0.2, it might use any value between 8 thousand and 12 thousand
# for your transactions.
tx_fees_factor = 0.2
# For users getting transaction fee estimates over an API,
# place a sanity check limit on the sats/kilo-vbyte to be paid.
# This limit is also applied to users using Core, even though
# Core has its own sanity check limit, which is currently
# 1 million satoshis.
#
# Example: N=350000 will use 350 thousand sats/kilo-vbyte (350 sats/vB) as a
# maximum fee.
absurd_fee_per_kb = 350000
# In decimal, the maximum allowable change either lower or
# higher, that the fee rate used for coinjoin sweeps is
# allowed to be.
# (note: coinjoin sweeps *must estimate* fee rates;
# they cannot be exact due to the lack of change output.)
#
# Example: max_sweep_fee_change = 0.4, with tx_fees = 10000,
# means actual fee rate achieved in the sweep can be as low
# as 6 thousand sats/kilo-vbyte up to 14 thousand sats/kilo-vbyte.
#
# If this is not achieved, the transaction is aborted. For tumbler,
# it will then be retried until successful.
# WARNING: too-strict setting may result in using up a lot
# of PoDLE commitments, hence the default 0.8 (80%).
max_sweep_fee_change = 0.8
# Maximum absolute coinjoin fee in satoshi to pay to a single
# market maker for a transaction. Both the limits given in
# max_cj_fee_abs and max_cj_fee_rel must be exceeded in order
# to not consider a certain offer.
#max_cj_fee_abs = x
# Maximum relative coinjoin fee, in fractions of the coinjoin value
# e.g. if your coinjoin amount is 2 btc (200 million satoshi) and
# max_cj_fee_rel = 0.001 (0.1%), the maximum fee allowed would
# be 0.002 btc (200 thousand satoshi)
#max_cj_fee_rel = x
# The range of confirmations passed to the `listunspent` bitcoind RPC call
# 1st value is the inclusive minimum, defaults to one confirmation
# 2nd value is the exclusive maximum, defaults to most-positive-bignum (Google Me!)
# leaving it unset or empty defers to bitcoind's default values, ie [1, 9999999]
#listunspent_args = []
# That's what you should do, unless you have a specific reason, eg:
# !!! WARNING !!! CONFIGURING THIS WHILE TAKING LIQUIDITY FROM
# !!! WARNING !!! THE PUBLIC ORDERBOOK LEAKS YOUR INPUT MERGES
# spend from unconfirmed transactions: listunspent_args = [0]
# display only unconfirmed transactions: listunspent_args = [0, 1]
# defend against small reorganizations: listunspent_args = [3]
# who is at risk of reorganization?: listunspent_args = [0, 2]
# NB: using 0 for the 1st value with scripts other than wallet-tool could cause
# spends from unconfirmed inputs, which may then get malleated or double-spent!
# other counterparties are likely to reject unconfirmed inputs... don't do it.
# tx_broadcast: options: self, random-peer, not-self.
#
# self = broadcast transaction with your own bitcoin node.
#
# random-peer = everyone who took part in the coinjoin has a chance of broadcasting
# Note: if your counterparties do not support it, you will fall back
# to broadcasting via your own node.
#
# not-self = never broadcast with your own bitcoin node.
#
# Note: in this case if your counterparties do not broadcast for you, you
# will have to broadcast the tx manually (you can take the tx hex from the log
# or terminal) via some other channel. It is not recommended to choose this
# option when running schedules/tumbler.
tx_broadcast = random-peer
# If makers do not respond while creating a coinjoin transaction,
# the non-responding ones will be ignored. This is the minimum
# amount of makers which we are content with for the coinjoin to
# succeed. Less makers means that the whole process will restart
# after a timeout.
minimum_makers = 1
# Threshold number of satoshis below which an incoming utxo
# to a reused address in the wallet will be AUTOMATICALLY frozen.
# This avoids forced address reuse attacks; see:
# https://en.bitcoin.it/wiki/Privacy#Forced_address_reuse
#
# The default is to ALWAYS freeze a utxo to an already used address,
# whatever the value of it, and this is set with the value -1.
max_sats_freeze_reuse = -1
# Interest rate used when calculating the value of fidelity bonds created
# by locking bitcoins in timelocked addresses
# See also:
# https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b#determining-interest-rate-r
# Set as a real number, i.e. 1 = 100% and 0.01 = 1%
interest_rate = 0.015
# Some makers run their bots to mix their funds not just to earn money
# So to improve privacy very slightly takers dont always choose a maker based
# on his fidelity bond but allow a certain small percentage to be chosen completely
# randomly without taking into account fidelity bonds
# This parameter sets how many makers on average will be chosen regardless of bonds
# A real number, i.e. 1 = 100%, 0.125 = 1/8 = 1 in every 8 makers on average will be bondless
bondless_makers_allowance = 0.125
# To (strongly) disincentivize Sybil behaviour, the value assessment of the bond
# is based on the (time value of the bond)^x where x is the bond_value_exponent here,
# where x > 1. It is a real number (so written as a decimal).
bond_value_exponent = 1.3
##############################
# THE FOLLOWING SETTINGS ARE REQUIRED TO DEFEND AGAINST SNOOPERS.
# DON'T ALTER THEM UNLESS YOU UNDERSTAND THE IMPLICATIONS.
##############################
# Number of retries allowed for a specific utxo, to prevent DOS/snooping.
# Lower settings make snooping more expensive, but also prevent honest users
# from retrying if an error occurs.
taker_utxo_retries = 3
# Number of confirmations required for the commitment utxo mentioned above.
# this effectively rate-limits a snooper.
taker_utxo_age = 5
# Percentage of coinjoin amount that the commitment utxo must have
# as a minimum BTC amount. Thus 20 means a 1BTC coinjoin requires the
# utxo to be at least 0.2 btc.
taker_utxo_amtpercent = 20
# Set to 1 to accept broadcast PoDLE commitments from other bots, and
# add them to your blacklist (only relevant for Makers).
# There is no way to spoof these values, so the only "risk" is that
# someone fills your blacklist file with a lot of data.
accept_commitment_broadcasts = 1
# Location of your commitments.json file (stores commitments you've used
# and those you want to use in future), relative to the scripts directory.
commit_file_location = cmtdata/commitments.json
# Location of the file used by makers to keep track of used/blacklisted
# commitments. For remote daemon, set to `.` to have it stored locally
# (but note that *all* bots using the same code installation share it,
# in this case, which can be bad in testing).
commitment_list_location = cmtdata/commitmentlist
##############################
# END OF ANTI-SNOOPING SETTINGS
##############################
[PAYJOIN]
# For the majority of situations, the defaults
# need not be altered - they will ensure you don't pay
# a significantly higher fee.
# MODIFICATION OF THESE SETTINGS IS DISADVISED.
# Payjoin protocol version; currently only '1' is supported.
payjoin_version = 1
# Servers can change their destination address by default (0).
# if '1', they cannot. Note that servers can explicitly request
# that this is activated, in which case we respect that choice.
disable_output_substitution = 0
# "default" here indicates that we will allow the receiver to
# increase the fee we pay by:
# 1.2 * (our_fee_rate_per_vbyte * vsize_of_our_input_type)
# (see https://github.com/bitcoin/bips/blob/master/bip-0078.mediawiki#span_idfeeoutputspanFee_output)
# (and 1.2 to give breathing room)
# which indicates we are allowing roughly one extra input's fee.
# If it is instead set to an integer, then that many satoshis are allowed.
# Additionally, note that we will also set the parameter additionafeeoutputindex
# to that of our change output, unless there is none in which case this is disabled.
max_additional_fee_contribution = default
# This is the minimum sats/vbyte we allow in the payjoin
# transaction; note it is decimal, not integer.
min_fee_rate = 1.1
# For payjoins as sender (i.e. client) to hidden service endpoints,
# the socks5 configuration:
onion_socks5_host = localhost
onion_socks5_port = 9050
# For payjoin onion service creation:
# the tor control configuration:
tor_control_host = localhost
# or, to use a UNIX socket
# control_host = unix:/var/run/tor/control
# note: port needs to be provided (but is ignored for UNIX socket)
tor_control_port = 9051
# the host/port actually serving the hidden service
# (note the *virtual port*, that the client uses,
# is hardcoded to 80):
onion_serving_host = 127.0.0.1
onion_serving_port = 8080
# in some exceptional case the HS may be SSL configured,
# this feature is not yet implemented in code, but here for the
# future:
hidden_service_ssl = false
[YIELDGENERATOR]
# [string, 'reloffer' or 'absoffer'], which fee type to actually use
ordertype = absoffer
# [satoshis, any integer] / absolute offer fee you wish to receive for coinjoins (cj)
cjfee_a = 0
# [fraction, any str between 0-1] / relative offer fee you wish to receive based on a cj's amount
cjfee_r = 0.00001
# [fraction, 0-1] / variance around the average fee. Ex: 200 fee, 0.2 var = fee is btw 160-240
cjfee_factor = 0.1
# [satoshis, any integer] / the average transaction fee you're adding to coinjoin transactions
# (note: this will soon be deprecated; leave at zero)
txfee_contribution = 0
# [fraction, 0-1] / variance around the average fee. Ex: 1000 fee, 0.2 var = fee is btw 800-1200
txfee_contribution_factor = 0.3
# [satoshis, any integer] / minimum size of your cj offer. Lower cj amounts will be disregarded
minsize = 100000
# [fraction, 0-1] / variance around all offer sizes. Ex: 500k minsize, 0.1 var = 450k-550k
size_factor = 0.1
gaplimit = 6
[SNICKER]
# Any other value than 'true' will be treated as False,
# and no SNICKER actions will be enabled in that case:
enabled = false
# In satoshis, we require any SNICKER to pay us at least
# this much (can be negative), otherwise we will refuse
# to sign it:
lowest_net_gain = 0
# Comma separated list of servers (if port is omitted as :port, it
# is assumed to be 80) which we will poll against (all, in sequence); note
# that they are allowed to be *.onion or cleartext servers, and no
# scheme (http(s) etc) needs to be added to the start.
servers = cn5lfwvrswicuxn3gjsxoved6l2gu5hdvwy5l3ev7kg6j7lbji2k7hqd.onion,
# How many minutes between each polling event to each server above:
polling_interval_minutes = 60OK, so I checked my running signet directory nodes and saw something that may be relevant (my guess, yes, but I can't be sure):
Starting Dec 19th and also continuing through to yesterday (25th) I am seeing the stack trace listed below happening regularly. It's a bug in as much as the directory node should of course be able to handle this case by rejecting the bot sending the message, not crashing (so I will make a patch shortly). The subsidiary question is (which might be relevant to you), is a function of what this stack trace means, which I will endeavour to explain below.
Basically, what I believe happens here, is something like:
OnionPeer object was created on connection, and currently only stores the location of the peer (specifically, a localhost port, because it is inbound not outbound)nick (a string starting with J5..).nick field (see jmdaemon.onionmc.OnionPeer.get_nick_peerlocation_ser(), and explicitly crashes when it sees that the nick is an empty string.So as before, I will fix that bug, but I'm still not 100% sure whether this is totally, partially, or not at all related to your case.
If we temporarily assume that it is related: that your bot is simply repeatedly trying to connect to signet directory node 1, and getting the connection rejected, and causing that crash message, then the obvious question is .. why? And why wouldn't it happen on both signet d-nodes? Beats me. Signet node 2 has nothing but an entirely normal flow of connection (successful) logging messages.
stn021
commented
1 year ago I started testing JM a few weeks ago. In my experience so far testnet would not have been very informative.
It becomes interesting when funds are set via coinjoin or when someone actually uses my offers + bonds. Both will not happen on testnet. Instead I use small amounts of BTC on mainnet. That way my offers and bonds are actually used occasionally and I have sent funds a few times, again using small amounts for now.
The JM-offers are on normal bitcoin-addresses that can also be redundently accessed with another wallet like electrum. So I am not worried about those.
The bonds are on special addresses with special derivation-paths that can only be accessed with the JM-wallet. AFAIK No other wallet can import the private keys. Frankly I am still trying to decide if I trust this "closed" arrangement without a second independent way to access my funds. For now I would have to trust that timelocked funds are always reliably unlocked on the expected date and can then reliably be transferred to other addresses using only the JM-wallet.
So far I have no definitive information about anyone permanently losing funds on JM. But (again frankly) that does not mean that this has never happened.
AdamISZ
commented
1 year ago AFAIK No other wallet can import the private keys.
You are correct. And the docs explicitly state this. There is a proposal to create a BIP standard for such derivation paths, see https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/993#issuecomment-1114175147 .
Frankly I am still trying to decide if I trust this "closed" arrangement without a second independent way to access my funds.
I 100% agree that you should be extremely cautious about all wallet functionalities, and especially custom functionalities that are not broadly supported. But, I do object to using the term "closed" (even if I get what you mean), because it's categorically not closed - the source is open, here, and so is the algorithm. I personally spent a lot of time testing this because it's extremely security-critical to people's funds, but I would never expect people to "just trust it". Don't use it if you're unsure.
stn021
commented
1 year ago But, I do object to using the term "closed" (even if I get what you mean), ...
Yes, apologies, I could not think of a better word. Your term "custom functionalities" is what I meant.
I've been following Raspibolt's Joinmarket installation guide: https://raspibolt.org/guide/bonus/bitcoin/joinmarket.html, except that I've configured it to work on testnet.
I want to run a yield generator bot. I run the yg-privacyenhanced.py script with:
./yg-privacyenhanced.py wallet.jmdatAnd get this:
Apparently, onion messaging doesn't function, which is weird, because I'm properly connected with the Tor network, and propagate blocks and transactions only via Tor. My electrs server is also connectable with Tor.
Then, I repeatedly get bunch of these messages: