search

JoinMarket-Org / joinmarket-clientserver

Bitcoin CoinJoin implementation with incentive structure to convince people to take part
GNU General Public License v3.0
721 stars 176 forks source link

Regtest wallet creation via the API fails #1382

Closed Fonta1n3 closed 1 year ago

Fonta1n3 commented 1 year ago

I am using bitcoin v23.0 on regtest. When I run https://xxx:28183/api/v1/wallet/create to my jm server I get the following error:

2022-10-13 17:59:49,038 [WARNING]  Invalid address "tb1qc7h7ge6hk3ld4yeg3d8kcduk2dfq2s88f2s4wh342jeaq8txvyfq4f7w5x"
2022-10-13 17:59:49,038 [WARNING]  Invalid address "tb1qnn9ga8zgm3gc4mu2lclpnyd6e9gr3hes5k54w2rxh2m8jgpjhgwq7rfvr2"
2022-10-13 17:59:49,038 [WARNING]  Invalid address "tb1qwa4t32l0hy0c667k342ljt7w2578yzk0h9gspwfvwaj2ecx7annqj5em6m"
2022-10-13 17:59:49,039 [WARNING]  Invalid address "tb1q0ep8xvtaugkcr2nckwflmlmfv6gls8z9zzn4f89cxl25362jx5usg24esg"
2022-10-13 17:59:49,039 [WARNING]  Invalid address "tb1q4fha7wyeesa3v5vc35dunnfae7ldkef5ztxs0jjarlkzzkr7x62swxl9yh"
2022-10-13 17:59:49,039 [WARNING]  Invalid address "tb1qzuceejghvlq7754knjrgmt9y4wp4srjpjz9twpy58n0pmmyva93qupc5lk"
2022-10-13 17:59:49,039 [WARNING]  Invalid address "tb1qztqtzezgcp9ssl6hpuf34kvu5zv8ha2eqlac0jm9fjcdau7qc84sg5qrk9"
2022-10-13 17:59:49,039 [WARNING]  Invalid address "tb1qlxl22jmhs94ntnnn86zc4x8mmzashcep863whkayf40n344egf9spas0lf"
2022-10-13 17:59:49,039 [WARNING]  Invalid address "tb1qszy62emyp54rqrr7zrmq3ehl5y3p9p4dt8jc77kpujl3apu7ptcsuvgz2z"
Fatal sync error: import of 1020 address(es) failed for some reason. To prevent coin or privacy loss, Joinmarket will not load a wallet in this conflicted state. Try using a new Bitcoin Core wallet to sync this Joinmarket wallet, or use a new Joinmarket wallet.
Unhandled error in Deferred:

Traceback (most recent call last):
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/base.py", line 1325, in mainLoop
    reactorBaseSelf.runUntilCurrent()
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/base.py", line 991, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/defer.py", line 661, in callback
    self._startRunCallbacks(result)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/defer.py", line 763, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/defer.py", line 857, in _runCallbacks
    current.result = callback(  # type: ignore[misc]
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/task.py", line 869, in cb
    return callable(*args, **kw)
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_service.py", line 490, in sync_wallet
    self.sync_wallet_fast()
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_service.py", line 518, in sync_wallet_fast
    self.sync_addresses_fast()
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_service.py", line 563, in sync_addresses_fast
    self.sync_addresses()
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_service.py", line 752, in sync_addresses
    import_needed = self.bci.import_addresses_if_needed(addresses,
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/blockchaininterface.py", line 300, in import_addresses_if_needed
    self.import_addresses(addresses - imported_addresses, wallet_name)
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/blockchaininterface.py", line 290, in import_addresses
    sys.exit(EXIT_FAILURE)
builtins.SystemExit: 1

My joinmarket.cfg:

[DAEMON]
# set to 1 to run the daemon service within this process;
# set to 0 if the daemon is run separately (using script joinmarketd.py)
no_daemon = 1

# Port on which daemon serves; note that communication still
# occurs over this port even if no_daemon = 1
daemon_port = 27183

# Currently, running the daemon on a remote host is
# *NOT* supported, so don't change this variable
daemon_host = localhost

# by default the client-daemon connection is plaintext, set to 'true' to use TLS;
# for this, you need to have a valid (self-signed) certificate installed
use_ssl = false

[BLOCKCHAIN]
# options: bitcoin-rpc, regtest, bitcoin-rpc-no-history, no-blockchain
# When using bitcoin-rpc-no-history remember to increase the gap limit to scan for more addresses, try -g 5000
# Use 'no-blockchain' to run the ob-watcher.py script in scripts/obwatch without current access
# to Bitcoin Core; note that use of this option for any other purpose is currently unsupported.
blockchain_source = bitcoin-rpc

# options: signet, testnet, mainnet
# Note: for regtest, use network = testnet
network = testnet

rpc_host = localhost
# default ports are 8332 for mainnet, 18443 for regtest, 18332 for testnet, 38332 for signet
rpc_port = 18443

# Use either rpc_user / rpc_password pair or rpc_cookie_file.
rpc_user = bitcoinrpc
rpc_password = password
#rpc_cookie_file =

# rpc_wallet_file is Bitcoin Core wallet which is used for address and
# transaction monitoring (it is watchonly, no private keys are stored there).
# It must be created manually if does not exist, see docs/USAGE.md for more
# information.
rpc_wallet_file = jm-test-wallet

[MESSAGING:onion]
# onion based message channels must have the exact type 'onion'
# (while the section name above can be MESSAGING:whatever), and there must
# be only ONE such message channel configured (note the directory servers
# can be multiple, below):
type = onion

socks5_host = localhost
socks5_port = 9050

# the tor control configuration.
# for most people running the tor daemon
# on Linux, no changes are required here:
tor_control_host = localhost
# or, to use a UNIX socket
# tor_control_host = unix:/var/run/tor/control
# note: port needs to be provided (but is ignored for UNIX socket)
tor_control_port = 9051

# the host/port actually serving the hidden service
# (note the *virtual port*, that the client uses,
# is hardcoded to as per below 'directory node configuration'.
onion_serving_host = 127.0.0.1
onion_serving_port = 8080

# directory node configuration
#
# This is mandatory for directory nodes (who must also set their
# own *.onion:port as the only directory in directory_nodes, below),
# but NOT TO BE USED by non-directory nodes (which is you, unless
# you know otherwise!), as it will greatly degrade your privacy.
# (note the default is no value, don't replace it with "").
hidden_service_dir =
#
# This is a comma separated list (comma can be omitted if only one item).
# Each item has format host:port ; both are required, though port will
# be 5222 if created in this code.
# for MAINNET:
directory_nodes = 3kxw6lf5vf6y26emzwgibzhrzhmhqiw6ekrek3nqfjjmhwznb2moonad.onion:5222,jmdirjmioywe2s5jad7ts6kgcqg66rj6wujj6q77n6wbdrgocqwexzid.onion:5222,bqlpq6ak24mwvuixixitift4yu42nxchlilrcqwk2ugn45tdclg42qid.onion:5222

# for SIGNET (testing network):
# directory_nodes = rr6f6qtleiiwic45bby4zwmiwjrj3jsbmcvutwpqxjziaydjydkk5iad.onion:5222,k74oyetjqgcamsyhlym2vgbjtvhcrbxr4iowd4nv4zk5sehw4v665jad.onion:5222,y2ruswmdbsfl4hhwwiqz4m3sx6si5fr6l3pf62d4pms2b53wmagq3eqd.onion:5222

# This setting is ONLY for developer regtest setups,
# running multiple bots at once. Don't alter it otherwise
regtest_count = 0,0

## IRC SERVER 1: Darkscience IRC (Tor, IP)
################################################################################
[MESSAGING:server1]
# by default the legacy format without a `type` field is
# understood to be IRC, but you can, optionally, add it:
# type = irc
channel = joinmarket-pit
port = 6697
usessl = true

# For traditional IP:
#host = irc.darkscience.net
#socks5 = false

# For Tor (recommended as clearnet alternative):
host = darkirc6tqgpnwd3blln3yfv5ckl47eg7llfxkmtovrv7c7iwohhb6ad.onion
socks5 = true
socks5_host = localhost
socks5_port = 9050

## IRC SERVER 2: ILITA IRC (optional IRC alternate, Tor only)
################################################################################
[MESSAGING:server2]
channel = joinmarket-pit
port = 6667
usessl = false
socks5 = true
socks5_host = localhost

host = ilitafrzzgxymv6umx2ux7kbz3imyeko6cnqkvy4nisjjj4qpqkrptid.onion
socks5_port = 9050

## IRC SERVER 3: (backup) hackint IRC (Tor, IP)
################################################################################
#[MESSAGING:server3]
# channel = joinmarket-pit
# For traditional IP:
## host = irc.hackint.org
## port = 6697
## usessl = true
## socks5 = false
# For Tor (default):
#host = ncwkrwxpq2ikcngxq3dy2xctuheniggtqeibvgofixpzvrwpa77tozqd.onion
#port = 6667
#usessl = false
#socks5 = true
#socks5_host = localhost
#socks5_port = 9050

[LOGGING]
# Set the log level for the output to the terminal/console
# Possible choices: DEBUG / INFO / WARNING / ERROR
# Log level for the files in the logs-folder will always be DEBUG
console_log_level = INFO

# Use color-coded log messages to help distinguish log levels?:
color = true

[TIMEOUT]
maker_timeout_sec = 60
unconfirm_timeout_sec = 180
confirm_timeout_hours = 6

[POLICY]
# Use segwit style wallets and transactions
# Only set to false for old wallets, Joinmarket is now segwit only.
segwit = true

# Use native segwit (bech32) wallet. If set to false, p2sh-p2wkh
# will be used when generating the addresses for this wallet.
# Notes: 1. The default joinmarket pit is native segwit.
#        2. You cannot change the type of a pre-existing wallet.
native = true

# for dust sweeping, try merge_algorithm = gradual
# for more rapid dust sweeping, try merge_algorithm = greedy
# for most rapid dust sweeping, try merge_algorithm = greediest
# but don't forget to bump your miner fees!
merge_algorithm = default

# The fee estimate is based on a projection of how many sats/kilo-vbyte
# are needed to get in one of the next N blocks. N is set here as
# the value of 'tx_fees'. This cost estimate is high if you set
# N=1, so we choose 3 for a more reasonable figure, as our default.
# You can also set your own fee/kilo-vbyte: any number higher than 1 thousand
# will be interpreted as the fee in sats/kilo-vbyte that you wish to use.
#
# Example: N=30000 will use 30 thousand sats/kilo-vbyte (30 sats/vB) as a fee,
# while N=5 will use the 5 block estimate from your selected blockchain source.
tx_fees = 3

# Transaction fee rate variance factor, 0.2 means 20% variation around
# any manually chosen values, so if you set tx_fees=10000 and
# tx_fees_factor=0.2, it might use any value between 8 thousand and 12 thousand
# for your transactions.
tx_fees_factor = 0.2

# For users getting transaction fee estimates over an API,
# place a sanity check limit on the sats/kilo-vbyte to be paid.
# This limit is also applied to users using Core, even though
# Core has its own sanity check limit, which is currently
# 1 million satoshis.
#
# Example: N=350000 will use 350 thousand sats/kilo-vbyte (350 sats/vB) as a
# maximum fee.
absurd_fee_per_kb = 350000

# In decimal, the maximum allowable change either lower or
# higher, that the fee rate used for coinjoin sweeps is
# allowed to be.
# (note: coinjoin sweeps *must estimate* fee rates;
# they cannot be exact due to the lack of change output.)
#
# Example: max_sweep_fee_change = 0.4, with tx_fees = 10000,
# means actual fee rate achieved in the sweep can be as low
# as 6 thousand sats/kilo-vbyte up to 14 thousand sats/kilo-vbyte.
#
# If this is not achieved, the transaction is aborted. For tumbler,
# it will then be retried until successful.
# WARNING: too-strict setting may result in using up a lot
# of PoDLE commitments, hence the default 0.8 (80%).
max_sweep_fee_change = 0.8

# Maximum absolute coinjoin fee in satoshi to pay to a single
# market maker for a transaction. Both the limits given in
# max_cj_fee_abs and max_cj_fee_rel must be exceeded in order
# to not consider a certain offer.
max_cj_fee_abs = 100000

# Maximum relative coinjoin fee, in fractions of the coinjoin value
# e.g. if your coinjoin amount is 2 btc (200 million satoshi) and
# max_cj_fee_rel = 0.001 (0.1%), the maximum fee allowed would
# be 0.002 btc (200 thousand satoshi)
max_cj_fee_rel = 0.002

# The range of confirmations passed to the `listunspent` bitcoind RPC call
# 1st value is the inclusive minimum, defaults to one confirmation
# 2nd value is the exclusive maximum, defaults to most-positive-bignum (Google Me!)
# leaving it unset or empty defers to bitcoind's default values, ie [1, 9999999]
#listunspent_args = []
# That's what you should do, unless you have a specific reason, eg:
#  !!! WARNING !!! CONFIGURING THIS WHILE TAKING LIQUIDITY FROM
#  !!! WARNING !!! THE PUBLIC ORDERBOOK LEAKS YOUR INPUT MERGES
#  spend from unconfirmed transactions:  listunspent_args = [0]
# display only unconfirmed transactions: listunspent_args = [0, 1]
# defend against small reorganizations:  listunspent_args = [3]
#   who is at risk of reorganization?:   listunspent_args = [0, 2]
# NB: using 0 for the 1st value with scripts other than wallet-tool could cause
# spends from unconfirmed inputs, which may then get malleated or double-spent!
# other counterparties are likely to reject unconfirmed inputs... don't do it.

# tx_broadcast: options: self, random-peer, not-self.
#
# self = broadcast transaction with your own bitcoin node.
#
# random-peer = everyone who took part in the coinjoin has a chance of broadcasting
# Note: if your counterparties do not support it, you will fall back
# to broadcasting via your own node.
#
# not-self = never broadcast with your own bitcoin node.
#
# Note: in this case if your counterparties do not broadcast for you, you
# will have to broadcast the tx manually (you can take the tx hex from the log
# or terminal) via some other channel. It is not recommended to choose this
# option when running schedules/tumbler.
tx_broadcast = random-peer

# If makers do not respond while creating a coinjoin transaction,
# the non-responding ones will be ignored. This is the minimum
# amount of makers which we are content with for the coinjoin to
# succeed. Less makers means that the whole process will restart
# after a timeout.
minimum_makers = 4

# Threshold number of satoshis below which an incoming utxo
# to a reused address in the wallet will be AUTOMATICALLY frozen.
# This avoids forced address reuse attacks; see:
# https://en.bitcoin.it/wiki/Privacy#Forced_address_reuse
#
# The default is to ALWAYS freeze a utxo to an already used address,
# whatever the value of it, and this is set with the value -1.
max_sats_freeze_reuse = -1

# Interest rate used when calculating the value of fidelity bonds created
# by locking bitcoins in timelocked addresses
# See also:
# https://gist.github.com/chris-belcher/87ebbcbb639686057a389acb9ab3e25b#determining-interest-rate-r
# Set as a real number, i.e. 1 = 100% and 0.01 = 1%
interest_rate = 0.015

# Some makers run their bots to mix their funds not just to earn money
# So to improve privacy very slightly takers dont always choose a maker based
# on his fidelity bond but allow a certain small percentage to be chosen completely
# randomly without taking into account fidelity bonds
# This parameter sets how many makers on average will be chosen regardless of bonds
# A real number, i.e. 1 = 100%, 0.125 = 1/8 = 1 in every 8 makers on average will be bondless
bondless_makers_allowance = 0.125

# To (strongly) disincentivize Sybil behaviour, the value assessment of the bond
# is based on the (time value of the bond)^x where x is the bond_value_exponent here,
# where x > 1. It is a real number (so written as a decimal).
bond_value_exponent = 1.3

##############################
# THE FOLLOWING SETTINGS ARE REQUIRED TO DEFEND AGAINST SNOOPERS.
# DON'T ALTER THEM UNLESS YOU UNDERSTAND THE IMPLICATIONS.
##############################

# Number of retries allowed for a specific utxo, to prevent DOS/snooping.
# Lower settings make snooping more expensive, but also prevent honest users
# from retrying if an error occurs.
taker_utxo_retries = 3

# Number of confirmations required for the commitment utxo mentioned above.
# this effectively rate-limits a snooper.
taker_utxo_age = 5

# Percentage of coinjoin amount that the commitment utxo must have
# as a minimum BTC amount. Thus 20 means a 1BTC coinjoin requires the
# utxo to be at least 0.2 btc.
taker_utxo_amtpercent = 20

# Set to 1 to accept broadcast PoDLE commitments from other bots, and
# add them to your blacklist (only relevant for Makers).
# There is no way to spoof these values, so the only "risk" is that
# someone fills your blacklist file with a lot of data.
accept_commitment_broadcasts = 1

# Location of your commitments.json file (stores commitments you've used
# and those you want to use in future), relative to the scripts directory.
commit_file_location = cmtdata/commitments.json

# Location of the file used by makers to keep track of used/blacklisted
# commitments. For remote daemon, set to `.` to have it stored locally
# (but note that *all* bots using the same code installation share it,
# in this case, which can be bad in testing).
commitment_list_location = cmtdata/commitmentlist

##############################
# END OF ANTI-SNOOPING SETTINGS
##############################

[PAYJOIN]
# For the majority of situations, the defaults
# need not be altered - they will ensure you don't pay
# a significantly higher fee.
# MODIFICATION OF THESE SETTINGS IS DISADVISED.

# Payjoin protocol version; currently only '1' is supported.
payjoin_version = 1

# Servers can change their destination address by default (0).
# if '1', they cannot. Note that servers can explicitly request
# that this is activated, in which case we respect that choice.
disable_output_substitution = 0

# "default" here indicates that we will allow the receiver to
# increase the fee we pay by:
# 1.2 * (our_fee_rate_per_vbyte * vsize_of_our_input_type)
# (see https://github.com/bitcoin/bips/blob/master/bip-0078.mediawiki#span_idfeeoutputspanFee_output)
# (and 1.2 to give breathing room)
# which indicates we are allowing roughly one extra input's fee.
# If it is instead set to an integer, then that many satoshis are allowed.
# Additionally, note that we will also set the parameter additionafeeoutputindex
# to that of our change output, unless there is none in which case this is disabled.
max_additional_fee_contribution = default

# This is the minimum sats/vbyte we allow in the payjoin
# transaction; note it is decimal, not integer.
min_fee_rate = 1.1

# For payjoins as sender (i.e. client) to hidden service endpoints,
# the socks5 configuration:
onion_socks5_host = localhost
onion_socks5_port = 9050

# For payjoin onion service creation:
# the tor control configuration:
tor_control_host = localhost

# or, to use a UNIX socket
# control_host = unix:/var/run/tor/control
# note: port needs to be provided (but is ignored for UNIX socket)
tor_control_port = 9051

# the host/port actually serving the hidden service
# (note the *virtual port*, that the client uses,
# is hardcoded to 80):
onion_serving_host = 127.0.0.1
onion_serving_port = 8080

# in some exceptional case the HS may be SSL configured,
# this feature is not yet implemented in code, but here for the
# future:
hidden_service_ssl = false

[YIELDGENERATOR]
# [string, 'reloffer' or 'absoffer'], which fee type to actually use
ordertype = reloffer

# [satoshis, any integer] / absolute offer fee you wish to receive for coinjoins (cj)
cjfee_a = 500

# [fraction, any str between 0-1] / relative offer fee you wish to receive based on a cj's amount
cjfee_r = 0.00002

# [fraction, 0-1] / variance around the average fee. Ex: 200 fee, 0.2 var = fee is btw 160-240
cjfee_factor = 0.1

# [satoshis, any integer] / the average transaction fee you're adding to coinjoin transactions
# (note: this will soon be deprecated; leave at zero)
txfee_contribution = 0

# [fraction, 0-1] / variance around the average fee. Ex: 1000 fee, 0.2 var = fee is btw 800-1200
txfee_contribution_factor = 0.3

# [satoshis, any integer] / minimum size of your cj offer. Lower cj amounts will be disregarded
minsize = 100000

# [fraction, 0-1] / variance around all offer sizes. Ex: 500k minsize, 0.1 var = 450k-550k
size_factor = 0.1

gaplimit = 6

[SNICKER]
# Any other value than 'true' will be treated as False,
# and no SNICKER actions will be enabled in that case:
enabled = false

# In satoshis, we require any SNICKER to pay us at least
# this much (can be negative), otherwise we will refuse
# to sign it:
lowest_net_gain = 0

# Comma separated list of servers (if port is omitted as :port, it
# is assumed to be 80) which we will poll against (all, in sequence); note
# that they are allowed to be *.onion or cleartext servers, and no
# scheme (http(s) etc) needs to be added to the start.
servers = cn5lfwvrswicuxn3gjsxoved6l2gu5hdvwy5l3ev7kg6j7lbji2k7hqd.onion,

# How many minutes between each polling event to each server above:
polling_interval_minutes = 60
kristapsk commented 1 year ago

2022-10-13 17:59:49,038 [WARNING] Invalid address "tb1qc7h7ge6hk3ld4yeg3d8kcduk2dfq2s88f2s4wh342jeaq8txvyfq4f7w5x" 2022-10-13 17:59:49,038 [WARNING] Invalid address "tb1qnn9ga8zgm3gc4mu2lclpnyd6e9gr3hes5k54w2rxh2m8jgpjhgwq7rfvr2" 2022-10-13 17:59:49,038 [WARNING] Invalid address "tb1qwa4t32l0hy0c667k342ljt7w2578yzk0h9gspwfvwaj2ecx7annqj5em6m" 2022-10-13 17:59:49,039 [WARNING] Invalid address "tb1q0ep8xvtaugkcr2nckwflmlmfv6gls8z9zzn4f89cxl25362jx5usg24esg" 2022-10-13 17:59:49,039 [WARNING] Invalid address "tb1q4fha7wyeesa3v5vc35dunnfae7ldkef5ztxs0jjarlkzzkr7x62swxl9yh" 2022-10-13 17:59:49,039 [WARNING] Invalid address "tb1qzuceejghvlq7754knjrgmt9y4wp4srjpjz9twpy58n0pmmyva93qupc5lk" 2022-10-13 17:59:49,039 [WARNING] Invalid address "tb1qztqtzezgcp9ssl6hpuf34kvu5zv8ha2eqlac0jm9fjcdau7qc84sg5qrk9" 2022-10-13 17:59:49,039 [WARNING] Invalid address "tb1qlxl22jmhs94ntnnn86zc4x8mmzashcep863whkayf40n344egf9spas0lf" 2022-10-13 17:59:49,039 [WARNING] Invalid address "tb1qszy62emyp54rqrr7zrmq3ehl5y3p9p4dt8jc77kpujl3apu7ptcsuvgz2z" Fatal sync error: import of 1020 address(es) failed for some reason. To prevent coin or privacy loss, Joinmarket will not load a wallet in this conflicted state. Try using a new Bitcoin Core wallet to sync this Joinmarket wallet, or use a new Joinmarket wallet.

These are signet / testnet not regtest addresses, that's the source of the problem.

This seems strange, as wallet creation via RPC API is covered by test suite and it runs on regtest...

Fonta1n3 commented 1 year ago

Ok let me try and reproduce it.

Fonta1n3 commented 1 year ago

@kristapsk I am able to reproduce this. I don't understand as my node is definitely on regtest, JM is configured for regtest, the wallet I created for JM to use is also on regtest. I was using signet prior to attempting to create this regtest wallet.

This is the log output from JM before it shows all the rejected signet/testnet addresses:

(jmvenv) standup@demo:~/joinmarket-clientserver/scripts$ python jmwalletd.py
User data location: /home/standup/. joinmarket/
2022-10-14 15:43:00,238 [INFO] Starting jmwalletd on port: 28183
Enter PEM pass phrase:
2022-10-14 15:43:04,049 [INFO] Joinmarket daemon listening on port 27183
‹Request at 0x7f95c54fafbo method=POST uri=/api/v1/wallet/create clientproto=HTTP/1.1>
b' POST b'/api/v1/wallet/create {7 /api/vl/wallet/create < io.BytesIO object at 0x7£95c5509080> [ (b 'Host', [b'fkvfwqj6ckwx3sxk3g7wqvcu2pnvi2d5pteo2ev5gl6ktrhkh72iduad.onion:28183']), (b'Content-Type', [b'application/json'1),
('Connection', Ib'keep-alive•1). (b'Accept', [b'*/*']), (b'User-Agent', S, en; q=0.9'1), (b'Accept-Encoding' [b'FullyNoded/355 CFNetwork/1331.0.7 Darwin/21.4.0'] , (b'Content-Length', [b'123' ]), (b'Accept-Language', [b'en-u, Ib'gzip, deflate, br'])]
2022-10-14 15:43:29,127 [ERROR] Critical error updating blockheight.
2022-10-14 15:43:29,178 [ERROR] Failure of RPC connection to Bitcoin Core in wallet service startup. Application cannot continue, shutting down.
kristapsk commented 1 year ago

2022-10-14 15:43:29,127 [ERROR] Critical error updating blockheight.

By looking at code and your log output, it seems that getblockcount Bitcoin RPC returns 0.

Will try to reproduce later.

Fonta1n3 commented 1 year ago

2022-10-14 15:43:29,127 [ERROR] Critical error updating blockheight.

By looking at code and your log output, it seems that getblockcount Bitcoin RPC returns 0.

Will try to reproduce later.

Ah I haven't mined any blocks yet... That's probably why. Will try again.

Fonta1n3 commented 1 year ago

I mined more then 100 blocks, wallet creation now succeeds but the main issue of incorrect address format persists :

(jmvenv) standup@demo:~/joinmarket-clientserver/scripts$ python jmwalletd.py
User data location: /home/standup/.joinmarket/
2022-10-14 17:15:53,330 [INFO]  Starting jmwalletd on port: 28183
Enter PEM pass phrase:
2022-10-14 17:15:56,478 [INFO]  Joinmarket daemon listening on port 27183
<Request at 0x7fcdf0a0ab90 method=POST uri=/api/v1/wallet/create clientproto=HTTP/1.1>
b'POST'
b'/api/v1/wallet/create'
{}
b'/api/v1/wallet/create'
<_io.BytesIO object at 0x7fcdf0a17970>
[(b'Host', [b'xxx.onion:28183']), (b'Content-Type', [b'application/json']), (b'Connection', [b'keep-alive']), (b'Accept', [b'*/*']), (b'User-Agent', [b'FullyNoded/355 CFNetwork/1331.0.7 Darwin/21.4.0']), (b'Content-Length', [b'125']), (b'Accept-Language', [b'en-US,en;q=0.9']), (b'Accept-Encoding', [b'gzip, deflate, br'])]
2022-10-14 17:17:03,971 [WARNING]  Connection had broken pipe, attempting reconnect.
2022-10-14 17:17:03,984 [INFO]  Detected new wallet, performing initial import
2022-10-14 17:17:10,018 [WARNING]  Invalid address "tb1qz5du5estdy335ghvkgeps47tq75c4224y4y9mr4plf55rmrr5hkq9c9q50"
2022-10-14 17:17:10,018 [WARNING]  Invalid address "tb1q2ecza9aumzlzrenl3m9tw4j9d0hqjrta8u3rk3c28f2juw83zw6sxyul8c"
2022-10-14 17:17:10,018 [WARNING]  Invalid address "tb1qj9rsmja58fpuffeh4qazqg7w0murgu757p4gdq44cuhxyh536w9srkkwsa"
....(this goes on for awhile, have deleted for brevity)...
2022-10-14 17:17:10,169 [WARNING]  Invalid address "tb1qszqe36slyad3hc5gls9937fgyhyu79q6h5m47drmn5mvx9z4am2q7mepgu"
2022-10-14 17:17:10,170 [WARNING]  Invalid address "tb1qs00tu37ue5h3nsdqk465u2a3rt2tzpyd9xn9xllmus47k5sudvgq00382g"
Fatal sync error: import of 1020 address(es) failed for some reason. To prevent coin or privacy loss, Joinmarket will not load a wallet in this conflicted state. Try using a new Bitcoin Core wallet to sync this Joinmarket wallet, or use a new Joinmarket wallet.
<Request at 0x7fcdf0a0a020 method=POST uri=/api/v1/wallet/FullyNoded-b3fkeJrP6C.jmdat/unlock clientproto=HTTP/1.1>
b'POST'
b'/api/v1/wallet/FullyNoded-b3fkeJrP6C.jmdat/unlock'
{}
b'/api/v1/wallet/FullyNoded-b3fkeJrP6C.jmdat/unlock'
<_io.BytesIO object at 0x7fcdf15609f0>
[(b'Host', [b'xxx.onion:28183']), (b'Content-Type', [b'text/plain']), (b'Connection', [b'keep-alive']), (b'Accept', [b'*/*']), (b'User-Agent', [b'FullyNoded/355 CFNetwork/1331.0.7 Darwin/21.4.0']), (b'Content-Length', [b'61']), (b'Accept-Language', [b'en-US,en;q=0.9']), (b'Accept-Encoding', [b'gzip, deflate, br'])]
<Request at 0x7fcdf08dba30 method=GET uri=/api/v1/wallet/FullyNoded-b3fkeJrP6C.jmdat/display clientproto=HTTP/1.1>
b'GET'
b'/api/v1/wallet/FullyNoded-b3fkeJrP6C.jmdat/display'
{}
b'/api/v1/wallet/FullyNoded-b3fkeJrP6C.jmdat/display'
<_io.BytesIO object at 0x7fcdf15609f0>
[(b'Host', [b'xxx.onion:28183']), (b'Content-Type', [b'text/plain']), (b'Connection', [b'keep-alive']), (b'Accept', [b'*/*']), (b'User-Agent', [b'FullyNoded/355 CFNetwork/1331.0.7 Darwin/21.4.0']), (b'Authorization', [b'Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ3YWxsZXQiOiJGdWxseU5vZGVkLWIzZmtlSnJQNkMuam1kYXQiLCJleHAiOjE2NjU3Njk5MTl9.GlIA-bm4fioNxzUHRMMjaQNpYUxEYgEvZocabKl-6To']), (b'Accept-Language', [b'en-US,en;q=0.9']), (b'Accept-Encoding', [b'gzip, deflate, br'])]
2022-10-14 17:22:03,894 [WARNING]  Connection had broken pipe, attempting reconnect.
2022-10-14 17:22:03,896 [WARNING]  Invalid address "tb1qf776xfcta4eefzlamp0hyuetw2w8v0h8j8za83"
2022-10-14 17:22:03,897 [WARNING]  Invalid address "tb1qzef3tc4shjwlww8vlmvlpnqgx4sdm09m89dsx8"
2022-10-14 17:22:03,897 [WARNING]  Invalid address "tb1qmp3vzcqegqucek26g62qyc7qt7wjffz89szpqh"
2022-10-14 17:22:03,897 [WARNING]  Invalid address "tb1qy35u5psse9tdvwxle8xejrrxgp6hkrjytkrq3w"
2022-10-14 17:22:03,897 [WARNING]  Invalid address "tb1qyrm7a2qe4pyg69dr0xpw2p5h3rpyx5yd564uqx"
2022-10-14 17:22:03,898 [WARNING]  Invalid address "tb1qrspt0zd5mzhlt702egecwjy2a95s423gcsapn3"
Fatal sync error: import of 6 address(es) failed for some reason. To prevent coin or privacy loss, Joinmarket will not load a wallet in this conflicted state. Try using a new Bitcoin Core wallet to sync this Joinmarket wallet, or use a new Joinmarket wallet.

Traceback (most recent call last):
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/web/server.py", line 292, in render
    body = resrc.render(self)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_resource.py", line 204, in render
    d = defer.maybeDeferred(_execute)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/defer.py", line 190, in maybeDeferred
    result = f(*args, **kwargs)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_resource.py", line 196, in _execute
    d = defer.maybeDeferred(
--- <exception caught here> ---
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/defer.py", line 190, in maybeDeferred
    result = f(*args, **kwargs)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_app.py", line 129, in execute_endpoint
    return endpoint_f(self._instance, *args, **kwargs)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_app.py", line 241, in _f
    return _call(instance, f, request, *a, **kw)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_app.py", line 60, in _call
    result = __klein_f__(*args, **kwargs)
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_rpc.py", line 576, in displaywallet
    walletinfo = wallet_display(self.services["wallet"], False, jsonified=True)
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_utils.py", line 579, in wallet_display
    wallet_service.bci.import_addresses(gap_addrs,
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/blockchaininterface.py", line 290, in import_addresses
    sys.exit(EXIT_FAILURE)
builtins.SystemExit: 1

2022-10-14 17:22:08,494 [WARNING]  Invalid address "tb1qrefxxlu22p63tgh55utp9ufca5866vy789dcf8"
Fatal sync error: import of 1 address(es) failed for some reason. To prevent coin or privacy loss, Joinmarket will not load a wallet in this conflicted state. Try using a new Bitcoin Core wallet to sync this Joinmarket wallet, or use a new Joinmarket wallet.

Traceback (most recent call last):
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/web/server.py", line 292, in render
    body = resrc.render(self)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_resource.py", line 204, in render
    d = defer.maybeDeferred(_execute)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/defer.py", line 190, in maybeDeferred
    result = f(*args, **kwargs)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_resource.py", line 196, in _execute
    d = defer.maybeDeferred(
--- <exception caught here> ---
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/twisted/internet/defer.py", line 190, in maybeDeferred
    result = f(*args, **kwargs)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_app.py", line 129, in execute_endpoint
    return endpoint_f(self._instance, *args, **kwargs)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_app.py", line 241, in _f
    return _call(instance, f, request, *a, **kw)
  File "/home/standup/joinmarket-clientserver/jmvenv/lib/python3.10/site-packages/klein/_app.py", line 60, in _call
    result = __klein_f__(*args, **kwargs)
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_rpc.py", line 960, in getaddress
    address = self.services["wallet"].get_external_addr(mixdepth)
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_service.py", line 996, in get_external_addr
    self.import_addr(addr)
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/wallet_service.py", line 938, in import_addr
    self.bci.import_addresses([addr], self.wallet.get_wallet_name())
  File "/home/standup/joinmarket-clientserver/jmclient/jmclient/blockchaininterface.py", line 290, in import_addresses
    sys.exit(EXIT_FAILURE)
builtins.SystemExit: 1
Fonta1n3 commented 1 year ago

fwiw i've attempted to start from scratch and still get the same error... to reproduce:

options: signet, testnet, mainnet

Note: for regtest, use network = testnet

network = testnet

rpc_host = localhost

default ports are 8332 for mainnet, 18443 for regtest, 18332 for testnet, 38332 for signet

rpc_port = 18443

Use either rpc_user / rpc_password pair or rpc_cookie_file.

rpc_user = bitcoinrpc rpc_password = password

rpc_cookie_file =

rpc_wallet_file is Bitcoin Core wallet which is used for address and

transaction monitoring (it is watchonly, no private keys are stored there).

It must be created manually if does not exist, see docs/USAGE.md for more

information.

rpc_wallet_file = jm-test-wallet


- Setup a self signed cert:
```cd .joinmarket
mkdir ssl
cd ssl
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 365
rnandan273 commented 1 year ago

Fatal Sync error: Joinmarket will not load wallet in this conflicted state. Create a new Joinmarket wallet or a bitcoin core wallet I have tried creating a new joinmarket wallet too, but still the same error. Any suggestions

theborakompanioni commented 1 year ago 
# options: bitcoin-rpc, regtest, bitcoin-rpc-no-history, no-blockchain
# When using bitcoin-rpc-no-history remember to increase the gap limit to scan for more addresses, try -g 5000
# Use 'no-blockchain' to run the ob-watcher.py script in scripts/obwatch without current access
# to Bitcoin Core; note that use of this option for any other purpose is currently unsupported.
blockchain_source = bitcoin-rpc

Try setting blockchain_source to regtest. This is what is working for the Jam regtest env.

rnandan273 commented 1 year ago

Thanks a lot, this worked. So now i hope to use all the interface api's

Fonta1n3 commented 1 year ago

I didn't read the config comment, always stopped reading that part at bitcoin-rpc 😅 Thanks for clarifying!