Error when processing both Coinbase Pro and Coinbase data through REST API

[kiates]

When I process both Coinbase Pro and Coinbase transactions there is some kind of conflict or glitch that keeps the transactions from passing the resolver. Basically the same unique Id exists on both but one says the transaction came from Coinbase Pro and the other Coinbase.

2023-04-04 10:05:28,065/dali/ERROR: Fatal exception occurred: Traceback (most recent call last): File "C:\Users\cyates\gitwork\dali-rp2\src\dali\dali_main.py", line 169, in _dali_main_internal resolved_transactions: List[AbstractTransaction] = resolve_transactions(transactions, dali_configuration, args.read_spot_price_from_web) File "C:\Users\cyates\gitwork\dali-rp2\src\dali\transaction_resolver.py", line 291, in resolve_transactions transaction = _resolve_intra_intra_transaction(transaction1, transaction2, None) File "C:\Users\cyates\gitwork\dali-rp2\src\dali\transaction_resolver.py", line 488, in _resolve_intra_intra_transaction from_exchange: str = _resolve_fields( File "C:\Users\cyates\gitwork\dali-rp2\src\dali\transaction_resolver.py", line 71, in _resolve_fields raise RP2RuntimeError(f"Internal error: {name1} and {name2} have different values: {transaction1}\n{transaction2}") rp2.rp2_error.RP2RuntimeError: Internal error: transaction1.from_exchange and transaction2.from_exchange have different values: IntraTransaction: plugin=Coinbase Pro unique_id=3uVGBNLZ8EEJ2S1hVrpckbUBCjaPMw2AhZLcX2fk9Hypn16rkug536mVbq8MusqPBqHTemRnCavB88is4REi1CwC raw_data={"id": "10187068009", "amount": "-5.8100050000000000", "balance": "3.3459950000000000", "created_at": "2021-09-10T00:56:06.237481Z", "type": "transfer", "details": {"transfer_id": "85ed3e78-5a58-48ce-bd40-0091fe0f1cb9", "transfer_type": "withdraw"}}//{"id": "85ed3e78-5a58-48ce-bd40-0091fe0f1cb9", "type": "withdraw", "created_at": "2021-09-10 00:55:33.687247+00", "completed_at": "2021-09-10 00:56:06.237481+00", "account_id": "d9a96c37-092f-4176-920f-9eee256e6086", "user_id": "611493b2684edc02e2f974fa", "amount": "5.81000500", "details": {"fee": "0.000005", "subtotal": "5.81", "sent_to_address": "8oTytat5naEMboQohgkUvZcTzReeLFn4K1uSofdQ7rkB", "coinbase_account_id": "22258e81-b898-5091-ba08-4f55825e08ac", "crypto_transaction_hash": "3uVGBNLZ8EEJ2S1hVrpckbUBCjaPMw2AhZLcX2fk9Hypn16rkug536mVbq8MusqPBqHTemRnCavB88is4REi1CwC", "coinbase_payment_method_id": ""}, "canceled_at": null, "processed_at": "2021-09-10 00:56:06.241682+00", "user_nonce": "1631235137318", "idem": null, "profile_id": "22d73a0f-c463-4775-a850-865b79a88b59", "currency": "SOL"} timestamp=2021-09-10 00:56:06.237481+0000 asset=SOL from_exchange=Coinbase Pro from_holder=Chad Yates to_exchange=unknown to_holder=__unknown spot_price=None crypto_sent=5.8100050000000000 crypto_received=unknown notes=None IntraTransaction: plugin=Coinbase unique_id=3uVGBNLZ8EEJ2S1hVrpckbUBCjaPMw2AhZLcX2fk9Hypn16rkug536mVbq8MusqPBqHTemRnCavB88is4REi1CwC raw_data={"id": "3d5b37d7-be44-5a44-8183-b70856322ccc", "type": "send", "status": "completed", "amount": {"amount": "-5.810005000", "currency": "SOL"}, "native_amount": {"amount": "-1099.01", "currency": "USD"}, "description": null, "created_at": "2021-09-10T00:55:36Z", "updated_at": "2023-03-31T20:08:06Z", "resource": "transaction", "resource_path": "/v2/accounts/22258e81-b898-5091-ba08-4f55825e08ac/transactions/3d5b37d7-be44-5a44-8183-b70856322ccc", "instant_exchange": false, "network": {"status": "unconfirmed", "status_description": "Pending (less than a minute)", "hash": "3uVGBNLZ8EEJ2S1hVrpckbUBCjaPMw2AhZLcX2fk9Hypn16rkug536mVbq8MusqPBqHTemRnCavB88is4REi1CwC", "transaction_url": "https://explorer.solana.com/tx/3uVGBNLZ8EEJ2S1hVrpckbUBCjaPMw2AhZLcX2fk9Hypn16rkug536mVbq8MusqPBqHTemRnCavB88is4REi1CwC", "transaction_fee": {"amount": "0.000005000", "currency": "SOL"}, "transaction_amount": {"amount": "5.810000000", "currency": "SOL"}, "confirmations": 0}, "to": {"resource": "solana_address", "address": "8oTytat5naEMboQohgkUvZcTzReeLFn4K1uSofdQ7rkB", "currency": "SOL", "address_info": {"address": "8oTytat5naEMboQohgkUvZcTzReeLFn4K1uSofdQ7rkB"}, "address_url": "https://explorer.solana.com/address/8oTytat5naEMboQohgkUvZcTzReeLFn4K1uSofdQ7rkB"}, "idem": "85ed3e78-5a58-48ce-bd40-0091fe0f1cb9send", "application": {"id": "5756ab6e-836b-553b-8950-5e389451225d", "resource": "application", "resource_path": "/v2/applications/5756ab6e-836b-553b-8950-5e389451225d"}, "details": {"title": "Sent Solana", "subtitle": "To Solana address", "header": "Sent 5.810005 SOL ($1,099.01)", "health": "positive"}, "hide_native_amount": false} timestamp=2021-09-10 00:55:36+0000 asset=SOL from_exchange=Coinbase from_holder=Chad Yates to_exchange=unknown to_holder=__unknown spot_price=189.1581848896859813373654584 crypto_sent=5.810005000 crypto_received=unknown notes=None

Running each separately to look at the result files, here is the transaction from the Coinbase Pro ods:

... and here is the transaction from the Coinbase ods:

The interesting part is there is a transaction right before that shows a resolved Coinbase Pro to Coinbase transaction with a different unique ID.

I'm not sure what to make of this yet.

[eprbell]

Thanks for reporting issues! I have already seen similar cases of transfer transactions that are duplicated in Coinbase and Coinbase Pro: see these examples. The way I resolved these issues was to ignore the Coinbase Pro transaction and just process the Coinbase one (when I recognized it as a duplicate).

The tricky part is finding the pattern in these transactions that makes them recognizable as duplicates: that's what the if statements in the examples above do. We have to do the same thing for this transaction and add a new if statement: one possible candidate for the pattern is that your CBPro transaction doesn't have details/tx_service_transaction_id.

Here's what I mean:

diff --git a/src/dali/plugin/input/rest/coinbase_pro.py b/src/dali/plugin/input/rest/coinbase_pro.py
index 4f37e3d..2bb79cc 100644
--- a/src/dali/plugin/input/rest/coinbase_pro.py
+++ b/src/dali/plugin/input/rest/coinbase_pro.py
@@ -70,6 +70,7 @@ _TRADE_ID: str = "trade_id"
 _TRANSFER: str = "transfer"
 _TRANSFER_ID: str = "transfer_id"
 _TRANSFER_TYPE: str = "transfer_type"
+_TX_SERVICE_TRANSACTION_ID: str = "tx_service_transaction_id"
 _TYPE: str = "type"
 _USD_VOLUME: str = "usd_volume"
 _WITHDRAW: str = "withdraw"
@@ -239,6 +240,10 @@ class InputPlugin(AbstractInputPlugin):
             self.__logger.debug("Transfer is a Coinbase transaction already captured by Coinbase plugin: ignoring.")
             return

+        if _TX_SERVICE_TRANSACTION_ID not in transfer_details:
+            self.__logger.debug("Transfer is a Coinbase transaction already captured by Coinbase plugin: ignoring.")
+            return
+
         crypto_hash = transfer_details[_CRYPTO_TRANSACTION_HASH]

         if transaction_details[_TRANSFER_TYPE] == _DEPOSIT:

Can you try if this solves the problem? I'll also test this with my own data.

[kiates]

That change definitely makes the resolver happy.

[eprbell]

My own tests also passed. Question: what kind of transaction is it: does it have any peculiar features or is it just a normal transfer to an external address?

[kiates]

Here is how it looks int the Coinbase interface...

I know at some point Coinbase, Inc. transferred all my assets from Coinbase Pro to Coinbase as they were consolidating their brand. I expect that is the special part of this pattern. I'll see if I can access the Coinbase Pro side still.

[kiates]

Here is what I could find on the Coinbase Pro side...

[eprbell]

This transaction doesn't look like a CBPro to CB transfer: it has the signature of a transfer to an external address and I was just curious if you remembered anything that stood out about it that might explain the slight difference compared to normal transfers (the missing details/tx_service_transaction_id). If not, it's no big deal: I think it's ok to merge the fix.