account_tx comes back blank

RichardAH commented 1 month ago

Issue Description

Steps to Reproduce

Expected Result

Actual Result

Environment

Supporting Files

vasil-nesterov commented 4 weeks ago

@RichardAH When I run the same account_tx command 20 times against https://xahau.network, from time to time I get a response with 0 txns and no marker. Is this the same issue as yours, or would it better for me to open a separate issue?

(account: rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C, ledger_index_min: -1, ledger_index_max: -1, no marker)

Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509089
 Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509063
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509096
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509097
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509097
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509098
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509098
Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509098
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509099
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509099
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509100
Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509100
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509100
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509101
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509101
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509102
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509102
Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509103
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509103
Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509103

Script

require 'http'
require 'json'
require 'securerandom'

account = 'rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C'

20.times do |i|
  query = {
    'jsonrpc': '2.0',
    id: SecureRandom.uuid,
    method: 'account_tx',
    params: [
      {
        ledger_index_min: -1,
        ledger_index_max: -1,
        account: account
      }.compact
    ]
  }

  response =
    HTTP
      .headers('Content-Type' => 'application/json')
      .post('https://xahau.network', body: query.to_json)
      .then { JSON.parse(_1.to_s) }

  if response.dig('result', 'status') != 'success'
    raise "Request failed"
  end

  marker, txns, ledger_index_min, ledger_index_max =
    response['result'].values_at(
      'marker', 'transactions', 'ledger_index_min', 'ledger_index_max'
    )

  puts "Txns: #{txns.count}, Marker: #{marker}, ledger_index_min: #{ledger_index_min}, ledger_index_max: #{ledger_index_max}"
end

dangell7 commented 4 weeks ago

Are you sure you're not being rate limited?

vasil-nesterov commented 4 weeks ago

@dangell7

When I encounter rate limit, the response code is 429 and the body is Rate limited.

This is a different story.

Here's the output with bodies of "0 txs" responses:

0: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509532
1: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509532
2: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
3: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
4: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
5: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509533
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509533,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
6: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509534
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509534,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
7: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509534
8: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509534
9: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509535
10: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509535
11: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
12: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
13: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
14: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
15: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509537
16: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509537
17: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509537
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509537,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
18: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509538
19: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509538
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509538,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}

vasil-nesterov commented 4 weeks ago

Workaround: retry each request until response.ledger_index_min == 1. This way I'm able to fetch all txs from a big account in a stable manner.

dangell7 commented 4 weeks ago

@dangell7

When I encounter rate limit, the response code is 429 and the body is Rate limited.

This is a different story.

Here's the output with bodies of "0 txs" responses:

0: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509532
1: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509532
2: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
3: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
4: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
5: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509533
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509533,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
6: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509534
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509534,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
7: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509534
8: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509534
9: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509535
10: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509535
11: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
12: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
13: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
14: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
15: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509537
16: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509537
17: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509537
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509537,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
18: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509538
19: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509538
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509538,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}

I see. I'm curious what you're trying to accomplish? Why are you increasing the ledger_max by 1? And in your last request you are looking for transactions between 9508962 - 9509538 and there are none... So its correct. If you want all the transactions for an account what you are doing isn't correct.

First you should query the account_tx and leave out ledger min/max with a limit of 1000. Then loop though the markers. (it will auto fill ledger min/max). Store ledger_max response to use later
Later when you want to re-query you use the ledger_max response as the ledger_min so you're not re querying the information.

vasil-nesterov commented 3 weeks ago

@dangell7

I'm curious what you're trying to accomplish?

I'm showing that the API is unstable.

In my example, I'm repeating the same request with exactly the same params 20 times. The request params are: account: pEZRGPj3miukn48yfQGePQFjqVhHJLm1C, ledger_index_min: -1, ledger_index_max: -1, no marker.

Some responses are ok (those that have a marker and txs.count == 200), others are broken. The broken ones have empty txs list, no marker and a very high ledger_index_min value, indicating that the server didn't search the entire ledger as requested.

tequdev commented 3 weeks ago

It appears that some nodes in the Xahaud cluster are holding only the most recent ledger instead of the full history node. (or in backfill).

This is the result of running server_info rpc multiple times and getting complete_ledgers.

complete_ledgers 9548898-9549739
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 9548898-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740

RichardAH commented 2 weeks ago

I'll leave this issue open and rename it, but the original issued was fixed by https://github.com/Xahau/xahaud/pull/378/commits/8b0d42785cc1f4fd3f2b2582d8db4f38ef93b31d I believe

Xahau / xahaud