search

ripple / rippled-historical-database

SQL database as a canonical source of historical data
99 stars 68 forks source link

Data API v2 exchanges call limits before reducing #153

Closed hammertoe closed 5 years ago

hammertoe commented 6 years ago

The Data API v2 has an 'exchanges' endpoint. According to the docs you should be able to specify both a limit and a reduce parameter.

https://ripple.com/build/data-api-v2/#get-exchanges

However it seems the limit is applied before reduce, see as an example:

https://data.ripple.com/v2/exchanges/XRP/BTC+rchGBxcD1A1C2tdxF6papQYZ8kjRKMYcL?start=2017-09-10T21:27:02Z&limit=20000&reduce=true

which returns:

{"result":"success","count":1,"marker":"BTC|rchGBxcD1A1C2tdxF6papQYZ8kjRKMYcL|XRP||20170911023701|000032673230|00018|00007","exchanges":[{"open":"0.000050957800","high":"0.000051097462","low":"0.000050040678","close":"0.000050198119","base_volume":"388720.6866489995","counter_volume":"19.650270749808836","buy_volume":146717.51915300012,"count":400,"open_time":"2017-09-10T21:33:52Z","close_time":"2017-09-11T02:37:01Z","vwap":"0.000050551132","start":"2017-09-17T22:52:34Z","base_currency":"XRP","counter_currency":"BTC","counter_issuer":"rchGBxcD1A1C2tdxF6papQYZ8kjRKMYcL"}]}

Running this at a current time of 2017-09-17T21:27 I'd expect to get an aggregate over the last 7 days. However it appears that the results are being constrained by the built-in limit before being aggregated.

The docs state about the limit parameter:

Maximum results per page. Defaults to 200. Cannot be more than 20,000 if reduce is true. Otherwise cannot be more than 1,000.

From testing it would appear this is not the case, and the limit appears to be 1,000 regardless of if limit is true.

shekenahglory commented 6 years ago

We had to eliminate that as a 20,000 row scan on the database is pretty intensive. You'll have to use the marker to get all the exchanges, then reduce on the client side. I'll file a ticket to get the docs updated.