duneanalytics / dune-client

A framework for interacting with Dune Analytics' officially supported API service
Apache License 2.0
85 stars 22 forks source link

max datapoints default param #55

Open andrewhong5297 opened 1 year ago

andrewhong5297 commented 1 year ago

For new teams and users, we've instituted a max datapoints of 250,000 as default so that they don't spend too many credits on accident. This is configurable in their Dune settings, but you can also ignore this maximum using the parameter ignore_max_datapoints_per_request = True.

Just adding it as an issue here as an FYI when the team has time to add this in!

bh2smith commented 1 year ago

So I see a few somewhat informal things about data points and how they are calculated here in API pricing. This then leads me to look here

where I am seeing

curl -X GET "https://api.dune.com/api/v1/execution/{{execution_id}}/results/?ignore_max_datapoints_per_request/=true" -H x-dune-api-key:{{api_key}}'

namely

ignore_max_datapoints_per_request/=true

Is there really a random slash at the end?

I tried this for 01GZE567WK578FP9E9EGKEKQQE and got this weird thing back:

<a href="/api/v1/execution/01GZE567WK578FP9E9EGKEKQQE/results?ignore_max_datapoints_per_request/=true">Moved Permanently</a>.

This appears to mean that the dataset is too old and can not longer be recovered... actually it appears to mean that query execution failed.

I will create issues in the relevant repo.

msf commented 1 year ago

no slash, but i didn’t know about this request option, need to go read the code

On Sun, 4 Jun 2023 at 10:04, Benjamin Smith @.***> wrote:

So I see a few somewhat informal things about data points and how they are calculated here in API pricing https://dune.com/docs/api/#all-api-endpoints. This then leads me to look here https://dune.com/docs/api/api-reference/execution-results/#curl

where I am seeing

curl -X GET "https://api.dune.com/api/v1/execution/{{execution_id}}/results/?ignore_max_datapoints_per_request/=true" -H x-dune-api-key:{{api_key}}'

namely

ignore_max_datapoints_per_request/=true

Is there really a random slash at the end?

I tried this for 01GZE567WK578FP9E9EGKEKQQE and got this weird thing back:

Moved Permanently.

  • it appears the random slash can be included or excluded here and it doesn't make a difference.
  • there are some typos in the curl strings (like one to few single quotes on the headers.
  • It would be a lot more convenient if the curl commands used actual bash notation like ${DUNE_API_KEY} instead of dune query parameter notation {{api_key}}

I will create issues in the relevant repo.

— Reply to this email directly, view it on GitHub https://github.com/cowprotocol/dune-client/issues/55#issuecomment-1575479142, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAH3DCPMFWAQREPJOJWEXLXJRFQDANCNFSM6AAAAAAYZNAA5M . You are receiving this because you are subscribed to this thread.Message ID: @.***>

bh2smith commented 1 year ago

Also, I am pretty sure this endpoint does not exist:

$ export JOB_ID=01H22T5ZYMWVCAWJP8DTX3NXE2                       
$ curl -X GET "https://api.dune.com/api/v1/execution/${JOB_ID}/results/?ignore_max_datapoints_per_request=true" -H x-dune-api-key:${DUNE_API_KEY}
<a href="/api/v1/execution/01H22T5ZYMWVCAWJP8DTX3NXE2/results?ignore_max_datapoints_per_request=true">Moved Permanently</a>.
msf commented 1 year ago

yes, the trailing slash is wrong?

whats the doc page that has this?

On Sun, 4 Jun 2023 at 10:50, Benjamin Smith @.***> wrote:

Also, I am pretty sure this endpoint does not exist:

$ export JOB_ID=01H22T5ZYMWVCAWJP8DTX3NXE2 $ curl -X GET "https://api.dune.com/api/v1/execution/${JOB_ID}/results/?ignore_max_datapoints_per_request=true" -H x-dune-api-key:${DUNE_API_KEY} Moved Permanently.

— Reply to this email directly, view it on GitHub https://github.com/cowprotocol/dune-client/issues/55#issuecomment-1575494519, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAH3DC5VN3NDR3JBS4E7WTXJRK55ANCNFSM6AAAAAAYZNAA5M . You are receiving this because you commented.Message ID: @.***>

bh2smith commented 1 year ago

whats the doc page that has this?

Here I made a PR:

https://github.com/duneanalytics/docs/pull/333#issuecomment-1575497681

andrewhong5297 commented 1 year ago

ah my bad, just copied and pasted what I was given in some other ticket - thanks for fixing

bh2smith commented 1 year ago

Still not sure if the endpoint actually works though...

msf commented 1 year ago

the docs are still buggy, you still have an extra trailing slash on /.../results/?ignore_max...