cardano-community / koios-artifacts

Artifacts for https://koios.rest and https://api.koios.rest websites
Creative Commons Attribution 4.0 International
20 stars 24 forks source link

FEATURE: Fetch `plutus_contracts` in `tx_info` endpoint without the `bytecode` field #293

Closed fallen-icarus closed 2 weeks ago

fallen-icarus commented 1 month ago

I am trying to create an event log for a specific DApp of mine and I need to get the plutus contracts that were executed in each transaction for that DApp (specifically which redeemer was used for each input). I can use the tx_info endpoint for this, but I do not need the bytecode field. Given the bytecode field is by far the largest field in the response, I would like to omit this field when I query the tx_info endpoint. I already have the bytecode locally so including it with each response is a huge waste of bandwidth. This issue is exacerbated for me since my scripts can be executed together in the same transaction.

Is there a way to do this using just the url? If not, would it be possible to make the bytecode field toggle-able? Alternatively, could it be removed from the plutus_contracts array entirely? There is already the script_info endpoint one could use to get the bytecode if necessary. Since bytecodes are static, I do not think they need to be included with every transaction result. An application can lookup the bytecodes when necessary using the script_hash field returned by the tx_info endpoint's plutus_contracts field.

xray-robot commented 1 month ago

You can select only the fields you need: https://api.koios.rest/#overview--vertical-filtering

https://api.koios.rest/api/v1/blocks?select=hash
fallen-icarus commented 1 month ago

You can select only the fields you need: https://api.koios.rest/#overview--vertical-filtering

https://api.koios.rest/api/v1/blocks?select=hash

I'm guessing this was an automated response. I want to select a sub-field that is inside of an array from one of the main fields. I can use select to get plutus_contracts, but I want to further select what fields are included in the resulting array (eg, omit the bytecode field from each element in the array). AFAICT, the docs don't address this situation.

xray-robot commented 1 month ago

Try this: https://postgrest.org/en/stable/references/api/tables_views.html#json-columns

fallen-icarus commented 1 month ago

Thanks for the tip. Perhaps I am doing it wrong still, but -> doesn't seem to let me apply it to all elements in the array; I seem to have to specify a specific index. This is what I've gotten to work, but only for the first element in the array:

curl -X POST "https://preprod.koios.rest/api/v1/tx_info?select=plutus_contracts->0->input,plutus_contracts->0->spends_input"  -H "accept: application/json" -H "content-type: application/json"  -d '{"_tx_hashes":["b2401354cda8d1cf7d06a01e440ab22cd5d4a70bcfdf4ef016a4b569e7342ac7"]}' | jq

Am I doing something wrong?

fallen-icarus commented 1 month ago

@xray-robot Hey, sorry for assuming you were a bot... It was a quick reply linking to the documentation and "robot" was in the username so I just thought "this is a bot" :sweat_smile:. I realized later that I was probably mistaken. I appreciate the help. (If you actually are a bot, then you pass the turing test. Congrats!)

xray-robot commented 1 month ago

@xray-robot Hey, sorry for assuming you were a bot... It was a quick reply linking to the documentation and "robot" was in the username so I just thought "this is a bot" 😅. I realized later that I was probably mistaken. I appreciate the help. (If you actually are a bot, then you pass the turing test. Congrats!)

What if you never know the right answer to that question? Who knows, maybe robots are already among humans and no one realizes it?

Regarding the question from the topic, the example in the link seems to include all objects from the array. Perhaps you need to better understand how it works (remap is there, not just field selection). Anyway, if you find the answer, let everyone here know too

fallen-icarus commented 1 month ago

I'll keep trying, but I still haven't found a solution. I can't find any examples online of selecting only certain columns in an array. Everything is either choosing a specific array index or filtering by a column's value; neither of these are applicable to me. I just want to omit the bytecode from each element in the array.

I'm changing this issue from QUESTION to FEATURE because I am wasting over 10 kb per transaction just from the bytecode field in tx_info.

Edit: Actually, since my scripts can be executed at least 10 times per tx and the bytecode is included with each execution, I could potentially be wasting 50+ kb per transaction. Some cases could be 100+ kb. The average will likely be 20-30 kb wasted per transaction.

rdlrt commented 1 month ago

Apologies for not getting back sooner, the schema definition being consistent across endpoints, adding a toggle or removal of field would mean there is a mismatch of object. Would it help if we have a seperate endpoint for tx_plutus_contracts?

(PS: @xray-robot - it isnt possible to filter JSONB arrays created within postgres response by keys w/o knowing the index beforehand - it can be done to filter rows themselves, but not filter the output JSONB array from a single row)

fallen-icarus commented 1 month ago

Apologies for not getting back sooner ...

All good!

Would it help if we have a seperate endpoint for tx_plutus_contracts?

No, this would make the situation worse for me. I need to query the tx_info endpoint anyway because I need the rest of the information for other things.

... the schema definition being consistent across endpoints, adding a toggle or removal of field would mean there is a mismatch of object.

Personally, I struggle to see the use case in returning the bytecode in the other endpoints as well. If a DApp keeps referencing a specific script, why keep returning the bytecode for the script for every DApp transaction? I think the same applies to all UTxO fields (across all endpoints) with reference script information. In most situations, the script hash is enough. If I really need a specific script's bytecode, I can just use the script_info endpoint once.

rdlrt commented 1 month ago

Thanks for your input - while attempting to keep our delete columns to minimum, added _bytecode as an input toggle field , which will default to false , so that we can ensure that people are not using before removing in a future release. The default value for this field will be set to false (thus, change will still be breaking - but should be easy to fix/revert).

Also, the attached PR will be bringing across other toggles that will allow you to lighten the response from server further by only enabling what's needed using the toggle fields.

This will have to accompany with multiple new Conway endpoints, but should make our testnets pretty soon.