Support for Ethereum data type (uint256)

[tpmccallum]

Feature request

This is a feature request to enable Elasticsearch to support Ethereum's Smart Contract data types. I have harvested about 10 thousand event logs from the Ethereum blockchain using Elasticsearch. The documentation [1] and event log data [2] are publicly available. I realised after using Kibana to display the data that all of the Ethereum token values (which are 2 ^ 256 -1) were indexed as text. I created a mapping for the index which specified "long" (integer) but of course that did not work because the Elasticsearch long data type only supports up to 2 ^ 63 -1.

A bit of background

1 Ethereum token is stored in the Ethereum blockchain as 1000000000000000000. Essentially has 18 decimal places by default. So, 1 hundred ETH is therefore stored as 100000000000000000000 and going back the other way, one tenth of an ETH is stored as 100000000000000000.

This request is important because without a data type which supports these numbers we will not be able to perform any queries or filtering such as

if token_amount <= 123456789123456789

Also the data will not be able to be displayed in Kibana.

In my view Elasticsearch is the perfect tool for indexing blockchain data; blockchain data is essentially immaculate and consistent JSON (all data is validated for security/safety reasons).

Also obviously, having access to this data via a public facing API will really enhance the functionality of decentralised applications DApps which interact with the blockchain; instead of having to run a full node (GBs of data) mobile and handheld devices could quickly display information to the user of the DApp (there are also ways to verify the authenticity of the data but that's another story for another time).

Many thanks Tim

[1] https://github.com/CyberMiles/tim-research/tree/master/reading_event_logs [2] http://13.211.130.70:9200/uniswap_exchange_events/_search/?size=1000&pretty=true

[tpmccallum]

I am aware of this closed issue https://github.com/elastic/elasticsearch/issues/17006 and would really like to continue the conversation to see if we can find a creative way to get this done.

[elasticmachine]

Pinging @elastic/es-search

[tpmccallum]

Pinging @jpountz

[jpountz]

Thanks for sharing this use-case. Lucene doesn't expose ways to efficiently index more than 16 bytes in a way that allows for efficient range queries so this isn't something that we can address in the short term. I'm leaving this issue open, at least for now, to see how much traction it gets. If this becomes a very common need we can start a discussion about raising this limit.

[tpmccallum]

Interesting ... I see that Google now provide a public Ethereum smart contract analytics dataset.

https://cloud.google.com/blog/products/data-analytics/ethereum-bigquery-public-dataset-smart-contract-analytics

Any further thoughts or developments on this @colings86 @jpountz @elasticmachine ?

Many thanks Tim

[jpountz]

We are keeping an eye on this feature, but it is not on the short-term roadmap.

[hackmyway]

@tpmccallum thank you for creating the issue. @jpountz we also need this feature. It's kind of blocker for use to start using ElasticSearch. Was trying to index ethereum data in ES but got lot of below errors because of this limitation.

java.lang.IllegalArgumentException: Value [*] is out of range for a long

Is there a timeline when this feature release can be expected?

[hackmyway]

In Node.js we use below library to perform BigNumber(uint256) operations: https://github.com/MikeMcl/bignumber.js

[jpountz]

Sorry, no ETA at this point.

[gkozyryatskyy]

Hi guys! Is there any update on this issue?

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)