Feature request: Mappings for BigDecimal and BigInteger

[hleb-albau]

As an elassandra user i want next mappings(at least without index) should be possible.

• BigDecimal -> Keyword
• BigInteger -> Keyword

For example, we have next table:

CREATE TABLE IF NOT EXISTS city_budgets (
    ....,
    state text,
    city text,
    budget decimal,
   .....
)

I want to search this table by city or state fields and result entity should contains budget field too.

[vroyer]

As explain in #122, BigDecimal type is not supported (BigInteger is supported, = long). This is possible in the Enterprise version when requesting elasticsearch through CQL as follow :

cassandra@cqlsh> desc KEYSPACE twitter;

CREATE KEYSPACE twitter WITH replication = {'class': 'NetworkTopologyStrategy', 'DC1': '2'} AND durable_writes = true;

CREATE TABLE twitter.tweet ( "_id" text PRIMARY KEY, es_query text, foo list, zip decimal ) WITH bloom_filter_fp_chance = 0.01 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'} AND comment = 'Auto-created by Elassandra' AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'} AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'} AND crc_check_chance = 1.0 AND dclocal_read_repair_chance = 0.1 AND default_time_to_live = 0 AND gc_grace_seconds = 864000 AND max_index_interval = 2048 AND memtable_flush_period_in_ms = 0 AND min_index_interval = 128 AND read_repair_chance = 0.0 AND speculative_retry = '99PERCENTILE'; CREATE CUSTOM INDEX elastic_tweet_idx ON twitter.tweet () USING 'org.elassandra.index.ExtendedElasticSecondaryIndex';

cassandra@cqlsh> select from twitter.tweet where es_query='{"query":{"query_string":{"query":"foo:par"}}}';

_id | es_query | foo | zip -----+----------+-----------+--------- 70 | null | ['paris'] | 4556544

(1 rows)

Le 21 nov. 2017 à 12:36, Hleb Albau notifications@github.com a écrit :

As an elassandra user i want next mappings(at least without index) should be possible.

BigDecimal -> Keyword BigInteger -> Keyword For example, we have next table:

CREATE TABLE IF NOT EXISTS city_budgets ( ...., state text, city text, budget decimal, ..... ) I want to search this table by city or state fields and result entity should contains budget field too.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/strapdata/elassandra/issues/139, or mute the thread https://github.com/notifications/unsubscribe-auth/AJzHmQn_VPLbXKPOxLeMI40UWpWKrlXGks5s4rXagaJpZM4Qlsr-.

[hleb-albau]

Yes, i got you point. I just ask about case, when we do not want to have index on decimal filed, just to get this field in elastic entities(search results) as a keyword field. Right now, i have to do such fields in cql as text field rather than decimals to support this mapping. Is it too complicated to add such mapping possibility(just mapping without index)?

CREATE TABLE IF NOT EXISTS city_budgets (
    ....,
    state text,
    city text,
    budget decimal,
   .....
)

{
    "budgets": {
      "properties": {
        "state": {
          "type": "text",
          "index": "no",
          "include_in_all": true,
          "cql_collection": "singleton"
        },
        "city": {
          "type": "text",
          "index": "no",
          "include_in_all": true,
          "cql_collection": "singleton"
        },
        "budget": {
          "type": "keyword",
          "index": "no",
          "include_in_all": false,
          "cql_collection": "singleton"
        }
      }
    }
  }

[viniciusfaleiro]

We have been testing Confluent Cassandra Connect Sinks plus Elassandra and we've been struggling with the same scenario described on this feature request. The Cassandra Connector sometimes uses decimal column types when creating tables and Elassandra can't map them. Also, we tried to manually changing the column type but then the connector won't work.

Any plans to support decimal types or any idea on how we can workaround this? Already tried manually mapping to text and got this:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Failed to execute query:null : Existing column [valor] type [decimal] mismatch with inferred type [text]"}],"type":"mapper_parsing_exception","reason":"Failed to execute query:null : Existing column [valor] type [decimal] mismatch with inferred type [text]","caused_by":{"type":"configuration_exception","reason":"Existing column [valor] type [decimal] mismatch with inferred type [text]"}},"status":400

[vroyer]

This is supported in the last release 6.2.3.16, decimal are indexed as text (but be careful, range query may be wrong).

You architecture with kafka and elassandra is interesting, is it possible for you to share some details or even write an article about it ?

[hleb-albau]

@vroyer Thanks! I will close issue than.

[viniciusfaleiro]

Thanks a lot @vroyer. We've tested and it works fine now. Regarding the architecture details: I'll try to come up with an article or something next week and let you know.

[viniciusfaleiro]

@vroyer Building Informational Databases Using Kafka Streaming and Elassandra at Ben Visa Vale https://www.linkedin.com/pulse/building-informational-databases-using-kafka-ben-visa-faleiro

[vroyer]

Great article, thanks a lot for sharing that. Vincent.