search

strapdata / elassandra

Elassandra = Elasticsearch + Apache Cassandra
http://www.elassandra.io
Apache License 2.0
1.71k stars 198 forks source link

indexing failing #386

Open stela-leon opened 3 years ago

stela-leon commented 3 years ago

Hi people,

I tried to upgrade a cassandra cluster to elassandra(6.8.4.13), all went well until i tried to create an index in ES via rest call, curl just hangs while i can see the log entries below. The problem might be the X2 fields, different for each node, but I am not sure how to fix it, any help is more than welcome.

Questions: How did this happen? How do I fix it? How do I prevent it? (if possible)

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including (e.g.) index creation, mappings, settings, query etc. The easier you make for us to reproduce it, the more likely that somebody will take the time to look at it.

  1. migrated an existing cassandra cluster to elassandra
  2. started the new elassandra nodes with es disabled,
  3. drained the old nodes' data then shut scaled them down to have only elassandra updated nodes,
    1. repair the nodes, enable ES and recreate the nodes

Please provide the following information:

{"cluster_name":"cassandra-staging","cluster_uuid":"587328c5-3be1-48a9-907c-9b0d9fdfc399","version":1983,"state_uuid":"iJhsKS6MSVa2hs4vYUV2pA","master_node":"587328c5-3be1-48a9-907c-9b0d9fdfc399","blocks":{},"nodes":{"587328c5-3be1-48a9-907c-9b0d9fdfc399":{"name":"192.168.16.9","status":"ALIVE","ephemeral_id":"587328c5-3be1-48a9-907c-9b0d9fdfc399","transport_address":"192.168.16.9:9300","attributes":{"rack":"r1","dc":"dc1-staging"}},"1c97c04f-117c-4d8b-8cb8-5fba1720dcaa":{"name":"192.168.52.31","status":"ALIVE","ephemeral_id":"1c97c04f-117c-4d8b-8cb8-5fba1720dcaa","transport_address":"192.168.52.31:9300","attributes":{"dc":"dc1-staging","rack":"r1"}},"0cf0233f-a833-4a52-aadd-cefb40b584dd":{"name":"192.168.68.49","status":"ALIVE","ephemeral_id":"0cf0233f-a833-4a52-aadd-cefb40b584dd","transport_address":"192.168.68.49:9300","attributes":{"dc":"dc1-staging","rack":"r1"}}},"metadata":{"version":0,"cluster_uuid":"587328c5-3be1-48a9-907c-9b0d9fdfc399","templates":{},"indices":{},"index-graveyard":{"tombstones":[]}},"routing_table":{"indices":{}},"routing_nodes":{"unassigned":[],"nodes":{"1c97c04f-117c-4d8b-8cb8-5fba1720dcaa":[],"587328c5-3be1-48a9-907c-9b0d9fdfc399":[],"0cf0233f-a833-4a52-aadd-cefb40b584dd":[]}},"restore":{"snapshots":[]},"snapshot_deletions":{"snapshot_deletions":[]},"snapshots":{"snapshots":[]}}r

CREATE KEYSPACE es_test WITH replication = {'class': 'NetworkTopologyStrategy', 'dc1-staging': '2'} AND durable_writes = true;

CREATE TABLE es_test.test ( test_id text PRIMARY KEY ) WITH bloom_filter_fp_chance = 0.01 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'} AND comment = '' AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'} AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'} AND crc_check_chance = 1.0 AND dclocal_read_repair_chance = 0.1 AND default_time_to_live = 0 AND gc_grace_seconds = 864000 AND max_index_interval = 2048 AND memtable_flush_period_in_ms = 0 AND min_index_interval = 128 AND read_repair_chance = 0.0 AND speculative_retry = '99PERCENTILE'; `