jaegertracing / jaeger-kubernetes

Support for deploying Jaeger into Kubernetes
https://jaegertracing.io/
Apache License 2.0
448 stars 158 forks source link

Collector/Query pods crashloopbackoff due to storage/keyspace issues #96

Closed tjb3 closed 6 years ago

tjb3 commented 6 years ago

My company is currently in the process of implementing Jaeger solutions for our dev environment, our implementation of the cassandra backed helm chart is exhibiting an issue where our collector and query pods are in consistent CrashLoopBackoff state due to keyspace does not exist and `no connections were made when creating the session errors.

The cluster was created Friday 06/29/18 on GKE managed clusters. Based off pod ages it appears the minion hosting Cassandra was migrated 7/1/18, all the other jaeger related pods came back online as expected however Query and Collector went into CrashLoopBackoff with the following errors reported. The cassandra schema job completed successfully and we were able to run traces through the system prior to it entering this state.

Query pod logs:

{"level":"info","ts":1530557025.932619,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":16687,"status":"unavailable"}
2018/07/02 18:43:45 error: failed to connect to 10.88.233.5:9042 due to error: Keyspace 'jaeger_v1_dc1' does not exist
2018/07/02 18:43:46 error: failed to connect to 10.88.233.5:9042 due to error: Keyspace 'jaeger_v1_dc1' does not exist
{"level":"fatal","ts":1530557026.146703,"caller":"query/main.go:100","msg":"Failed to init storage factory","error":"no connections were made when creating the session","stacktrace":"main.main.func1\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/cmd/query/main.go:100\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).execute\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:698\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).ExecuteC\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:783\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).Execute\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:736\nmain.main\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/cmd/query/main.go:165\nruntime.main\n\t/home/travis/.gimme/versions/go1.9.linux.amd64/src/runtime/proc.go:185"}

Collector pod logs:

{"level":"info","ts":1530556753.660484,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":14269,"status":"unavailable"}
2018/07/02 18:39:13 error: failed to connect to 10.88.233.5:9042 due to error: Keyspace 'jaeger_v1_dc1' does not exist
2018/07/02 18:39:13 error: failed to connect to 10.88.233.5:9042 due to error: Keyspace 'jaeger_v1_dc1' does not exist
{"level":"fatal","ts":1530556753.8979948,"caller":"collector/main.go:95","msg":"Failed to init storage factory","error":"no connections were made when creating the session","stacktrace":"main.main.func1\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/cmd/collector/main.go:95\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).execute\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:698\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).ExecuteC\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:783\ngithub.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra.(*Command).Execute\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/vendor/github.com/spf13/cobra/command.go:736\nmain.main\n\t/home/travis/gopath/src/github.com/jaegertracing/jaeger/cmd/collector/main.go:178\nruntime.main\n\t/home/travis/.gimme/versions/go1.9.linux.amd64/src/runtime/proc.go:185"}

Cassandra pod logs:

CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset (Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
CompilerOracle: dontinline org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.advanceAllocatingFrom (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V
CompilerOracle: dontinline org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents ()Z
CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stop ()V
CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stopInPartition ()V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.doFlush (I)V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeExcessSlow ()V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeSlow (JI)V
CompilerOracle: dontinline org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
CompilerOracle: inline org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody (Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes (Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes (JJIJ[J)V
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare (Ljava/nio/ByteBuffer;[B)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare ([BLjava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/lang/Object;JI)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt (JI)[B
INFO  [main] 2018-07-02 00:23:56,208 YamlConfigurationLoader.java:89 - Configuration location: file:/etc/cassandra/cassandra.yaml
INFO  [main] 2018-07-02 00:23:56,573 Config.java:495 - Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.88.233.5; broadcast_rpc_address=10.88.233.5; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=0; client_encryption_options=<REDACTED>; cluster_name=jaeger; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/var/lib/cassandra/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@3e3047e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_materialized_views=true; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=GossipingPropertyFileSnitch; file_cache_round_up=null; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=dc; internode_recv_buff_size_in_bytes=0; internode_send_buff_size_in_bytes=0; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.88.233.5; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=0; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=256; otc_backlog_expiration_interval_ms=200; otc_coalescing_enough_coalesced_messages=8; otc_coalescing_strategy=DISABLED; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/var/lib/cassandra/saved_caches; seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local,}; server_encryption_options=<REDACTED>; slow_query_log_timeout_in_ms=500; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_keep_alive_period_in_secs=300; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@37e547da; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
INFO  [main] 2018-07-02 00:23:56,574 DatabaseDescriptor.java:367 - DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
INFO  [main] 2018-07-02 00:23:56,574 DatabaseDescriptor.java:425 - Global memtable on-heap threshold is enabled at 249MB
INFO  [main] 2018-07-02 00:23:56,575 DatabaseDescriptor.java:429 - Global memtable off-heap threshold is enabled at 249MB
INFO  [main] 2018-07-02 00:23:56,712 RateBasedBackPressure.java:123 - Initialized back-pressure with high ratio: 0.9, factor: 5, flow: FAST, window size: 2000.
INFO  [main] 2018-07-02 00:23:56,713 DatabaseDescriptor.java:729 - Back-pressure is disabled with strategy org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}.
INFO  [main] 2018-07-02 00:23:56,851 GossipingPropertyFileSnitch.java:64 - Loaded cassandra-topology.properties for compatibility
INFO  [main] 2018-07-02 00:23:56,945 JMXServerUtils.java:246 - Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
INFO  [main] 2018-07-02 00:23:56,953 CassandraDaemon.java:473 - Hostname: tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local
INFO  [main] 2018-07-02 00:23:56,956 CassandraDaemon.java:480 - JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_171
INFO  [main] 2018-07-02 00:23:56,959 CassandraDaemon.java:481 - Heap size: 998.438MiB/998.438MiB
INFO  [main] 2018-07-02 00:23:56,960 CassandraDaemon.java:486 - Code Cache Non-heap memory: init = 2555904(2496K) used = 4411776(4308K) committed = 4456448(4352K) max = 251658240(245760K)
INFO  [main] 2018-07-02 00:23:56,974 CassandraDaemon.java:486 - Metaspace Non-heap memory: init = 0(0K) used = 18583264(18147K) committed = 19136512(18688K) max = -1(-1K)
INFO  [main] 2018-07-02 00:23:56,974 CassandraDaemon.java:486 - Compressed Class Space Non-heap memory: init = 0(0K) used = 2241304(2188K) committed = 2359296(2304K) max = 1073741824(1048576K)
INFO  [main] 2018-07-02 00:23:56,975 CassandraDaemon.java:486 - Par Eden Space Heap memory: init = 214827008(209792K) used = 111719168(109100K) committed = 214827008(209792K) max = 214827008(209792K)
INFO  [main] 2018-07-02 00:23:56,975 CassandraDaemon.java:486 - Par Survivor Space Heap memory: init = 26804224(26176K) used = 0(0K) committed = 26804224(26176K) max = 26804224(26176K)
INFO  [main] 2018-07-02 00:23:56,975 CassandraDaemon.java:486 - CMS Old Gen Heap memory: init = 805306368(786432K) used = 0(0K) committed = 805306368(786432K) max = 805306368(786432K)
INFO  [main] 2018-07-02 00:23:56,976 CassandraDaemon.java:488 - Classpath: /etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.9.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.13.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jctools-core-1.2.1.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.2.2.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/jstackjunit-0.0.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.44.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.4.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.3.jar:/usr/share/cassandra/lib/reporter-config3-3.0.3.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.11.2.jar:/usr/share/cassandra/apache-cassandra-thrift-3.11.2.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
INFO  [main] 2018-07-02 00:23:56,979 CassandraDaemon.java:490 - JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+UseNUMA, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:+CMSClassUnloadingEnabled, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Xms1024M, -Xmx1024M, -Xmn256M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -XX:OnOutOfMemoryError=kill -9 %p, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
WARN  [main] 2018-07-02 00:23:57,044 NativeLibrary.java:187 - Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
INFO  [main] 2018-07-02 00:23:57,045 StartupChecks.java:140 - jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
WARN  [main] 2018-07-02 00:23:57,046 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
INFO  [main] 2018-07-02 00:23:57,048 SigarLibrary.java:44 - Initializing SIGAR library
INFO  [main] 2018-07-02 00:23:57,060 SigarLibrary.java:180 - Checked OS settings and found them configured for optimal performance.
WARN  [main] 2018-07-02 00:23:57,063 StartupChecks.java:311 - Maximum number of memory map areas per process (vm.max_map_count) 65530 is too low, recommended value: 1048575, you can change it with sysctl.
WARN  [main] 2018-07-02 00:23:57,073 StartupChecks.java:332 - Directory /var/lib/cassandra/data doesn't exist
WARN  [main] 2018-07-02 00:23:57,077 StartupChecks.java:332 - Directory /var/lib/cassandra/commitlog doesn't exist
WARN  [main] 2018-07-02 00:23:57,078 StartupChecks.java:332 - Directory /var/lib/cassandra/saved_caches doesn't exist
WARN  [main] 2018-07-02 00:23:57,079 StartupChecks.java:332 - Directory /var/lib/cassandra/hints doesn't exist
INFO  [main] 2018-07-02 00:23:57,138 QueryProcessor.java:116 - Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
INFO  [main] 2018-07-02 00:23:57,677 ColumnFamilyStore.java:411 - Initializing system.IndexInfo
INFO  [main] 2018-07-02 00:23:58,987 ColumnFamilyStore.java:411 - Initializing system.batches
INFO  [main] 2018-07-02 00:23:59,001 ColumnFamilyStore.java:411 - Initializing system.paxos
INFO  [main] 2018-07-02 00:23:59,028 ColumnFamilyStore.java:411 - Initializing system.local
INFO  [main] 2018-07-02 00:23:59,034 ColumnFamilyStore.java:411 - Initializing system.peers
INFO  [main] 2018-07-02 00:23:59,040 ColumnFamilyStore.java:411 - Initializing system.peer_events
INFO  [main] 2018-07-02 00:23:59,045 ColumnFamilyStore.java:411 - Initializing system.range_xfers
INFO  [main] 2018-07-02 00:23:59,051 ColumnFamilyStore.java:411 - Initializing system.compaction_history
INFO  [main] 2018-07-02 00:23:59,056 ColumnFamilyStore.java:411 - Initializing system.sstable_activity
INFO  [main] 2018-07-02 00:23:59,063 ColumnFamilyStore.java:411 - Initializing system.size_estimates
INFO  [main] 2018-07-02 00:23:59,069 ColumnFamilyStore.java:411 - Initializing system.available_ranges
INFO  [main] 2018-07-02 00:23:59,075 ColumnFamilyStore.java:411 - Initializing system.transferred_ranges
INFO  [main] 2018-07-02 00:23:59,081 ColumnFamilyStore.java:411 - Initializing system.views_builds_in_progress
INFO  [main] 2018-07-02 00:23:59,087 ColumnFamilyStore.java:411 - Initializing system.built_views
INFO  [main] 2018-07-02 00:23:59,092 ColumnFamilyStore.java:411 - Initializing system.hints
INFO  [main] 2018-07-02 00:23:59,099 ColumnFamilyStore.java:411 - Initializing system.batchlog
INFO  [main] 2018-07-02 00:23:59,105 ColumnFamilyStore.java:411 - Initializing system.prepared_statements
INFO  [main] 2018-07-02 00:23:59,111 ColumnFamilyStore.java:411 - Initializing system.schema_keyspaces
INFO  [main] 2018-07-02 00:23:59,116 ColumnFamilyStore.java:411 - Initializing system.schema_columnfamilies
INFO  [main] 2018-07-02 00:23:59,122 ColumnFamilyStore.java:411 - Initializing system.schema_columns
INFO  [main] 2018-07-02 00:23:59,132 ColumnFamilyStore.java:411 - Initializing system.schema_triggers
INFO  [main] 2018-07-02 00:23:59,141 ColumnFamilyStore.java:411 - Initializing system.schema_usertypes
INFO  [main] 2018-07-02 00:23:59,151 ColumnFamilyStore.java:411 - Initializing system.schema_functions
INFO  [main] 2018-07-02 00:23:59,161 ColumnFamilyStore.java:411 - Initializing system.schema_aggregates
INFO  [main] 2018-07-02 00:23:59,163 ViewManager.java:137 - Not submitting build tasks for views in keyspace system as storage service is not initialized
INFO  [main] 2018-07-02 00:23:59,301 ApproximateTime.java:44 - Scheduling approximate time-check task with a precision of 10 milliseconds
INFO  [main] 2018-07-02 00:23:59,335 ColumnFamilyStore.java:411 - Initializing system_schema.keyspaces
INFO  [main] 2018-07-02 00:23:59,342 ColumnFamilyStore.java:411 - Initializing system_schema.tables
INFO  [main] 2018-07-02 00:23:59,349 ColumnFamilyStore.java:411 - Initializing system_schema.columns
INFO  [main] 2018-07-02 00:23:59,356 ColumnFamilyStore.java:411 - Initializing system_schema.triggers
INFO  [main] 2018-07-02 00:23:59,363 ColumnFamilyStore.java:411 - Initializing system_schema.dropped_columns
INFO  [main] 2018-07-02 00:23:59,369 ColumnFamilyStore.java:411 - Initializing system_schema.views
INFO  [main] 2018-07-02 00:23:59,377 ColumnFamilyStore.java:411 - Initializing system_schema.types
INFO  [main] 2018-07-02 00:23:59,429 ColumnFamilyStore.java:411 - Initializing system_schema.functions
INFO  [main] 2018-07-02 00:23:59,435 ColumnFamilyStore.java:411 - Initializing system_schema.aggregates
INFO  [main] 2018-07-02 00:23:59,442 ColumnFamilyStore.java:411 - Initializing system_schema.indexes
INFO  [main] 2018-07-02 00:23:59,444 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
INFO  [MemtableFlushWriter:1] 2018-07-02 00:24:00,086 CacheService.java:112 - Initializing key cache with capacity of 49 MBs.
INFO  [MemtableFlushWriter:1] 2018-07-02 00:24:00,131 CacheService.java:134 - Initializing row cache with capacity of 0 MBs
INFO  [MemtableFlushWriter:1] 2018-07-02 00:24:00,133 CacheService.java:163 - Initializing counter cache with capacity of 24 MBs
INFO  [MemtableFlushWriter:1] 2018-07-02 00:24:00,134 CacheService.java:174 - Scheduling counter cache save to every 7200 seconds (going to save all keys).
INFO  [CompactionExecutor:2] 2018-07-02 00:24:00,459 BufferPool.java:230 - Global buffer pool is enabled, when pool is exhausted (max is 249.000MiB) it will allocate on heap
INFO  [main] 2018-07-02 00:24:00,570 StorageService.java:600 - Populating token metadata from system tables
INFO  [main] 2018-07-02 00:24:00,677 StorageService.java:607 - Token metadata: 
INFO  [pool-3-thread-1] 2018-07-02 00:24:00,765 AutoSavingCache.java:173 - Completed loading (4 ms; 8 keys) KeyCache cache
INFO  [main] 2018-07-02 00:24:00,827 CommitLog.java:152 - No commitlog files found; skipping replay
INFO  [main] 2018-07-02 00:24:00,828 StorageService.java:600 - Populating token metadata from system tables
INFO  [main] 2018-07-02 00:24:00,847 StorageService.java:607 - Token metadata: 
INFO  [main] 2018-07-02 00:24:01,042 QueryProcessor.java:163 - Preloaded 0 prepared statements
INFO  [main] 2018-07-02 00:24:01,043 StorageService.java:618 - Cassandra version: 3.11.2
INFO  [main] 2018-07-02 00:24:01,044 StorageService.java:619 - Thrift API version: 20.1.0
INFO  [main] 2018-07-02 00:24:01,044 StorageService.java:620 - CQL supported versions: 3.4.4 (default: 3.4.4)
INFO  [main] 2018-07-02 00:24:01,045 StorageService.java:622 - Native protocol supported versions: 3/v3, 4/v4, 5/v5-beta (default: 4/v4)
INFO  [main] 2018-07-02 00:24:01,135 IndexSummaryManager.java:85 - Initializing index summary manager with a memory pool size of 49 MB and a resize interval of 60 minutes
INFO  [main] 2018-07-02 00:24:01,145 MessagingService.java:753 - Starting Messaging Service on /10.88.233.5:7000 (eth0)
WARN  [main] 2018-07-02 00:24:01,154 SystemKeyspace.java:1089 - No host ID found, created ec0a1667-7cf4-4527-81c4-82826eb2cb57 (Note: This should happen exactly once per node).
INFO  [main] 2018-07-02 00:24:01,176 OutboundTcpConnection.java:108 - OutboundTcpConnection using coalescing strategy DISABLED
INFO  [ScheduledTasks:1] 2018-07-02 00:24:01,852 TokenMetadata.java:498 - Updating topology for all endpoints that have changed
WARN  [main] 2018-07-02 00:24:32,240 Gossiper.java:1425 - Unable to gossip with any seeds but continuing since node is in its own seed list
INFO  [main] 2018-07-02 00:24:32,335 StorageService.java:707 - Loading persisted ring state
INFO  [main] 2018-07-02 00:24:32,336 StorageService.java:825 - Starting up server gossip
INFO  [main] 2018-07-02 00:24:32,422 StorageService.java:886 - This node will not auto bootstrap because it is configured to be a seed node.
INFO  [main] 2018-07-02 00:24:32,455 BootStrapper.java:228 - Generated random tokens. tokens are [-8598687621909694459, 2732304674628695086, -8860938353720218287, 3603121280014290014, -3274839593087906376, 2540826393528779592, 7141523298092347247, 5744873283902064866, -7921900519888437381, -1146463088658294905, -2999363497196282396, -2509460247053675936, -7274126478848838301, -5385980385914145695, -6104459401405295966, 6294445431741383814, 1078682267355037729, -5200096417112327145, -8913844052103848577, -6356331455035816923, 3251725814565572167, -1062718915627337136, 5545551232001845400, 1978695947911724014, 2361963748491582556, 7426946334475283750, 7422936577861223419, 6069968566359303156, 5669726325736028576, 5132135004995014301, 4124600807624542534, 1537450561155796272, 5277762343774889040, -983983987811862429, -7818523433678184293, -73668304523309456, -7257757848279164084, -5474078356652921352, 6311456682753612663, -654459396026615325, 5929971473513933658, -5217623993511960468, -5030925211957581340, 1228459630447193964, 2337688762763043393, -8482424441451239225, 8675276978224022891, -1318663781791636595, 1549881071847697622, 3735103928859555416, -6095625247075891193, 3713223273642648611, 1896543874232548925, -4449172050220549698, 2400107068246423605, -8813540133190584405, -882893964790777086, 3682214358731872277, -3798723842540595595, 2620810713072276324, -7214280725441651099, 3628378292546740494, 1594306956278608396, -2177805640746426495, -3768481370523578713, -6745793347561775568, 1086104651517358432, 5461056951350717034, 4606348639422674099, 4074506251894573710, -1459319788231470199, -5975973699600900059, 7757652435294530645, 2909691532615133877, 6870610390547832049, -1609644791176405822, -8379532566252402143, 2274801834156045382, 2129016329598736012, -6412557530528876451, -8651959309161087027, -2659395783193598071, -4160287556494896696, 3625494648636353474, 7397400596515149361, -6719937702269294798, -7014209276554578571, -7829151726480408478, -2381514338488666631, 9075709079654999426, -2778735504756399314, 4025089221088405616, -5717785889529630022, -8416725898748321072, 3952554299354715476, -7998202923141724238, -2207316021011988258, 3431890366216180499, -7219796072602005644, 349858152339773396, 8841281322393138286, 8603821531494313018, -366279638548686465, 4115297281877901743, -3150624570265420913, -652531314244463177, -6289942773733940217, 3767356552566091660, -8859572097856862544, -6218309270723478790, -3616694505960240760, -8062673431176448650, 3753024802079289848, -7101159810307419648, 5576257543571359127, 7763161217865159775, 6697717178824549770, 4240996597697744049, 8463501737338486671, 5307899266743309521, 7208788331510222486, 8178067228650587053, -1084873020099187423, 7790899799641712317, 805419693715848148, 4009789801887706546, 8486941818562642569, -4963926790863722115, -3613053096250010387, 4194578307505886451, -850644247544742793, 5837324935034890588, -184302913095183617, 7396656578956559375, -6493473036446463915, 7883620420721120354, 3737351893721971358, 1435074390668031607, -4645965456288214865, -5633252849672630296, 4356617146688878481, -7563773075165020808, -8516922155735766260, 6241138012769610520, 1181484967975311031, 336845208926491965, -4602254685735012775, -5150261006070385172, -496725161381954053, -2020498316479235748, -406013006284347877, 8056670129610300377, 8874222966151677894, 1356343850148676864, 7265716360661489304, -2762391465502602968, -765136435890521056, -6873265108376059847, -9889580490323966, 3152763081117363495, 768103020618635948, 1816558363428037037, 7934483731989406950, -7161414495688725632, 2577302725028764762, 4620049087810043567, 7184774257380084860, 8192393541062106456, 897599977287852510, 5762138658750069079, -3038329508404979407, 5627659242710494607, -2658334413752010112, -5957399573499042279, 8401966165356590767, -7164504126636102015, 7967686051435415142, 1467960940513357549, -2870061654887031419, -6599430867292309819, 3227523364807078733, -3197836052681875938, 8514359278007927363, 1476672070836018552, 5911640681502797523, 6184323446464446874, 4838834359642898927, -2304764370970336810, 1126417663279943242, 9093475620453874312, -7562893333745941516, -3569157858219359323, 9167487741004124771, -2597358713114786589, -9105696804995655395, -7989617891707543557, 2117494638467453302, -1892031643124978528, 9014422604467346342, 290255090746440929, -1943118428583499908, 8777731963810488391, -247445843110597030, -8271992775286376315, 3176080438523396513, 8102563939361032871, 1395373565843091261, -147775919615967173, -3363612067636846579, -2363608114989635143, 4273229217999207175, -6813219357366538617, 1042950920147530265, -2121560467921862965, -4706779168881952056, 4901126794150215453, -8391094857140220910, -4529585814444892304, 3672335618017708916, 5914220975675763033, -4157561411348775217, -56369168916630513, 6889634048929369898, -8252525254234240554, 8064657547312260056, 7910323944156404736, 2045357292614179647, -2793911123117424948, 3471818282376737912, -764538689129647653, 4433758549117562280, -4251853295404369260, -358585045533933268, -4391418224258099637, -4856299549488344634, -3116272244100162840, -7606608110001435580, 396308056732067231, -2847194527534495270, 4267719102328488573, -1623659299228293651, -8356897408464072752, 8553514187999435385, -5973790122480952389, -1831467611004040479, -6444911650432728764, 8691751154983327289, -6152474320379918272, -8420002825873485631, 3319638135011409030, 1916073268641761287, -8110259650500729621, -2925331046029816700, -134334794948397375, 1966050039354742983, 306652076743437859]
INFO  [main] 2018-07-02 00:24:32,462 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_traces, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=2}}, tables=[org.apache.cassandra.config.CFMetaData@5a6809a6[cfId=c5e99f16-8677-3914-b17e-960613512345,ksName=system_traces,cfName=sessions,flags=[COMPOUND],params=TableParams{comment=tracing sessions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [client command coordinator duration request started_at parameters]],partitionKeyColumns=[session_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[client, command, session_id, coordinator, request, started_at, duration, parameters],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@710975f9[cfId=8826e8e9-e16a-3728-8753-3bc1fc713c25,ksName=system_traces,cfName=events,flags=[COMPOUND],params=TableParams{comment=tracing events, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [activity source source_elapsed thread]],partitionKeyColumns=[session_id],clusteringColumns=[event_id],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[activity, event_id, session_id, source, thread, source_elapsed],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-02 00:24:32,846 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_traces as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-02 00:24:32,850 ColumnFamilyStore.java:411 - Initializing system_traces.events
INFO  [MigrationStage:1] 2018-07-02 00:24:32,855 ColumnFamilyStore.java:411 - Initializing system_traces.sessions
INFO  [main] 2018-07-02 00:24:32,875 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_distributed, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=3}}, tables=[org.apache.cassandra.config.CFMetaData@57015db3[cfId=759fffad-624b-3181-80ee-fa9a52d1f627,ksName=system_distributed,cfName=repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [coordinator exception_message exception_stacktrace finished_at parent_id range_begin range_end started_at status participants]],partitionKeyColumns=[keyspace_name, columnfamily_name],clusteringColumns=[id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[status, id, coordinator, finished_at, participants, exception_stacktrace, parent_id, range_end, range_begin, exception_message, keyspace_name, started_at, columnfamily_name],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@28a848aa[cfId=deabd734-b99d-3b9c-92e5-fd92eb5abf14,ksName=system_distributed,cfName=parent_repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [exception_message exception_stacktrace finished_at keyspace_name started_at columnfamily_names options requested_ranges successful_ranges]],partitionKeyColumns=[parent_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimeUUIDType,columnMetadata=[requested_ranges, exception_message, keyspace_name, successful_ranges, started_at, finished_at, options, exception_stacktrace, parent_id, columnfamily_names],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@12349b05[cfId=5582b59f-8e4e-35e1-b913-3acada51eb04,ksName=system_distributed,cfName=view_build_status,flags=[COMPOUND],params=TableParams{comment=Materialized View build status, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UUIDType),partitionColumns=[[] | [status]],partitionKeyColumns=[keyspace_name, view_name],clusteringColumns=[host_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[view_name, status, keyspace_name, host_id],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-02 00:24:33,016 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_distributed as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-02 00:24:33,019 ColumnFamilyStore.java:411 - Initializing system_distributed.parent_repair_history
INFO  [MigrationStage:1] 2018-07-02 00:24:33,024 ColumnFamilyStore.java:411 - Initializing system_distributed.repair_history
INFO  [MigrationStage:1] 2018-07-02 00:24:33,028 ColumnFamilyStore.java:411 - Initializing system_distributed.view_build_status
INFO  [main] 2018-07-02 00:24:33,041 StorageService.java:1449 - JOINING: Finish joining ring
INFO  [main] 2018-07-02 00:24:33,179 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_auth, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=1}}, tables=[org.apache.cassandra.config.CFMetaData@1e317848[cfId=5bc52802-de25-35ed-aeab-188eecebb090,ksName=system_auth,cfName=roles,flags=[COMPOUND],params=TableParams{comment=role definitions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [can_login is_superuser salted_hash member_of]],partitionKeyColumns=[role],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[salted_hash, member_of, role, can_login, is_superuser],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@666b1d14[cfId=0ecdaa87-f8fb-3e60-88d1-74fb36fe5c0d,ksName=system_auth,cfName=role_members,flags=[COMPOUND],params=TableParams{comment=role memberships lookup table, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[role],clusteringColumns=[member],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, member],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@60b8239b[cfId=3afbe79f-2194-31a7-add7-f5ab90d8ec9c,ksName=system_auth,cfName=role_permissions,flags=[COMPOUND],params=TableParams{comment=permissions granted to db roles, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | [permissions]],partitionKeyColumns=[role],clusteringColumns=[resource],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, resource, permissions],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@21989c9f[cfId=5f2fbdad-91f1-3946-bd25-d5da3a5c35ec,ksName=system_auth,cfName=resource_role_permissons_index,flags=[COMPOUND],params=TableParams{comment=index of db roles with permissions granted on a resource, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[resource],clusteringColumns=[role],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[resource, role],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-02 00:24:33,309 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_auth as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-02 00:24:33,312 ColumnFamilyStore.java:411 - Initializing system_auth.resource_role_permissons_index
INFO  [MigrationStage:1] 2018-07-02 00:24:33,321 ColumnFamilyStore.java:411 - Initializing system_auth.role_members
INFO  [MigrationStage:1] 2018-07-02 00:24:33,326 ColumnFamilyStore.java:411 - Initializing system_auth.role_permissions
INFO  [MigrationStage:1] 2018-07-02 00:24:33,332 ColumnFamilyStore.java:411 - Initializing system_auth.roles
INFO  [main] 2018-07-02 00:24:33,346 Gossiper.java:1670 - Waiting for gossip to settle...
INFO  [main] 2018-07-02 00:24:41,347 Gossiper.java:1701 - No gossip backlog; proceeding
INFO  [main] 2018-07-02 00:24:41,659 NativeTransportService.java:70 - Netty using native Epoll event loop
INFO  [main] 2018-07-02 00:24:41,705 Server.java:155 - Using Netty Version: [netty-buffer=netty-buffer-4.0.44.Final.452812a, netty-codec=netty-codec-4.0.44.Final.452812a, netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a, netty-codec-http=netty-codec-http-4.0.44.Final.452812a, netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a, netty-common=netty-common-4.0.44.Final.452812a, netty-handler=netty-handler-4.0.44.Final.452812a, netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb, netty-transport=netty-transport-4.0.44.Final.452812a, netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a, netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a, netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a, netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]
INFO  [main] 2018-07-02 00:24:41,705 Server.java:156 - Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...
INFO  [main] 2018-07-02 00:24:41,741 CassandraDaemon.java:529 - Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it
INFO  [OptionalTasks:1] 2018-07-02 00:24:43,492 CassandraRoleManager.java:356 - Created default superuser role 'cassandra'
INFO  [IndexSummaryManager:1] 2018-07-02 01:24:01,139 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [Service Thread] 2018-07-02 01:41:03,375 GCInspector.java:284 - ConcurrentMarkSweep GC in 279ms.  CMS Old Gen: 170698008 -> 155521608; Par Eden Space: 96182408 -> 96645488; 
INFO  [IndexSummaryManager:1] 2018-07-02 02:24:01,152 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 03:24:01,160 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:39] 2018-07-02 04:24:00,141 AutoSavingCache.java:394 - Saved KeyCache (16 items) in 9 ms
INFO  [IndexSummaryManager:1] 2018-07-02 04:24:01,165 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 05:24:01,170 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 06:24:01,175 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 07:24:01,225 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:86] 2018-07-02 08:24:00,138 AutoSavingCache.java:394 - Saved KeyCache (17 items) in 6 ms
INFO  [IndexSummaryManager:1] 2018-07-02 08:24:01,229 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 09:24:01,326 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 10:24:01,331 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 11:24:01,334 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:125] 2018-07-02 12:24:00,138 AutoSavingCache.java:394 - Saved KeyCache (18 items) in 6 ms
INFO  [IndexSummaryManager:1] 2018-07-02 12:24:01,337 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 13:24:01,341 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 14:24:01,344 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 15:24:01,347 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:186] 2018-07-02 16:24:00,138 AutoSavingCache.java:394 - Saved KeyCache (21 items) in 5 ms
INFO  [IndexSummaryManager:1] 2018-07-02 16:24:01,351 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 17:24:01,356 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 18:24:01,359 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-02 19:24:01,362 IndexSummaryRedistribution.java:76 - Redistributing index summaries
pavolloffay commented 6 years ago

@tjb3 can you verify that jaeger-cassandra-schema-job has been executed and created the namespace?

tjb3 commented 6 years ago

@pavolloffay I don't have a record of the logs sadly, but kubectl get jobs reports that it completed successfully if this is sufficient information:

╰─ k get jobs
NAME                                             DESIRED   SUCCESSFUL   AGE
tjb-jgr-cassandra-v6-1-jaeger-cassandra-schema   1         1            3d

I can try and dig the actual job logs out of our es pod if that would be helpful.

jpkrohling commented 6 years ago

@tjb3 the create-schema job is idempotent, so, it's safe to re-run it. If you still get problems after running the job again, it would be worth double checking if you are indeed talking to the appropriate Cassandra cluster.

pavolloffay commented 6 years ago

Cassandra logs should show what keyspaces have been created - it's missing in the provided output.

tjb3 commented 6 years ago

Schema logs from initialization of our node, truncated to remove the Cassandra is still not up messages:

{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuo0wzXzfbQXzPfPE","_score":1,"_source":{"log":"Cassandra is still not up at tjb-jgr-cassandra-v6-1-cassandra. Waiting 1 second.\n","stream":"stdout","message":"Cassandra is still not up at tjb-jgr-cassandra-v6-1-cassandra. Waiting 1 second.\n","@timestamp":"2018-06-29T18:07:30.292474710+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfP-","_score":1,"_source":{"log":"\n","stream":"stdout","message":"\n","@timestamp":"2018-06-29T18:07:31.818573888+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfP_","_score":1,"_source":{"log":"system_traces  system_schema  system_auth  system  system_distributed\n","stream":"stdout","message":"system_traces  system_schema  system_auth  system  system_distributed\n","@timestamp":"2018-06-29T18:07:31.819952278+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQA","_score":1,"_source":{"log":"\n","stream":"stdout","message":"\n","@timestamp":"2018-06-29T18:07:31.820988448+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQB","_score":1,"_source":{"log":"Generating the schema for the keyspace jaeger_v1_dc1 and datacenter dc1\n","stream":"stdout","message":"Generating the schema for the keyspace jaeger_v1_dc1 and datacenter dc1\n","@timestamp":"2018-06-29T18:07:31.821987413+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQC","_score":1,"_source":{"log":"Using template file /cassandra-schema/v001.cql.tmpl with parameters:\n","stream":"stderr","message":"Using template file /cassandra-schema/v001.cql.tmpl with parameters:\n","@timestamp":"2018-06-29T18:07:31.834374097+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQD","_score":1,"_source":{"log":"    mode = prod\n","stream":"stderr","message":"    mode = prod\n","@timestamp":"2018-06-29T18:07:31.834717007+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQE","_score":1,"_source":{"log":"    datacenter = dc1\n","stream":"stderr","message":"    datacenter = dc1\n","@timestamp":"2018-06-29T18:07:31.835016286+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQF","_score":1,"_source":{"log":"    keyspace = jaeger_v1_dc1\n","stream":"stderr","message":"    keyspace = jaeger_v1_dc1\n","@timestamp":"2018-06-29T18:07:31.835644874+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQG","_score":1,"_source":{"log":"    replication = {'class': 'NetworkTopologyStrategy', 'dc1': '2' }\n","stream":"stderr","message":"    replication = {'class': 'NetworkTopologyStrategy', 'dc1': '2' }\n","@timestamp":"2018-06-29T18:07:31.836258211+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQH","_score":1,"_source":{"log":"    trace_ttl = 172800\n","stream":"stderr","message":"    trace_ttl = 172800\n","@timestamp":"2018-06-29T18:07:31.836657182+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqDmzXzfbQXzPfQI","_score":1,"_source":{"log":"    dependencies_ttl = 0\n","stream":"stderr","message":"    dependencies_ttl = 0\n","@timestamp":"2018-06-29T18:07:31.836929894+00:00","tag":"kubernetes.var.lib.docker.containers.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05.7a44874f6ce3dd1bcc72514e572e037b76b7bc9372429d4c670b86ec336b5d05-json.log"}}

Cassandra logs from initialization

{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF0","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset (Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset (Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;\n","@timestamp":"2018-06-29T18:06:59.741523586+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF1","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V\n","@timestamp":"2018-06-29T18:06:59.741885540+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF2","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I\n","@timestamp":"2018-06-29T18:06:59.742093447+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF3","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.advanceAllocatingFrom (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.advanceAllocatingFrom (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V\n","@timestamp":"2018-06-29T18:06:59.742274615+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF4","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents ()Z\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents ()Z\n","@timestamp":"2018-06-29T18:06:59.742452986+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF5","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stop ()V\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stop ()V\n","@timestamp":"2018-06-29T18:06:59.742606227+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF6","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stopInPartition ()V\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stopInPartition ()V\n","@timestamp":"2018-06-29T18:06:59.742782817+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF7","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.doFlush (I)V\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.doFlush (I)V\n","@timestamp":"2018-06-29T18:06:59.742978491+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF8","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeExcessSlow ()V\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeExcessSlow ()V\n","@timestamp":"2018-06-29T18:06:59.743187839+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF9","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeSlow (JI)V\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeSlow (JI)V\n","@timestamp":"2018-06-29T18:06:59.743346161+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF-","_score":1,"_source":{"log":"CompilerOracle: dontinline org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J\n","stream":"stdout","message":"CompilerOracle: dontinline org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J\n","@timestamp":"2018-06-29T18:06:59.743513528+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfF_","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody (Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody (Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V\n","@timestamp":"2018-06-29T18:06:59.743674791+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGA","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V\n","@timestamp":"2018-06-29T18:06:59.743855817+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGB","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V\n","@timestamp":"2018-06-29T18:06:59.744054605+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGC","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I\n","@timestamp":"2018-06-29T18:06:59.744270077+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGD","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I\n","@timestamp":"2018-06-29T18:06:59.744439698+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGE","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes (Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes (Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J\n","@timestamp":"2018-06-29T18:06:59.744603706+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGF","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes (JJIJ[J)V\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes (JJIJ[J)V\n","@timestamp":"2018-06-29T18:06:59.744757998+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGG","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare (Ljava/nio/ByteBuffer;[B)I\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare (Ljava/nio/ByteBuffer;[B)I\n","@timestamp":"2018-06-29T18:06:59.744933509+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGH","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare ([BLjava/nio/ByteBuffer;)I\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare ([BLjava/nio/ByteBuffer;)I\n","@timestamp":"2018-06-29T18:06:59.745150851+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGI","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I\n","@timestamp":"2018-06-29T18:06:59.745329792+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGJ","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/lang/Object;JI)I\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/lang/Object;JI)I\n","@timestamp":"2018-06-29T18:06:59.745532539+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGK","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I\n","@timestamp":"2018-06-29T18:06:59.745690100+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGL","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I\n","@timestamp":"2018-06-29T18:06:59.745894982+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGM","_score":1,"_source":{"log":"CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt (JI)[B\n","stream":"stdout","message":"CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt (JI)[B\n","@timestamp":"2018-06-29T18:06:59.746068411+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGN","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:05:59,443 YamlConfigurationLoader.java:89 - Configuration location: file:/etc/cassandra/cassandra.yaml\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:05:59,443 YamlConfigurationLoader.java:89 - Configuration location: file:/etc/cassandra/cassandra.yaml\n","@timestamp":"2018-06-29T18:06:59.746258324+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGO","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:05:59,799 Config.java:495 - Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.88.224.8; broadcast_rpc_address=10.88.224.8; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=0; client_encryption_options=<REDACTED>; cluster_name=jaeger; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/var/lib/cassandra/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@3e3047e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_materialized_views=true; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=GossipingPropertyFileSnitch; file_cache_round_up=null; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=dc; internode_recv_buff_size_in_bytes=0; internode_send_buff_size_in_bytes=0; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.88.224.8; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=0; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=256; otc_backlog_expiration_interval_ms=200; otc_coalescing_enough_coalesced_messages=8; otc_coalescing_strategy=DISABLED; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/var/lib/cassandra/saved_caches; seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local,}; server_encryption_options=<REDACTED>; slow_query_log_timeout_in_ms=500; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_keep_alive_period_in_secs=300; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@37e547da; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:05:59,799 Config.java:495 - Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.88.224.8; broadcast_rpc_address=10.88.224.8; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=0; client_encryption_options=<REDACTED>; cluster_name=jaeger; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/var/lib/cassandra/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@3e3047e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_materialized_views=true; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=GossipingPropertyFileSnitch; file_cache_round_up=null; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=dc; internode_recv_buff_size_in_bytes=0; internode_send_buff_size_in_bytes=0; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.88.224.8; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=0; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=256; otc_backlog_expiration_interval_ms=200; otc_coalescing_enough_coalesced_messages=8; otc_coalescing_strategy=DISABLED; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/var/lib/cassandra/saved_caches; seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local,}; server_encryption_options=<REDACTED>; slow_query_log_timeout_in_ms=500; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_keep_alive_period_in_secs=300; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@37e547da; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]\n","@timestamp":"2018-06-29T18:06:59.746693078+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGP","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:05:59,800 DatabaseDescriptor.java:367 - DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:05:59,800 DatabaseDescriptor.java:367 - DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap\n","@timestamp":"2018-06-29T18:06:59.746979773+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGQ","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:05:59,801 DatabaseDescriptor.java:425 - Global memtable on-heap threshold is enabled at 249MB\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:05:59,801 DatabaseDescriptor.java:425 - Global memtable on-heap threshold is enabled at 249MB\n","@timestamp":"2018-06-29T18:06:59.747180174+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGR","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:05:59,801 DatabaseDescriptor.java:429 - Global memtable off-heap threshold is enabled at 249MB\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:05:59,801 DatabaseDescriptor.java:429 - Global memtable off-heap threshold is enabled at 249MB\n","@timestamp":"2018-06-29T18:06:59.747348289+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGS","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:05:59,952 RateBasedBackPressure.java:123 - Initialized back-pressure with high ratio: 0.9, factor: 5, flow: FAST, window size: 2000.\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:05:59,952 RateBasedBackPressure.java:123 - Initialized back-pressure with high ratio: 0.9, factor: 5, flow: FAST, window size: 2000.\n","@timestamp":"2018-06-29T18:06:59.747514327+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGT","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:05:59,952 DatabaseDescriptor.java:729 - Back-pressure is disabled with strategy org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}.\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:05:59,952 DatabaseDescriptor.java:729 - Back-pressure is disabled with strategy org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}.\n","@timestamp":"2018-06-29T18:06:59.747694489+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGU","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,093 GossipingPropertyFileSnitch.java:64 - Loaded cassandra-topology.properties for compatibility\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,093 GossipingPropertyFileSnitch.java:64 - Loaded cassandra-topology.properties for compatibility\n","@timestamp":"2018-06-29T18:06:59.747917966+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGV","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,189 JMXServerUtils.java:246 - Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,189 JMXServerUtils.java:246 - Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi\n","@timestamp":"2018-06-29T18:06:59.748114671+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGW","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,199 CassandraDaemon.java:473 - Hostname: tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,199 CassandraDaemon.java:473 - Hostname: tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local\n","@timestamp":"2018-06-29T18:06:59.748341914+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGX","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,199 CassandraDaemon.java:480 - JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_171\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,199 CassandraDaemon.java:480 - JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_171\n","@timestamp":"2018-06-29T18:06:59.748507927+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGY","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,201 CassandraDaemon.java:481 - Heap size: 998.438MiB/998.438MiB\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,201 CassandraDaemon.java:481 - Heap size: 998.438MiB/998.438MiB\n","@timestamp":"2018-06-29T18:06:59.748708779+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGZ","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,201 CassandraDaemon.java:486 - Code Cache Non-heap memory: init = 2555904(2496K) used = 4510912(4405K) committed = 4521984(4416K) max = 251658240(245760K)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,201 CassandraDaemon.java:486 - Code Cache Non-heap memory: init = 2555904(2496K) used = 4510912(4405K) committed = 4521984(4416K) max = 251658240(245760K)\n","@timestamp":"2018-06-29T18:06:59.748944830+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGa","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,202 CassandraDaemon.java:486 - Metaspace Non-heap memory: init = 0(0K) used = 18582840(18147K) committed = 19136512(18688K) max = -1(-1K)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,202 CassandraDaemon.java:486 - Metaspace Non-heap memory: init = 0(0K) used = 18582840(18147K) committed = 19136512(18688K) max = -1(-1K)\n","@timestamp":"2018-06-29T18:06:59.749135085+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGb","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,205 CassandraDaemon.java:486 - Compressed Class Space Non-heap memory: init = 0(0K) used = 2241304(2188K) committed = 2359296(2304K) max = 1073741824(1048576K)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,205 CassandraDaemon.java:486 - Compressed Class Space Non-heap memory: init = 0(0K) used = 2241304(2188K) committed = 2359296(2304K) max = 1073741824(1048576K)\n","@timestamp":"2018-06-29T18:06:59.749316385+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGc","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,206 CassandraDaemon.java:486 - Par Eden Space Heap memory: init = 214827008(209792K) used = 111733624(109114K) committed = 214827008(209792K) max = 214827008(209792K)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,206 CassandraDaemon.java:486 - Par Eden Space Heap memory: init = 214827008(209792K) used = 111733624(109114K) committed = 214827008(209792K) max = 214827008(209792K)\n","@timestamp":"2018-06-29T18:06:59.749483632+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGd","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,208 CassandraDaemon.java:486 - Par Survivor Space Heap memory: init = 26804224(26176K) used = 0(0K) committed = 26804224(26176K) max = 26804224(26176K)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,208 CassandraDaemon.java:486 - Par Survivor Space Heap memory: init = 26804224(26176K) used = 0(0K) committed = 26804224(26176K) max = 26804224(26176K)\n","@timestamp":"2018-06-29T18:06:59.749663028+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGe","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,209 CassandraDaemon.java:486 - CMS Old Gen Heap memory: init = 805306368(786432K) used = 0(0K) committed = 805306368(786432K) max = 805306368(786432K)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,209 CassandraDaemon.java:486 - CMS Old Gen Heap memory: init = 805306368(786432K) used = 0(0K) committed = 805306368(786432K) max = 805306368(786432K)\n","@timestamp":"2018-06-29T18:06:59.749864685+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGf","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,209 CassandraDaemon.java:488 - Classpath: /etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.9.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.13.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jctools-core-1.2.1.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.2.2.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/jstackjunit-0.0.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.44.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.4.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.3.jar:/usr/share/cassandra/lib/reporter-config3-3.0.3.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.11.2.jar:/usr/share/cassandra/apache-cassandra-thrift-3.11.2.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,209 CassandraDaemon.java:488 - Classpath: /etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.9.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.13.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jctools-core-1.2.1.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.2.2.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/jstackjunit-0.0.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.44.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.4.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.3.jar:/usr/share/cassandra/lib/reporter-config3-3.0.3.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.11.2.jar:/usr/share/cassandra/apache-cassandra-thrift-3.11.2.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar\n","@timestamp":"2018-06-29T18:06:59.750176143+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGg","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,210 CassandraDaemon.java:490 - JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+UseNUMA, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:+CMSClassUnloadingEnabled, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Xms1024M, -Xmx1024M, -Xmn256M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -XX:OnOutOfMemoryError=kill -9 %p, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,210 CassandraDaemon.java:490 - JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+UseNUMA, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:+CMSClassUnloadingEnabled, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Xms1024M, -Xmx1024M, -Xmn256M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -XX:OnOutOfMemoryError=kill -9 %p, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]\n","@timestamp":"2018-06-29T18:06:59.750457629+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGh","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:00,263 NativeLibrary.java:187 - Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:00,263 NativeLibrary.java:187 - Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.\n","@timestamp":"2018-06-29T18:06:59.750681470+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGi","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,263 StartupChecks.java:140 - jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,263 StartupChecks.java:140 - jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1\n","@timestamp":"2018-06-29T18:06:59.750886897+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGj","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:00,263 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:00,263 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.\n","@timestamp":"2018-06-29T18:06:59.751132452+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGk","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,266 SigarLibrary.java:44 - Initializing SIGAR library\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,266 SigarLibrary.java:44 - Initializing SIGAR library\n","@timestamp":"2018-06-29T18:06:59.751321564+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGl","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,277 SigarLibrary.java:180 - Checked OS settings and found them configured for optimal performance.\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,277 SigarLibrary.java:180 - Checked OS settings and found them configured for optimal performance.\n","@timestamp":"2018-06-29T18:06:59.751480987+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGm","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:00,287 StartupChecks.java:311 - Maximum number of memory map areas per process (vm.max_map_count) 65530 is too low, recommended value: 1048575, you can change it with sysctl.\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:00,287 StartupChecks.java:311 - Maximum number of memory map areas per process (vm.max_map_count) 65530 is too low, recommended value: 1048575, you can change it with sysctl.\n","@timestamp":"2018-06-29T18:06:59.751651542+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGn","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:00,300 StartupChecks.java:332 - Directory /var/lib/cassandra/data doesn't exist\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:00,300 StartupChecks.java:332 - Directory /var/lib/cassandra/data doesn't exist\n","@timestamp":"2018-06-29T18:06:59.751846669+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGo","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:00,311 StartupChecks.java:332 - Directory /var/lib/cassandra/commitlog doesn't exist\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:00,311 StartupChecks.java:332 - Directory /var/lib/cassandra/commitlog doesn't exist\n","@timestamp":"2018-06-29T18:06:59.752034479+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGp","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:00,312 StartupChecks.java:332 - Directory /var/lib/cassandra/saved_caches doesn't exist\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:00,312 StartupChecks.java:332 - Directory /var/lib/cassandra/saved_caches doesn't exist\n","@timestamp":"2018-06-29T18:06:59.752213607+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGq","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:00,316 StartupChecks.java:332 - Directory /var/lib/cassandra/hints doesn't exist\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:00,316 StartupChecks.java:332 - Directory /var/lib/cassandra/hints doesn't exist\n","@timestamp":"2018-06-29T18:06:59.752406088+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGr","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,367 QueryProcessor.java:116 - Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,367 QueryProcessor.java:116 - Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)\n","@timestamp":"2018-06-29T18:06:59.752591648+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGs","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:00,907 ColumnFamilyStore.java:411 - Initializing system.IndexInfo\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:00,907 ColumnFamilyStore.java:411 - Initializing system.IndexInfo\n","@timestamp":"2018-06-29T18:06:59.752758642+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGt","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,135 ColumnFamilyStore.java:411 - Initializing system.batches\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,135 ColumnFamilyStore.java:411 - Initializing system.batches\n","@timestamp":"2018-06-29T18:06:59.752984377+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGu","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,148 ColumnFamilyStore.java:411 - Initializing system.paxos\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,148 ColumnFamilyStore.java:411 - Initializing system.paxos\n","@timestamp":"2018-06-29T18:06:59.753219191+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGv","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,169 ColumnFamilyStore.java:411 - Initializing system.local\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,169 ColumnFamilyStore.java:411 - Initializing system.local\n","@timestamp":"2018-06-29T18:06:59.753381456+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGw","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,180 ColumnFamilyStore.java:411 - Initializing system.peers\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,180 ColumnFamilyStore.java:411 - Initializing system.peers\n","@timestamp":"2018-06-29T18:06:59.753565078+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGx","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,190 ColumnFamilyStore.java:411 - Initializing system.peer_events\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,190 ColumnFamilyStore.java:411 - Initializing system.peer_events\n","@timestamp":"2018-06-29T18:06:59.753715565+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGy","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,197 ColumnFamilyStore.java:411 - Initializing system.range_xfers\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,197 ColumnFamilyStore.java:411 - Initializing system.range_xfers\n","@timestamp":"2018-06-29T18:06:59.753888243+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfGz","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,204 ColumnFamilyStore.java:411 - Initializing system.compaction_history\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,204 ColumnFamilyStore.java:411 - Initializing system.compaction_history\n","@timestamp":"2018-06-29T18:06:59.754111310+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG0","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,209 ColumnFamilyStore.java:411 - Initializing system.sstable_activity\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,209 ColumnFamilyStore.java:411 - Initializing system.sstable_activity\n","@timestamp":"2018-06-29T18:06:59.754267115+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG1","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,218 ColumnFamilyStore.java:411 - Initializing system.size_estimates\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,218 ColumnFamilyStore.java:411 - Initializing system.size_estimates\n","@timestamp":"2018-06-29T18:06:59.754432271+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG2","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,225 ColumnFamilyStore.java:411 - Initializing system.available_ranges\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,225 ColumnFamilyStore.java:411 - Initializing system.available_ranges\n","@timestamp":"2018-06-29T18:06:59.754606859+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG3","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,272 ColumnFamilyStore.java:411 - Initializing system.transferred_ranges\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,272 ColumnFamilyStore.java:411 - Initializing system.transferred_ranges\n","@timestamp":"2018-06-29T18:06:59.754758463+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG4","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,279 ColumnFamilyStore.java:411 - Initializing system.views_builds_in_progress\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,279 ColumnFamilyStore.java:411 - Initializing system.views_builds_in_progress\n","@timestamp":"2018-06-29T18:06:59.754938955+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG5","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,285 ColumnFamilyStore.java:411 - Initializing system.built_views\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,285 ColumnFamilyStore.java:411 - Initializing system.built_views\n","@timestamp":"2018-06-29T18:06:59.755154749+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG6","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,292 ColumnFamilyStore.java:411 - Initializing system.hints\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,292 ColumnFamilyStore.java:411 - Initializing system.hints\n","@timestamp":"2018-06-29T18:06:59.755317644+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG7","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,300 ColumnFamilyStore.java:411 - Initializing system.batchlog\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,300 ColumnFamilyStore.java:411 - Initializing system.batchlog\n","@timestamp":"2018-06-29T18:06:59.755470504+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG8","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,305 ColumnFamilyStore.java:411 - Initializing system.prepared_statements\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,305 ColumnFamilyStore.java:411 - Initializing system.prepared_statements\n","@timestamp":"2018-06-29T18:06:59.755663456+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG9","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,310 ColumnFamilyStore.java:411 - Initializing system.schema_keyspaces\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,310 ColumnFamilyStore.java:411 - Initializing system.schema_keyspaces\n","@timestamp":"2018-06-29T18:06:59.755851138+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG-","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,316 ColumnFamilyStore.java:411 - Initializing system.schema_columnfamilies\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,316 ColumnFamilyStore.java:411 - Initializing system.schema_columnfamilies\n","@timestamp":"2018-06-29T18:06:59.756027807+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfG_","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,321 ColumnFamilyStore.java:411 - Initializing system.schema_columns\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,321 ColumnFamilyStore.java:411 - Initializing system.schema_columns\n","@timestamp":"2018-06-29T18:06:59.756257559+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHA","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,370 ColumnFamilyStore.java:411 - Initializing system.schema_triggers\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,370 ColumnFamilyStore.java:411 - Initializing system.schema_triggers\n","@timestamp":"2018-06-29T18:06:59.756453254+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHB","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,378 ColumnFamilyStore.java:411 - Initializing system.schema_usertypes\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,378 ColumnFamilyStore.java:411 - Initializing system.schema_usertypes\n","@timestamp":"2018-06-29T18:06:59.756626657+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHC","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,385 ColumnFamilyStore.java:411 - Initializing system.schema_functions\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,385 ColumnFamilyStore.java:411 - Initializing system.schema_functions\n","@timestamp":"2018-06-29T18:06:59.756879001+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHD","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,393 ColumnFamilyStore.java:411 - Initializing system.schema_aggregates\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,393 ColumnFamilyStore.java:411 - Initializing system.schema_aggregates\n","@timestamp":"2018-06-29T18:06:59.757059505+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHE","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,395 ViewManager.java:137 - Not submitting build tasks for views in keyspace system as storage service is not initialized\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,395 ViewManager.java:137 - Not submitting build tasks for views in keyspace system as storage service is not initialized\n","@timestamp":"2018-06-29T18:06:59.758023287+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHF","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,510 ApproximateTime.java:44 - Scheduling approximate time-check task with a precision of 10 milliseconds\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,510 ApproximateTime.java:44 - Scheduling approximate time-check task with a precision of 10 milliseconds\n","@timestamp":"2018-06-29T18:06:59.758217601+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHG","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,572 ColumnFamilyStore.java:411 - Initializing system_schema.keyspaces\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,572 ColumnFamilyStore.java:411 - Initializing system_schema.keyspaces\n","@timestamp":"2018-06-29T18:06:59.758434103+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHH","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,579 ColumnFamilyStore.java:411 - Initializing system_schema.tables\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,579 ColumnFamilyStore.java:411 - Initializing system_schema.tables\n","@timestamp":"2018-06-29T18:06:59.758602268+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHI","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,586 ColumnFamilyStore.java:411 - Initializing system_schema.columns\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,586 ColumnFamilyStore.java:411 - Initializing system_schema.columns\n","@timestamp":"2018-06-29T18:06:59.758826578+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHJ","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,593 ColumnFamilyStore.java:411 - Initializing system_schema.triggers\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,593 ColumnFamilyStore.java:411 - Initializing system_schema.triggers\n","@timestamp":"2018-06-29T18:06:59.759050382+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHK","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,600 ColumnFamilyStore.java:411 - Initializing system_schema.dropped_columns\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,600 ColumnFamilyStore.java:411 - Initializing system_schema.dropped_columns\n","@timestamp":"2018-06-29T18:06:59.759219989+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHL","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,608 ColumnFamilyStore.java:411 - Initializing system_schema.views\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,608 ColumnFamilyStore.java:411 - Initializing system_schema.views\n","@timestamp":"2018-06-29T18:06:59.759419476+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHM","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,616 ColumnFamilyStore.java:411 - Initializing system_schema.types\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,616 ColumnFamilyStore.java:411 - Initializing system_schema.types\n","@timestamp":"2018-06-29T18:06:59.759629262+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHN","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,624 ColumnFamilyStore.java:411 - Initializing system_schema.functions\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,624 ColumnFamilyStore.java:411 - Initializing system_schema.functions\n","@timestamp":"2018-06-29T18:06:59.759781387+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHO","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,667 ColumnFamilyStore.java:411 - Initializing system_schema.aggregates\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,667 ColumnFamilyStore.java:411 - Initializing system_schema.aggregates\n","@timestamp":"2018-06-29T18:06:59.760000493+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHP","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,675 ColumnFamilyStore.java:411 - Initializing system_schema.indexes\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,675 ColumnFamilyStore.java:411 - Initializing system_schema.indexes\n","@timestamp":"2018-06-29T18:06:59.760176121+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHQ","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:02,676 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:02,676 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_schema as storage service is not initialized\n","@timestamp":"2018-06-29T18:06:59.760331943+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHR","_score":1,"_source":{"log":"INFO  [MemtableFlushWriter:1] 2018-06-29 18:06:03,271 CacheService.java:112 - Initializing key cache with capacity of 49 MBs.\n","stream":"stdout","message":"INFO  [MemtableFlushWriter:1] 2018-06-29 18:06:03,271 CacheService.java:112 - Initializing key cache with capacity of 49 MBs.\n","@timestamp":"2018-06-29T18:06:59.760559285+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHS","_score":1,"_source":{"log":"INFO  [MemtableFlushWriter:1] 2018-06-29 18:06:03,282 CacheService.java:134 - Initializing row cache with capacity of 0 MBs\n","stream":"stdout","message":"INFO  [MemtableFlushWriter:1] 2018-06-29 18:06:03,282 CacheService.java:134 - Initializing row cache with capacity of 0 MBs\n","@timestamp":"2018-06-29T18:06:59.760768013+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHT","_score":1,"_source":{"log":"INFO  [MemtableFlushWriter:1] 2018-06-29 18:06:03,284 CacheService.java:163 - Initializing counter cache with capacity of 24 MBs\n","stream":"stdout","message":"INFO  [MemtableFlushWriter:1] 2018-06-29 18:06:03,284 CacheService.java:163 - Initializing counter cache with capacity of 24 MBs\n","@timestamp":"2018-06-29T18:06:59.760984653+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHU","_score":1,"_source":{"log":"INFO  [MemtableFlushWriter:1] 2018-06-29 18:06:03,286 CacheService.java:174 - Scheduling counter cache save to every 7200 seconds (going to save all keys).\n","stream":"stdout","message":"INFO  [MemtableFlushWriter:1] 2018-06-29 18:06:03,286 CacheService.java:174 - Scheduling counter cache save to every 7200 seconds (going to save all keys).\n","@timestamp":"2018-06-29T18:06:59.761178539+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHV","_score":1,"_source":{"log":"INFO  [CompactionExecutor:2] 2018-06-29 18:06:03,527 BufferPool.java:230 - Global buffer pool is enabled, when pool is exhausted (max is 249.000MiB) it will allocate on heap\n","stream":"stdout","message":"INFO  [CompactionExecutor:2] 2018-06-29 18:06:03,527 BufferPool.java:230 - Global buffer pool is enabled, when pool is exhausted (max is 249.000MiB) it will allocate on heap\n","@timestamp":"2018-06-29T18:06:59.761360101+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHW","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:03,685 StorageService.java:600 - Populating token metadata from system tables\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:03,685 StorageService.java:600 - Populating token metadata from system tables\n","@timestamp":"2018-06-29T18:06:59.761529020+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHX","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:03,787 StorageService.java:607 - Token metadata: \n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:03,787 StorageService.java:607 - Token metadata: \n","@timestamp":"2018-06-29T18:06:59.761743291+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHY","_score":1,"_source":{"log":"INFO  [pool-3-thread-1] 2018-06-29 18:06:03,904 AutoSavingCache.java:173 - Completed loading (8 ms; 8 keys) KeyCache cache\n","stream":"stdout","message":"INFO  [pool-3-thread-1] 2018-06-29 18:06:03,904 AutoSavingCache.java:173 - Completed loading (8 ms; 8 keys) KeyCache cache\n","@timestamp":"2018-06-29T18:06:59.761923244+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHZ","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:03,918 CommitLog.java:152 - No commitlog files found; skipping replay\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:03,918 CommitLog.java:152 - No commitlog files found; skipping replay\n","@timestamp":"2018-06-29T18:06:59.762095996+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHa","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:03,919 StorageService.java:600 - Populating token metadata from system tables\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:03,919 StorageService.java:600 - Populating token metadata from system tables\n","@timestamp":"2018-06-29T18:06:59.762296423+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHb","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:03,993 StorageService.java:607 - Token metadata: \n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:03,993 StorageService.java:607 - Token metadata: \n","@timestamp":"2018-06-29T18:06:59.762449889+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHc","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:04,204 QueryProcessor.java:163 - Preloaded 0 prepared statements\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:04,204 QueryProcessor.java:163 - Preloaded 0 prepared statements\n","@timestamp":"2018-06-29T18:06:59.762614513+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHd","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:04,206 StorageService.java:618 - Cassandra version: 3.11.2\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:04,206 StorageService.java:618 - Cassandra version: 3.11.2\n","@timestamp":"2018-06-29T18:06:59.762816069+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHe","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:04,206 StorageService.java:619 - Thrift API version: 20.1.0\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:04,206 StorageService.java:619 - Thrift API version: 20.1.0\n","@timestamp":"2018-06-29T18:06:59.763017858+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHf","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:04,206 StorageService.java:620 - CQL supported versions: 3.4.4 (default: 3.4.4)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:04,206 StorageService.java:620 - CQL supported versions: 3.4.4 (default: 3.4.4)\n","@timestamp":"2018-06-29T18:06:59.763203101+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHg","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:04,207 StorageService.java:622 - Native protocol supported versions: 3/v3, 4/v4, 5/v5-beta (default: 4/v4)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:04,207 StorageService.java:622 - Native protocol supported versions: 3/v3, 4/v4, 5/v5-beta (default: 4/v4)\n","@timestamp":"2018-06-29T18:06:59.763389692+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHh","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:04,289 IndexSummaryManager.java:85 - Initializing index summary manager with a memory pool size of 49 MB and a resize interval of 60 minutes\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:04,289 IndexSummaryManager.java:85 - Initializing index summary manager with a memory pool size of 49 MB and a resize interval of 60 minutes\n","@timestamp":"2018-06-29T18:06:59.763548571+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHi","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:04,299 MessagingService.java:753 - Starting Messaging Service on /10.88.224.8:7000 (eth0)\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:04,299 MessagingService.java:753 - Starting Messaging Service on /10.88.224.8:7000 (eth0)\n","@timestamp":"2018-06-29T18:06:59.763712561+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHj","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:04,305 SystemKeyspace.java:1089 - No host ID found, created 817148ff-fed9-49c6-957f-00f34a8f82fa (Note: This should happen exactly once per node).\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:04,305 SystemKeyspace.java:1089 - No host ID found, created 817148ff-fed9-49c6-957f-00f34a8f82fa (Note: This should happen exactly once per node).\n","@timestamp":"2018-06-29T18:06:59.763923958+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHk","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:04,325 OutboundTcpConnection.java:108 - OutboundTcpConnection using coalescing strategy DISABLED\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:04,325 OutboundTcpConnection.java:108 - OutboundTcpConnection using coalescing strategy DISABLED\n","@timestamp":"2018-06-29T18:06:59.764117635+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHl","_score":1,"_source":{"log":"INFO  [ScheduledTasks:1] 2018-06-29 18:06:05,094 TokenMetadata.java:498 - Updating topology for all endpoints that have changed\n","stream":"stdout","message":"INFO  [ScheduledTasks:1] 2018-06-29 18:06:05,094 TokenMetadata.java:498 - Updating topology for all endpoints that have changed\n","@timestamp":"2018-06-29T18:06:59.764286603+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHm","_score":1,"_source":{"log":"WARN  [main] 2018-06-29 18:06:35,375 Gossiper.java:1425 - Unable to gossip with any seeds but continuing since node is in its own seed list\n","stream":"stdout","message":"WARN  [main] 2018-06-29 18:06:35,375 Gossiper.java:1425 - Unable to gossip with any seeds but continuing since node is in its own seed list\n","@timestamp":"2018-06-29T18:06:59.764513707+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHn","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:35,408 StorageService.java:707 - Loading persisted ring state\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:35,408 StorageService.java:707 - Loading persisted ring state\n","@timestamp":"2018-06-29T18:06:59.764716185+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHo","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:35,411 StorageService.java:825 - Starting up server gossip\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:35,411 StorageService.java:825 - Starting up server gossip\n","@timestamp":"2018-06-29T18:06:59.764986740+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHp","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:35,537 StorageService.java:886 - This node will not auto bootstrap because it is configured to be a seed node.\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:35,537 StorageService.java:886 - This node will not auto bootstrap because it is configured to be a seed node.\n","@timestamp":"2018-06-29T18:06:59.765210758+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHq","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:35,579 BootStrapper.java:228 - Generated random tokens. tokens are [2533506405606004703, 6460077322823086550, -6631374347712132562, -4826926727805466392, -6855050537723272018, 6234546288898091456, -2440790028175060708, -7895660844254878453, -5290035382457489145, 6263863677186121731, 2945940009901809637, -958661750856315154, 2621122975693533158, 2720695771166282688, -3661173697799355555, -5946855517918086961, 8503087369553714317, -2099201964727959912, -5860722945401000190, -7454303245644842693, -2411446388168910650, 7862965194362486109, -1408082337708764952, 2527395576795808365, 2357418341263672730, -6596493399779329821, 9201501037869299981, 1712792465837459974, -3491725567839958008, 8092269841558003762, 5016388126443371914, 6446507217114288425, -6469197280113328664, 8386212793024769293, 6326081623040208229, -3243035410557945023, -6281151440946178722, -1779259106222702526, -6972410051118335750, 6415087217439044812, -305359866298302664, 8270376529107741145, 3148789662752529527, 8750082900127389880, 4148203664692264374, 4936086134508847132, 6023793499279051674, 3047972456816343971, 6767797183891105123, 1596067832562703582, 2345608393799278822, 7264962766775379320, -7466695935432134698, 7727841125807755585, 1257785892303287922, 4418596785757027089, 4585268393055345578, 7015844153937611805, -2887133908065214323, 9094595594355959667, 6757949057231496640, -5498302000674705021, -5397586668766096394, -1488051341428043649, -677015269541363353, 599094596133546718, 9125996785858360793, 4346216158650265360, 8553738076347604754, 1388970825756511482, -9126083281264083555, 3833625275340059784, -6175196417434756716, 9010834813522130169, 6400117515808888933, 7593480028241659938, -6509189198701708263, 6626562655352152740, -6624149781028433716, 6869044171411776432, 7562147142072076436, -6548173863839624911, 5513691495398307630, 8461850500212810785, 2668093524190851546, 9122657866152055248, 6690788165091086274, -776900286202792349, -4616899972912111117, 3726682030874039246, -7503469524512301699, 4384584783478302803, -4798194537101326296, -4023788244180179687, -8424538950430786901, -8480052861320099299, 6746744040446949352, -197207179112802947, 9152743935999726261, 2285487799224980229, -5458018286983123660, -3375963307375507009, 2949913268539420145, 7972792763019969793, -4163366373228959887, -3289056359796701669, -1820936114252078936, 2306712507488068609, 4343637013388469203, -2436839493239911280, 6371171124109546716, -867169248595809406, -4400113061506114343, -5065020669520235246, -5121986676539990219, 3485899487506430944, 964859221847963822, -947001220222490525, 5639352527860729916, 3567735878691349413, 9170588680520600619, -1844455123555320066, -3563503500168426117, 3477109456533180466, -7748752816797235895, 3006854555042039384, -1752266606508462839, -4229977845112601173, 5399655147821999102, 4814552294800111151, -5053972224701265920, 2142176283575553952, 2384244085151800365, -7862845338407317894, -6265659833510040702, 5072952843778175833, 4371053640838286306, 1061211285035819447, -3206916484230380767, -7187956320560779973, 875315358127873557, -7522510843567593793, -3369592645622670155, 2991599727402497375, 952962443956308607, -6403832452341508317, -7349349737449086174, -2569044370899176822, -379467091388265833, -1919840200460250221, -6157543220952412449, 8152822326280310802, 1589725474244290845, 8493820861750045642, -1952682180627341781, 3477045936405202157, -3954251711458528950, -1690835919235193119, -2152101030679640725, 6793183007507943017, 4616817833086323684, 6144747279487253343, 5458508267467789582, -8787510483939605906, -4169660619818470077, 167142042905033105, 8649323040275305780, -4434569644804517168, -4420184496195378990, -969238198724045993, 5011923281390621599, 7488011268013433372, -3190016306300465508, -4797696901071113303, -8852362536271434268, -7873095683738854681, -2901337823144822100, -1234967135317004652, -4426480200663686013, -6580606154050200146, 7866939345389347444, -1185480522135063989, 7039385422137080834, 2522015084889890085, -4001683717326247058, 4166139730409732077, 9046052588761720899, -4539600330252339689, 5448980683905293119, -7136763119578757668, 4061979909875706382, -4519701699772509219, 539070378157162419, -8408206809093139123, -91007961184562610, -8028335872767732184, 5061992464791898970, -4130649026025195429, 1042429691263654074, 4321215564990416363, 3884416849707512487, 2043185993180407997, -6487248525159510081, -8788724323841749259, 8918279653567677956, 28872367625451680, -8419908974879075120, 3627159001062701097, 846228092001981789, -7907459980621952629, 1850538178760414968, -4307146185998421198, 2030675503820096079, -2444736501598798731, -2791878415270866924, -7746924876315858797, 5416748130811249044, 3755884556895187935, -6097093244438419574, 2700987430434228897, 3349562300313003948, 1018312520806492988, -3807212052266330563, -4094065533950116269, -3215956425692410019, -682772886134664875, -7245328755598176544, 7201763996062463381, 6579409661866171885, -4359304518723643325, -899257561146610150, 4512617186644403540, -7598743837811221000, 8820206644567222901, 6829131170061970833, 8516899980625424399, 2992341855483474391, 4764186910427581606, -3430462618109264451, -991808667691983990, 5678490203365041655, -56794865618869196, 8713010013624123650, 1794886412593418861, -6809337082501278217, 1225015657035588151, -8854650632627739956, -3856765088945231785, 3957297540000408189, -1408377478876034485, -3231598376489920989, -4279177650329724238, -777671048329224007, -5207585533495862711, -4498483962391742692, 2251098517944930411]\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:35,579 BootStrapper.java:228 - Generated random tokens. tokens are [2533506405606004703, 6460077322823086550, -6631374347712132562, -4826926727805466392, -6855050537723272018, 6234546288898091456, -2440790028175060708, -7895660844254878453, -5290035382457489145, 6263863677186121731, 2945940009901809637, -958661750856315154, 2621122975693533158, 2720695771166282688, -3661173697799355555, -5946855517918086961, 8503087369553714317, -2099201964727959912, -5860722945401000190, -7454303245644842693, -2411446388168910650, 7862965194362486109, -1408082337708764952, 2527395576795808365, 2357418341263672730, -6596493399779329821, 9201501037869299981, 1712792465837459974, -3491725567839958008, 8092269841558003762, 5016388126443371914, 6446507217114288425, -6469197280113328664, 8386212793024769293, 6326081623040208229, -3243035410557945023, -6281151440946178722, -1779259106222702526, -6972410051118335750, 6415087217439044812, -305359866298302664, 8270376529107741145, 3148789662752529527, 8750082900127389880, 4148203664692264374, 4936086134508847132, 6023793499279051674, 3047972456816343971, 6767797183891105123, 1596067832562703582, 2345608393799278822, 7264962766775379320, -7466695935432134698, 7727841125807755585, 1257785892303287922, 4418596785757027089, 4585268393055345578, 7015844153937611805, -2887133908065214323, 9094595594355959667, 6757949057231496640, -5498302000674705021, -5397586668766096394, -1488051341428043649, -677015269541363353, 599094596133546718, 9125996785858360793, 4346216158650265360, 8553738076347604754, 1388970825756511482, -9126083281264083555, 3833625275340059784, -6175196417434756716, 9010834813522130169, 6400117515808888933, 7593480028241659938, -6509189198701708263, 6626562655352152740, -6624149781028433716, 6869044171411776432, 7562147142072076436, -6548173863839624911, 5513691495398307630, 8461850500212810785, 2668093524190851546, 9122657866152055248, 6690788165091086274, -776900286202792349, -4616899972912111117, 3726682030874039246, -7503469524512301699, 4384584783478302803, -4798194537101326296, -4023788244180179687, -8424538950430786901, -8480052861320099299, 6746744040446949352, -197207179112802947, 9152743935999726261, 2285487799224980229, -5458018286983123660, -3375963307375507009, 2949913268539420145, 7972792763019969793, -4163366373228959887, -3289056359796701669, -1820936114252078936, 2306712507488068609, 4343637013388469203, -2436839493239911280, 6371171124109546716, -867169248595809406, -4400113061506114343, -5065020669520235246, -5121986676539990219, 3485899487506430944, 964859221847963822, -947001220222490525, 5639352527860729916, 3567735878691349413, 9170588680520600619, -1844455123555320066, -3563503500168426117, 3477109456533180466, -7748752816797235895, 3006854555042039384, -1752266606508462839, -4229977845112601173, 5399655147821999102, 4814552294800111151, -5053972224701265920, 2142176283575553952, 2384244085151800365, -7862845338407317894, -6265659833510040702, 5072952843778175833, 4371053640838286306, 1061211285035819447, -3206916484230380767, -7187956320560779973, 875315358127873557, -7522510843567593793, -3369592645622670155, 2991599727402497375, 952962443956308607, -6403832452341508317, -7349349737449086174, -2569044370899176822, -379467091388265833, -1919840200460250221, -6157543220952412449, 8152822326280310802, 1589725474244290845, 8493820861750045642, -1952682180627341781, 3477045936405202157, -3954251711458528950, -1690835919235193119, -2152101030679640725, 6793183007507943017, 4616817833086323684, 6144747279487253343, 5458508267467789582, -8787510483939605906, -4169660619818470077, 167142042905033105, 8649323040275305780, -4434569644804517168, -4420184496195378990, -969238198724045993, 5011923281390621599, 7488011268013433372, -3190016306300465508, -4797696901071113303, -8852362536271434268, -7873095683738854681, -2901337823144822100, -1234967135317004652, -4426480200663686013, -6580606154050200146, 7866939345389347444, -1185480522135063989, 7039385422137080834, 2522015084889890085, -4001683717326247058, 4166139730409732077, 9046052588761720899, -4539600330252339689, 5448980683905293119, -7136763119578757668, 4061979909875706382, -4519701699772509219, 539070378157162419, -8408206809093139123, -91007961184562610, -8028335872767732184, 5061992464791898970, -4130649026025195429, 1042429691263654074, 4321215564990416363, 3884416849707512487, 2043185993180407997, -6487248525159510081, -8788724323841749259, 8918279653567677956, 28872367625451680, -8419908974879075120, 3627159001062701097, 846228092001981789, -7907459980621952629, 1850538178760414968, -4307146185998421198, 2030675503820096079, -2444736501598798731, -2791878415270866924, -7746924876315858797, 5416748130811249044, 3755884556895187935, -6097093244438419574, 2700987430434228897, 3349562300313003948, 1018312520806492988, -3807212052266330563, -4094065533950116269, -3215956425692410019, -682772886134664875, -7245328755598176544, 7201763996062463381, 6579409661866171885, -4359304518723643325, -899257561146610150, 4512617186644403540, -7598743837811221000, 8820206644567222901, 6829131170061970833, 8516899980625424399, 2992341855483474391, 4764186910427581606, -3430462618109264451, -991808667691983990, 5678490203365041655, -56794865618869196, 8713010013624123650, 1794886412593418861, -6809337082501278217, 1225015657035588151, -8854650632627739956, -3856765088945231785, 3957297540000408189, -1408377478876034485, -3231598376489920989, -4279177650329724238, -777671048329224007, -5207585533495862711, -4498483962391742692, 2251098517944930411]\n","@timestamp":"2018-06-29T18:06:59.765557922+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHr","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:35,594 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_traces, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=2}}, tables=[org.apache.cassandra.config.CFMetaData@4e276cbf[cfId=c5e99f16-8677-3914-b17e-960613512345,ksName=system_traces,cfName=sessions,flags=[COMPOUND],params=TableParams{comment=tracing sessions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [client command coordinator duration request started_at parameters]],partitionKeyColumns=[session_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[client, command, session_id, coordinator, request, started_at, duration, parameters],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@f271868[cfId=8826e8e9-e16a-3728-8753-3bc1fc713c25,ksName=system_traces,cfName=events,flags=[COMPOUND],params=TableParams{comment=tracing events, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [activity source source_elapsed thread]],partitionKeyColumns=[session_id],clusteringColumns=[event_id],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[activity, event_id, session_id, source, thread, source_elapsed],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:35,594 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_traces, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=2}}, tables=[org.apache.cassandra.config.CFMetaData@4e276cbf[cfId=c5e99f16-8677-3914-b17e-960613512345,ksName=system_traces,cfName=sessions,flags=[COMPOUND],params=TableParams{comment=tracing sessions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [client command coordinator duration request started_at parameters]],partitionKeyColumns=[session_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[client, command, session_id, coordinator, request, started_at, duration, parameters],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@f271868[cfId=8826e8e9-e16a-3728-8753-3bc1fc713c25,ksName=system_traces,cfName=events,flags=[COMPOUND],params=TableParams{comment=tracing events, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [activity source source_elapsed thread]],partitionKeyColumns=[session_id],clusteringColumns=[event_id],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[activity, event_id, session_id, source, thread, source_elapsed],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}\n","@timestamp":"2018-06-29T18:06:59.765966872+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHs","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,077 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_traces as storage service is not initialized\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,077 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_traces as storage service is not initialized\n","@timestamp":"2018-06-29T18:06:59.766209972+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHt","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,083 ColumnFamilyStore.java:411 - Initializing system_traces.events\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,083 ColumnFamilyStore.java:411 - Initializing system_traces.events\n","@timestamp":"2018-06-29T18:06:59.766373234+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHu","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,091 ColumnFamilyStore.java:411 - Initializing system_traces.sessions\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,091 ColumnFamilyStore.java:411 - Initializing system_traces.sessions\n","@timestamp":"2018-06-29T18:06:59.766553543+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHv","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:36,120 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_distributed, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=3}}, tables=[org.apache.cassandra.config.CFMetaData@2d96d088[cfId=759fffad-624b-3181-80ee-fa9a52d1f627,ksName=system_distributed,cfName=repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [coordinator exception_message exception_stacktrace finished_at parent_id range_begin range_end started_at status participants]],partitionKeyColumns=[keyspace_name, columnfamily_name],clusteringColumns=[id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[status, id, coordinator, finished_at, participants, exception_stacktrace, parent_id, range_end, range_begin, exception_message, keyspace_name, started_at, columnfamily_name],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@96935a1[cfId=deabd734-b99d-3b9c-92e5-fd92eb5abf14,ksName=system_distributed,cfName=parent_repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [exception_message exception_stacktrace finished_at keyspace_name started_at columnfamily_names options requested_ranges successful_ranges]],partitionKeyColumns=[parent_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimeUUIDType,columnMetadata=[requested_ranges, exception_message, keyspace_name, successful_ranges, started_at, finished_at, options, exception_stacktrace, parent_id, columnfamily_names],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@416190f3[cfId=5582b59f-8e4e-35e1-b913-3acada51eb04,ksName=system_distributed,cfName=view_build_status,flags=[COMPOUND],params=TableParams{comment=Materialized View build status, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UUIDType),partitionColumns=[[] | [status]],partitionKeyColumns=[keyspace_name, view_name],clusteringColumns=[host_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[view_name, status, keyspace_name, host_id],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:36,120 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_distributed, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=3}}, tables=[org.apache.cassandra.config.CFMetaData@2d96d088[cfId=759fffad-624b-3181-80ee-fa9a52d1f627,ksName=system_distributed,cfName=repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [coordinator exception_message exception_stacktrace finished_at parent_id range_begin range_end started_at status participants]],partitionKeyColumns=[keyspace_name, columnfamily_name],clusteringColumns=[id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[status, id, coordinator, finished_at, participants, exception_stacktrace, parent_id, range_end, range_begin, exception_message, keyspace_name, started_at, columnfamily_name],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@96935a1[cfId=deabd734-b99d-3b9c-92e5-fd92eb5abf14,ksName=system_distributed,cfName=parent_repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [exception_message exception_stacktrace finished_at keyspace_name started_at columnfamily_names options requested_ranges successful_ranges]],partitionKeyColumns=[parent_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimeUUIDType,columnMetadata=[requested_ranges, exception_message, keyspace_name, successful_ranges, started_at, finished_at, options, exception_stacktrace, parent_id, columnfamily_names],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@416190f3[cfId=5582b59f-8e4e-35e1-b913-3acada51eb04,ksName=system_distributed,cfName=view_build_status,flags=[COMPOUND],params=TableParams{comment=Materialized View build status, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UUIDType),partitionColumns=[[] | [status]],partitionKeyColumns=[keyspace_name, view_name],clusteringColumns=[host_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[view_name, status, keyspace_name, host_id],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}\n","@timestamp":"2018-06-29T18:06:59.766925554+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHw","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,256 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_distributed as storage service is not initialized\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,256 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_distributed as storage service is not initialized\n","@timestamp":"2018-06-29T18:06:59.767170971+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHx","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,271 ColumnFamilyStore.java:411 - Initializing system_distributed.parent_repair_history\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,271 ColumnFamilyStore.java:411 - Initializing system_distributed.parent_repair_history\n","@timestamp":"2018-06-29T18:06:59.767362991+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHy","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,279 ColumnFamilyStore.java:411 - Initializing system_distributed.repair_history\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,279 ColumnFamilyStore.java:411 - Initializing system_distributed.repair_history\n","@timestamp":"2018-06-29T18:06:59.767521491+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfHz","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,286 ColumnFamilyStore.java:411 - Initializing system_distributed.view_build_status\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,286 ColumnFamilyStore.java:411 - Initializing system_distributed.view_build_status\n","@timestamp":"2018-06-29T18:06:59.767696386+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH0","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:36,306 StorageService.java:1449 - JOINING: Finish joining ring\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:36,306 StorageService.java:1449 - JOINING: Finish joining ring\n","@timestamp":"2018-06-29T18:06:59.767877287+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH1","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:36,530 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_auth, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=1}}, tables=[org.apache.cassandra.config.CFMetaData@48c3b626[cfId=5bc52802-de25-35ed-aeab-188eecebb090,ksName=system_auth,cfName=roles,flags=[COMPOUND],params=TableParams{comment=role definitions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [can_login is_superuser salted_hash member_of]],partitionKeyColumns=[role],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[salted_hash, member_of, role, can_login, is_superuser],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@59dd170e[cfId=0ecdaa87-f8fb-3e60-88d1-74fb36fe5c0d,ksName=system_auth,cfName=role_members,flags=[COMPOUND],params=TableParams{comment=role memberships lookup table, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[role],clusteringColumns=[member],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, member],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@2393dc8c[cfId=3afbe79f-2194-31a7-add7-f5ab90d8ec9c,ksName=system_auth,cfName=role_permissions,flags=[COMPOUND],params=TableParams{comment=permissions granted to db roles, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | [permissions]],partitionKeyColumns=[role],clusteringColumns=[resource],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, resource, permissions],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@64c7c400[cfId=5f2fbdad-91f1-3946-bd25-d5da3a5c35ec,ksName=system_auth,cfName=resource_role_permissons_index,flags=[COMPOUND],params=TableParams{comment=index of db roles with permissions granted on a resource, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[resource],clusteringColumns=[role],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[resource, role],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:36,530 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_auth, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=1}}, tables=[org.apache.cassandra.config.CFMetaData@48c3b626[cfId=5bc52802-de25-35ed-aeab-188eecebb090,ksName=system_auth,cfName=roles,flags=[COMPOUND],params=TableParams{comment=role definitions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [can_login is_superuser salted_hash member_of]],partitionKeyColumns=[role],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[salted_hash, member_of, role, can_login, is_superuser],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@59dd170e[cfId=0ecdaa87-f8fb-3e60-88d1-74fb36fe5c0d,ksName=system_auth,cfName=role_members,flags=[COMPOUND],params=TableParams{comment=role memberships lookup table, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[role],clusteringColumns=[member],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, member],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@2393dc8c[cfId=3afbe79f-2194-31a7-add7-f5ab90d8ec9c,ksName=system_auth,cfName=role_permissions,flags=[COMPOUND],params=TableParams{comment=permissions granted to db roles, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | [permissions]],partitionKeyColumns=[role],clusteringColumns=[resource],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, resource, permissions],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@64c7c400[cfId=5f2fbdad-91f1-3946-bd25-d5da3a5c35ec,ksName=system_auth,cfName=resource_role_permissons_index,flags=[COMPOUND],params=TableParams{comment=index of db roles with permissions granted on a resource, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[resource],clusteringColumns=[role],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[resource, role],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}\n","@timestamp":"2018-06-29T18:06:59.768271875+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH2","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,756 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_auth as storage service is not initialized\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,756 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_auth as storage service is not initialized\n","@timestamp":"2018-06-29T18:06:59.768579015+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH3","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,762 ColumnFamilyStore.java:411 - Initializing system_auth.resource_role_permissons_index\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,762 ColumnFamilyStore.java:411 - Initializing system_auth.resource_role_permissons_index\n","@timestamp":"2018-06-29T18:06:59.768767848+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH4","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,772 ColumnFamilyStore.java:411 - Initializing system_auth.role_members\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,772 ColumnFamilyStore.java:411 - Initializing system_auth.role_members\n","@timestamp":"2018-06-29T18:06:59.768984261+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH5","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,790 ColumnFamilyStore.java:411 - Initializing system_auth.role_permissions\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,790 ColumnFamilyStore.java:411 - Initializing system_auth.role_permissions\n","@timestamp":"2018-06-29T18:06:59.769165719+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH6","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,800 ColumnFamilyStore.java:411 - Initializing system_auth.roles\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:06:36,800 ColumnFamilyStore.java:411 - Initializing system_auth.roles\n","@timestamp":"2018-06-29T18:06:59.769385287+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH7","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:36,849 Gossiper.java:1670 - Waiting for gossip to settle...\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:36,849 Gossiper.java:1670 - Waiting for gossip to settle...\n","@timestamp":"2018-06-29T18:06:59.769565387+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH8","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:44,850 Gossiper.java:1701 - No gossip backlog; proceeding\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:44,850 Gossiper.java:1701 - No gossip backlog; proceeding\n","@timestamp":"2018-06-29T18:06:59.769726641+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH9","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:45,178 NativeTransportService.java:70 - Netty using native Epoll event loop\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:45,178 NativeTransportService.java:70 - Netty using native Epoll event loop\n","@timestamp":"2018-06-29T18:06:59.769968010+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH-","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:45,352 Server.java:155 - Using Netty Version: [netty-buffer=netty-buffer-4.0.44.Final.452812a, netty-codec=netty-codec-4.0.44.Final.452812a, netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a, netty-codec-http=netty-codec-http-4.0.44.Final.452812a, netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a, netty-common=netty-common-4.0.44.Final.452812a, netty-handler=netty-handler-4.0.44.Final.452812a, netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb, netty-transport=netty-transport-4.0.44.Final.452812a, netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a, netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a, netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a, netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:45,352 Server.java:155 - Using Netty Version: [netty-buffer=netty-buffer-4.0.44.Final.452812a, netty-codec=netty-codec-4.0.44.Final.452812a, netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a, netty-codec-http=netty-codec-http-4.0.44.Final.452812a, netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a, netty-common=netty-common-4.0.44.Final.452812a, netty-handler=netty-handler-4.0.44.Final.452812a, netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb, netty-transport=netty-transport-4.0.44.Final.452812a, netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a, netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a, netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a, netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]\n","@timestamp":"2018-06-29T18:06:59.770183818+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfH_","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:45,352 Server.java:156 - Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:45,352 Server.java:156 - Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...\n","@timestamp":"2018-06-29T18:06:59.770369542+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfIA","_score":1,"_source":{"log":"INFO  [main] 2018-06-29 18:06:45,394 CassandraDaemon.java:529 - Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it\n","stream":"stdout","message":"INFO  [main] 2018-06-29 18:06:45,394 CassandraDaemon.java:529 - Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it\n","@timestamp":"2018-06-29T18:06:59.770556676+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMui4CzXzfbQXzPfIB","_score":1,"_source":{"log":"INFO  [OptionalTasks:1] 2018-06-29 18:06:47,020 CassandraRoleManager.java:356 - Created default superuser role 'cassandra'\n","stream":"stdout","message":"INFO  [OptionalTasks:1] 2018-06-29 18:06:47,020 CassandraRoleManager.java:356 - Created default superuser role 'cassandra'\n","@timestamp":"2018-06-29T18:06:59.770737915+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQp","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-1] 2018-06-29 18:07:32,099 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=jaeger_v1_dc1, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.NetworkTopologyStrategy, dc1=2}}, tables=[], views=[], functions=[], types=[]}\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-1] 2018-06-29 18:07:32,099 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=jaeger_v1_dc1, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.NetworkTopologyStrategy, dc1=2}}, tables=[], views=[], functions=[], types=[]}\n","@timestamp":"2018-06-29T18:07:32.100380005+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQq","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-1] 2018-06-29 18:07:32,619 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@21263b57[cfId=4b42fdb0-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=traces,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.LongType, org.apache.cassandra.db.marshal.LongType),partitionColumns=[[] | [duration flags operation_name parent_id process start_time logs refs tags]],partitionKeyColumns=[trace_id],clusteringColumns=[span_id, span_hash],keyValidator=org.apache.cassandra.db.marshal.BytesType,columnMetadata=[logs, span_hash, refs, start_time, span_id, process, duration, tags, operation_name, trace_id, flags, parent_id],droppedColumns={},triggers=[],indexes=[]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-1] 2018-06-29 18:07:32,619 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@21263b57[cfId=4b42fdb0-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=traces,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.LongType, org.apache.cassandra.db.marshal.LongType),partitionColumns=[[] | [duration flags operation_name parent_id process start_time logs refs tags]],partitionKeyColumns=[trace_id],clusteringColumns=[span_id, span_hash],keyValidator=org.apache.cassandra.db.marshal.BytesType,columnMetadata=[logs, span_hash, refs, start_time, span_id, process, duration, tags, operation_name, trace_id, flags, parent_id],droppedColumns={},triggers=[],indexes=[]]\n","@timestamp":"2018-06-29T18:07:32.620903831+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQr","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:07:32,777 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.traces\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:07:32,777 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.traces\n","@timestamp":"2018-06-29T18:07:32.778450039+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQs","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-6] 2018-06-29 18:07:32,904 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@62d72cd1[cfId=4b6e5370-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=service_names,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | []],partitionKeyColumns=[service_name],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[service_name],droppedColumns={},triggers=[],indexes=[]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-6] 2018-06-29 18:07:32,904 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@62d72cd1[cfId=4b6e5370-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=service_names,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | []],partitionKeyColumns=[service_name],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[service_name],droppedColumns={},triggers=[],indexes=[]]\n","@timestamp":"2018-06-29T18:07:32.905689419+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQt","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,003 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_names\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,003 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_names\n","@timestamp":"2018-06-29T18:07:33.004391624+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQu","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-4] 2018-06-29 18:07:33,028 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@4c60a5bf[cfId=4b816640-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=operation_names,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[service_name],clusteringColumns=[operation_name],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[service_name, operation_name],droppedColumns={},triggers=[],indexes=[]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-4] 2018-06-29 18:07:33,028 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@4c60a5bf[cfId=4b816640-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=operation_names,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[service_name],clusteringColumns=[operation_name],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[service_name, operation_name],droppedColumns={},triggers=[],indexes=[]]\n","@timestamp":"2018-06-29T18:07:33.030388342+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQv","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,157 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.operation_names\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,157 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.operation_names\n","@timestamp":"2018-06-29T18:07:33.158107937+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQw","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-6] 2018-06-29 18:07:33,197 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@7647d410[cfId=4b9b2fd0-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=service_operation_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType)),partitionColumns=[[] | [trace_id]],partitionKeyColumns=[service_name, operation_name],clusteringColumns=[start_time],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[start_time, service_name, operation_name, trace_id],droppedColumns={},triggers=[],indexes=[]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-6] 2018-06-29 18:07:33,197 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@7647d410[cfId=4b9b2fd0-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=service_operation_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType)),partitionColumns=[[] | [trace_id]],partitionKeyColumns=[service_name, operation_name],clusteringColumns=[start_time],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[start_time, service_name, operation_name, trace_id],droppedColumns={},triggers=[],indexes=[]]\n","@timestamp":"2018-06-29T18:07:33.199171115+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQx","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,375 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_operation_index\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,375 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_operation_index\n","@timestamp":"2018-06-29T18:07:33.375887009+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQy","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-2] 2018-06-29 18:07:33,418 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@35a3070f[cfId=4bbce8a0-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=service_name_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType)),partitionColumns=[[] | [trace_id]],partitionKeyColumns=[service_name, bucket],clusteringColumns=[start_time],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.Int32Type),columnMetadata=[start_time, service_name, bucket, trace_id],droppedColumns={},triggers=[],indexes=[]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-2] 2018-06-29 18:07:33,418 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@35a3070f[cfId=4bbce8a0-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=service_name_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType)),partitionColumns=[[] | [trace_id]],partitionKeyColumns=[service_name, bucket],clusteringColumns=[start_time],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.Int32Type),columnMetadata=[start_time, service_name, bucket, trace_id],droppedColumns={},triggers=[],indexes=[]]\n","@timestamp":"2018-06-29T18:07:33.419447513+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQz","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,576 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_name_index\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,576 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_name_index\n","@timestamp":"2018-06-29T18:07:33.578236035+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQ0","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-2] 2018-06-29 18:07:33,667 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@1c5ba9e6[cfId=4be2e730-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=duration_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.BytesType),partitionColumns=[[] | []],partitionKeyColumns=[service_name, operation_name, bucket],clusteringColumns=[duration, start_time, trace_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.TimestampType),columnMetadata=[duration, start_time, service_name, operation_name, bucket, trace_id],droppedColumns={},triggers=[],indexes=[]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-2] 2018-06-29 18:07:33,667 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@1c5ba9e6[cfId=4be2e730-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=duration_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.BytesType),partitionColumns=[[] | []],partitionKeyColumns=[service_name, operation_name, bucket],clusteringColumns=[duration, start_time, trace_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.TimestampType),columnMetadata=[duration, start_time, service_name, operation_name, bucket, trace_id],droppedColumns={},triggers=[],indexes=[]]\n","@timestamp":"2018-06-29T18:07:33.669532600+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQ1","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,779 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.duration_index\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,779 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.duration_index\n","@timestamp":"2018-06-29T18:07:33.780314949+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQ2","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-4] 2018-06-29 18:07:33,825 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@39a4878[cfId=4bfb0310-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=tag_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.BytesType, org.apache.cassandra.db.marshal.LongType),partitionColumns=[[] | []],partitionKeyColumns=[service_name, tag_key, tag_value],clusteringColumns=[start_time, trace_id, span_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[start_time, service_name, tag_key, span_id, tag_value, trace_id],droppedColumns={},triggers=[],indexes=[]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-4] 2018-06-29 18:07:33,825 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@39a4878[cfId=4bfb0310-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=tag_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.BytesType, org.apache.cassandra.db.marshal.LongType),partitionColumns=[[] | []],partitionKeyColumns=[service_name, tag_key, tag_value],clusteringColumns=[start_time, trace_id, span_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[start_time, service_name, tag_key, span_id, tag_value, trace_id],droppedColumns={},triggers=[],indexes=[]]\n","@timestamp":"2018-06-29T18:07:33.827426103+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQ3","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,980 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.tag_index\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:07:33,980 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.tag_index\n","@timestamp":"2018-06-29T18:07:33.981161193+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQ4","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-1] 2018-06-29 18:07:34,178 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@2bec8253[cfId=4c30e020-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-1] 2018-06-29 18:07:34,178 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@2bec8253[cfId=4c30e020-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[]]\n","@timestamp":"2018-06-29T18:07:34.180886491+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQ5","_score":1,"_source":{"log":"INFO  [MigrationStage:1] 2018-06-29 18:07:34,302 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.dependencies\n","stream":"stdout","message":"INFO  [MigrationStage:1] 2018-06-29 18:07:34,302 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.dependencies\n","@timestamp":"2018-06-29T18:07:34.303531473+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMuqVKzXzfbQXzPfQ6","_score":1,"_source":{"log":"INFO  [Native-Transport-Requests-3] 2018-06-29 18:07:34,380 MigrationManager.java:454 - Update table 'jaeger_v1_dc1/dependencies' From org.apache.cassandra.config.CFMetaData@7de19cd9[cfId=4c30e020-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[]] To org.apache.cassandra.config.CFMetaData@6eab060f[cfId=4c30e020-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[org.apache.cassandra.schema.IndexMetadata@73637662[id=e756441e-bae6-3c1f-8e50-a5087c4f2d55,name=dependencies_ts_index_idx,kind=CUSTOM,options={mode=SPARSE, class_name=org.apache.cassandra.index.sasi.SASIIndex, target=ts_index}]]]\n","stream":"stdout","message":"INFO  [Native-Transport-Requests-3] 2018-06-29 18:07:34,380 MigrationManager.java:454 - Update table 'jaeger_v1_dc1/dependencies' From org.apache.cassandra.config.CFMetaData@7de19cd9[cfId=4c30e020-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[]] To org.apache.cassandra.config.CFMetaData@6eab060f[cfId=4c30e020-7bc7-11e8-b2f6-0f2d757dc2d7,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[org.apache.cassandra.schema.IndexMetadata@73637662[id=e756441e-bae6-3c1f-8e50-a5087c4f2d55,name=dependencies_ts_index_idx,kind=CUSTOM,options={mode=SPARSE, class_name=org.apache.cassandra.index.sasi.SASIIndex, target=ts_index}]]]\n","@timestamp":"2018-06-29T18:07:34.383309760+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRMvg5KzXzfbQXzPf4w","_score":1,"_source":{"log":"INFO  [Service Thread] 2018-06-29 18:11:15,979 GCInspector.java:284 - ConcurrentMarkSweep GC in 253ms.  CMS Old Gen: 28622160 -> 114281584; Code Cache: 23078464 -> 22894848; Metaspace: 42365304 -> 42368152; Par Eden Space: 214524752 -> 2590912; Par Survivor Space: 25387136 -> 26080224\n","stream":"stdout","message":"INFO  [Service Thread] 2018-06-29 18:11:15,979 GCInspector.java:284 - ConcurrentMarkSweep GC in 253ms.  CMS Old Gen: 28622160 -> 114281584; Code Cache: 23078464 -> 22894848; Metaspace: 42365304 -> 42368152; Par Eden Space: 214524752 -> 2590912; Par Survivor Space: 25387136 -> 26080224\n","@timestamp":"2018-06-29T18:11:15.980816978+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRM8DmgzXzfbQXzPkW0","_score":1,"_source":{"log":"INFO  [IndexSummaryManager:1] 2018-06-29 19:06:04,293 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","stream":"stdout","message":"INFO  [IndexSummaryManager:1] 2018-06-29 19:06:04,293 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","@timestamp":"2018-06-29T19:06:04.294747021+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRNJys5zXzfbQXzPoLB","_score":1,"_source":{"log":"INFO  [IndexSummaryManager:1] 2018-06-29 20:06:04,300 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","stream":"stdout","message":"INFO  [IndexSummaryManager:1] 2018-06-29 20:06:04,300 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","@timestamp":"2018-06-29T20:06:04.301939415+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRNXhkTzXzfbQXzPr-l","_score":1,"_source":{"log":"INFO  [IndexSummaryManager:1] 2018-06-29 21:06:04,317 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","stream":"stdout","message":"INFO  [IndexSummaryManager:1] 2018-06-29 21:06:04,317 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","@timestamp":"2018-06-29T21:06:04.318942336+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRNlQH0zXzfbQXzRWUj","_score":1,"_source":{"log":"INFO  [CompactionExecutor:39] 2018-06-29 22:06:03,370 AutoSavingCache.java:394 - Saved KeyCache (481 items) in 87 ms\n","stream":"stdout","message":"INFO  [CompactionExecutor:39] 2018-06-29 22:06:03,370 AutoSavingCache.java:394 - Saved KeyCache (481 items) in 87 ms\n","@timestamp":"2018-06-29T22:06:03.371878141+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRNlQH0zXzfbQXzRWUk","_score":1,"_source":{"log":"INFO  [IndexSummaryManager:1] 2018-06-29 22:06:04,370 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","stream":"stdout","message":"INFO  [IndexSummaryManager:1] 2018-06-29 22:06:04,370 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","@timestamp":"2018-06-29T22:06:04.371637987+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRNy_fgzXzfbQXzRaKZ","_score":1,"_source":{"log":"INFO  [IndexSummaryManager:1] 2018-06-29 23:06:04,375 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","stream":"stdout","message":"INFO  [IndexSummaryManager:1] 2018-06-29 23:06:04,375 IndexSummaryRedistribution.java:76 - Redistributing index summaries\n","@timestamp":"2018-06-29T23:06:04.376267781+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
{"_index":"logstash-2018.06.29","_type":"fluentd","_id":"AWRN-UipzXzfbQXzRdTl","_score":1,"_source":{"log":"WARN  [PERIODIC-COMMIT-LOG-SYNCER] 2018-06-29 23:55:34,799 NoSpamLogger.java:94 - Out of 0 commit log syncs over the past 0.00s with average duration of Infinityms, 1 have exceeded the configured commit interval by an average of 18.32ms\n","stream":"stdout","message":"WARN  [PERIODIC-COMMIT-LOG-SYNCER] 2018-06-29 23:55:34,799 NoSpamLogger.java:94 - Out of 0 commit log syncs over the past 0.00s with average duration of Infinityms, 1 have exceeded the configured commit interval by an average of 18.32ms\n","@timestamp":"2018-06-29T23:55:34.800237606+00:00","tag":"kubernetes.var.lib.docker.containers.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049.ec8f495279487657099b897bac1724584ab25d4add68521155d6316bb81ec049-json.log"}}
tjb3 commented 6 years ago

@jpkrohling Rerunning the schema job addressed the issue, i'm able to run our test suite and tracing is visible via the query portal. I can guarantee we're not connecting to another cassandra cluster, our application stack is constrained to a single namespace in an isolated kubernetes cluster and the service endpoints our modules are using all line up as expected.

As the schema job is idempotent converting it to a cronjob for our local use might address this if it persists for us?

Attached are logs for the re-run of the schema creation and cassandra pods:

schema rerun logs:


system_traces  system_schema  system_auth  system  system_distributed

Generating the schema for the keyspace jaeger_v1_dc1 and datacenter dc1
Using template file /cassandra-schema/v001.cql.tmpl with parameters:
    mode = prod
    datacenter = dc1
    keyspace = jaeger_v1_dc1
    replication = {'class': 'NetworkTopologyStrategy', 'dc1': '2' }
    trace_ttl = 172800
    dependencies_ttl = 0

cassandra rerun logs:

CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset (Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
CompilerOracle: dontinline org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.advanceAllocatingFrom (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V
CompilerOracle: dontinline org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents ()Z
CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stop ()V
CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stopInPartition ()V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.doFlush (I)V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeExcessSlow ()V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeSlow (JI)V
CompilerOracle: dontinline org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
CompilerOracle: inline org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody (Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes (Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes (JJIJ[J)V
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare (Ljava/nio/ByteBuffer;[B)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare ([BLjava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/lang/Object;JI)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt (JI)[B
INFO  [main] 2018-07-03 00:34:38,627 YamlConfigurationLoader.java:89 - Configuration location: file:/etc/cassandra/cassandra.yaml
INFO  [main] 2018-07-03 00:34:38,997 Config.java:495 - Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.88.235.6; broadcast_rpc_address=10.88.235.6; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=0; client_encryption_options=<REDACTED>; cluster_name=jaeger; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/var/lib/cassandra/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@3e3047e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_materialized_views=true; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=GossipingPropertyFileSnitch; file_cache_round_up=null; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=dc; internode_recv_buff_size_in_bytes=0; internode_send_buff_size_in_bytes=0; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.88.235.6; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=0; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=256; otc_backlog_expiration_interval_ms=200; otc_coalescing_enough_coalesced_messages=8; otc_coalescing_strategy=DISABLED; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/var/lib/cassandra/saved_caches; seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local,}; server_encryption_options=<REDACTED>; slow_query_log_timeout_in_ms=500; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_keep_alive_period_in_secs=300; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@37e547da; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
INFO  [main] 2018-07-03 00:34:38,998 DatabaseDescriptor.java:367 - DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
INFO  [main] 2018-07-03 00:34:38,998 DatabaseDescriptor.java:425 - Global memtable on-heap threshold is enabled at 249MB
INFO  [main] 2018-07-03 00:34:38,999 DatabaseDescriptor.java:429 - Global memtable off-heap threshold is enabled at 249MB
INFO  [main] 2018-07-03 00:34:39,127 RateBasedBackPressure.java:123 - Initialized back-pressure with high ratio: 0.9, factor: 5, flow: FAST, window size: 2000.
INFO  [main] 2018-07-03 00:34:39,128 DatabaseDescriptor.java:729 - Back-pressure is disabled with strategy org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}.
INFO  [main] 2018-07-03 00:34:39,254 GossipingPropertyFileSnitch.java:64 - Loaded cassandra-topology.properties for compatibility
INFO  [main] 2018-07-03 00:34:39,386 JMXServerUtils.java:246 - Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
INFO  [main] 2018-07-03 00:34:39,392 CassandraDaemon.java:473 - Hostname: tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local
INFO  [main] 2018-07-03 00:34:39,392 CassandraDaemon.java:480 - JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_171
INFO  [main] 2018-07-03 00:34:39,393 CassandraDaemon.java:481 - Heap size: 998.438MiB/998.438MiB
INFO  [main] 2018-07-03 00:34:39,394 CassandraDaemon.java:486 - Code Cache Non-heap memory: init = 2555904(2496K) used = 4594624(4486K) committed = 4653056(4544K) max = 251658240(245760K)
INFO  [main] 2018-07-03 00:34:39,395 CassandraDaemon.java:486 - Metaspace Non-heap memory: init = 0(0K) used = 18593384(18157K) committed = 19136512(18688K) max = -1(-1K)
INFO  [main] 2018-07-03 00:34:39,396 CassandraDaemon.java:486 - Compressed Class Space Non-heap memory: init = 0(0K) used = 2241720(2189K) committed = 2359296(2304K) max = 1073741824(1048576K)
INFO  [main] 2018-07-03 00:34:39,396 CassandraDaemon.java:486 - Par Eden Space Heap memory: init = 214827008(209792K) used = 111739328(109120K) committed = 214827008(209792K) max = 214827008(209792K)
INFO  [main] 2018-07-03 00:34:39,396 CassandraDaemon.java:486 - Par Survivor Space Heap memory: init = 26804224(26176K) used = 0(0K) committed = 26804224(26176K) max = 26804224(26176K)
INFO  [main] 2018-07-03 00:34:39,397 CassandraDaemon.java:486 - CMS Old Gen Heap memory: init = 805306368(786432K) used = 0(0K) committed = 805306368(786432K) max = 805306368(786432K)
INFO  [main] 2018-07-03 00:34:39,397 CassandraDaemon.java:488 - Classpath: /etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.9.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.13.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jctools-core-1.2.1.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.2.2.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/jstackjunit-0.0.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.44.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.4.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.3.jar:/usr/share/cassandra/lib/reporter-config3-3.0.3.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.11.2.jar:/usr/share/cassandra/apache-cassandra-thrift-3.11.2.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
INFO  [main] 2018-07-03 00:34:39,398 CassandraDaemon.java:490 - JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+UseNUMA, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:+CMSClassUnloadingEnabled, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Xms1024M, -Xmx1024M, -Xmn256M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -XX:OnOutOfMemoryError=kill -9 %p, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
WARN  [main] 2018-07-03 00:34:39,443 NativeLibrary.java:187 - Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
INFO  [main] 2018-07-03 00:34:39,444 StartupChecks.java:140 - jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
WARN  [main] 2018-07-03 00:34:39,445 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
INFO  [main] 2018-07-03 00:34:39,447 SigarLibrary.java:44 - Initializing SIGAR library
INFO  [main] 2018-07-03 00:34:39,487 SigarLibrary.java:180 - Checked OS settings and found them configured for optimal performance.
WARN  [main] 2018-07-03 00:34:39,490 StartupChecks.java:311 - Maximum number of memory map areas per process (vm.max_map_count) 65530 is too low, recommended value: 1048575, you can change it with sysctl.
WARN  [main] 2018-07-03 00:34:39,501 StartupChecks.java:332 - Directory /var/lib/cassandra/data doesn't exist
WARN  [main] 2018-07-03 00:34:39,505 StartupChecks.java:332 - Directory /var/lib/cassandra/commitlog doesn't exist
WARN  [main] 2018-07-03 00:34:39,506 StartupChecks.java:332 - Directory /var/lib/cassandra/saved_caches doesn't exist
WARN  [main] 2018-07-03 00:34:39,507 StartupChecks.java:332 - Directory /var/lib/cassandra/hints doesn't exist
INFO  [main] 2018-07-03 00:34:39,553 QueryProcessor.java:116 - Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
INFO  [main] 2018-07-03 00:34:40,049 ColumnFamilyStore.java:411 - Initializing system.IndexInfo
INFO  [main] 2018-07-03 00:34:41,205 ColumnFamilyStore.java:411 - Initializing system.batches
INFO  [main] 2018-07-03 00:34:41,218 ColumnFamilyStore.java:411 - Initializing system.paxos
INFO  [main] 2018-07-03 00:34:41,235 ColumnFamilyStore.java:411 - Initializing system.local
INFO  [main] 2018-07-03 00:34:41,245 ColumnFamilyStore.java:411 - Initializing system.peers
INFO  [main] 2018-07-03 00:34:41,281 ColumnFamilyStore.java:411 - Initializing system.peer_events
INFO  [main] 2018-07-03 00:34:41,292 ColumnFamilyStore.java:411 - Initializing system.range_xfers
INFO  [main] 2018-07-03 00:34:41,302 ColumnFamilyStore.java:411 - Initializing system.compaction_history
INFO  [main] 2018-07-03 00:34:41,310 ColumnFamilyStore.java:411 - Initializing system.sstable_activity
INFO  [main] 2018-07-03 00:34:41,319 ColumnFamilyStore.java:411 - Initializing system.size_estimates
INFO  [main] 2018-07-03 00:34:41,327 ColumnFamilyStore.java:411 - Initializing system.available_ranges
INFO  [main] 2018-07-03 00:34:41,335 ColumnFamilyStore.java:411 - Initializing system.transferred_ranges
INFO  [main] 2018-07-03 00:34:41,380 ColumnFamilyStore.java:411 - Initializing system.views_builds_in_progress
INFO  [main] 2018-07-03 00:34:41,387 ColumnFamilyStore.java:411 - Initializing system.built_views
INFO  [main] 2018-07-03 00:34:41,395 ColumnFamilyStore.java:411 - Initializing system.hints
INFO  [main] 2018-07-03 00:34:41,403 ColumnFamilyStore.java:411 - Initializing system.batchlog
INFO  [main] 2018-07-03 00:34:41,411 ColumnFamilyStore.java:411 - Initializing system.prepared_statements
INFO  [main] 2018-07-03 00:34:41,418 ColumnFamilyStore.java:411 - Initializing system.schema_keyspaces
INFO  [main] 2018-07-03 00:34:41,425 ColumnFamilyStore.java:411 - Initializing system.schema_columnfamilies
INFO  [main] 2018-07-03 00:34:41,431 ColumnFamilyStore.java:411 - Initializing system.schema_columns
INFO  [main] 2018-07-03 00:34:41,478 ColumnFamilyStore.java:411 - Initializing system.schema_triggers
INFO  [main] 2018-07-03 00:34:41,485 ColumnFamilyStore.java:411 - Initializing system.schema_usertypes
INFO  [main] 2018-07-03 00:34:41,492 ColumnFamilyStore.java:411 - Initializing system.schema_functions
INFO  [main] 2018-07-03 00:34:41,500 ColumnFamilyStore.java:411 - Initializing system.schema_aggregates
INFO  [main] 2018-07-03 00:34:41,502 ViewManager.java:137 - Not submitting build tasks for views in keyspace system as storage service is not initialized
INFO  [main] 2018-07-03 00:34:41,644 ApproximateTime.java:44 - Scheduling approximate time-check task with a precision of 10 milliseconds
INFO  [main] 2018-07-03 00:34:41,693 ColumnFamilyStore.java:411 - Initializing system_schema.keyspaces
INFO  [main] 2018-07-03 00:34:41,700 ColumnFamilyStore.java:411 - Initializing system_schema.tables
INFO  [main] 2018-07-03 00:34:41,708 ColumnFamilyStore.java:411 - Initializing system_schema.columns
INFO  [main] 2018-07-03 00:34:41,714 ColumnFamilyStore.java:411 - Initializing system_schema.triggers
INFO  [main] 2018-07-03 00:34:41,718 ColumnFamilyStore.java:411 - Initializing system_schema.dropped_columns
INFO  [main] 2018-07-03 00:34:41,724 ColumnFamilyStore.java:411 - Initializing system_schema.views
INFO  [main] 2018-07-03 00:34:41,730 ColumnFamilyStore.java:411 - Initializing system_schema.types
INFO  [main] 2018-07-03 00:34:41,736 ColumnFamilyStore.java:411 - Initializing system_schema.functions
INFO  [main] 2018-07-03 00:34:41,775 ColumnFamilyStore.java:411 - Initializing system_schema.aggregates
INFO  [main] 2018-07-03 00:34:41,780 ColumnFamilyStore.java:411 - Initializing system_schema.indexes
INFO  [main] 2018-07-03 00:34:41,782 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
INFO  [MemtableFlushWriter:1] 2018-07-03 00:34:42,255 CacheService.java:112 - Initializing key cache with capacity of 49 MBs.
INFO  [MemtableFlushWriter:1] 2018-07-03 00:34:42,278 CacheService.java:134 - Initializing row cache with capacity of 0 MBs
INFO  [MemtableFlushWriter:1] 2018-07-03 00:34:42,280 CacheService.java:163 - Initializing counter cache with capacity of 24 MBs
INFO  [MemtableFlushWriter:1] 2018-07-03 00:34:42,281 CacheService.java:174 - Scheduling counter cache save to every 7200 seconds (going to save all keys).
INFO  [CompactionExecutor:2] 2018-07-03 00:34:42,436 BufferPool.java:230 - Global buffer pool is enabled, when pool is exhausted (max is 249.000MiB) it will allocate on heap
INFO  [main] 2018-07-03 00:34:42,579 StorageService.java:600 - Populating token metadata from system tables
INFO  [main] 2018-07-03 00:34:42,679 StorageService.java:607 - Token metadata: 
INFO  [pool-3-thread-1] 2018-07-03 00:34:42,713 AutoSavingCache.java:173 - Completed loading (3 ms; 5 keys) KeyCache cache
INFO  [main] 2018-07-03 00:34:42,784 CommitLog.java:152 - No commitlog files found; skipping replay
INFO  [main] 2018-07-03 00:34:42,784 StorageService.java:600 - Populating token metadata from system tables
INFO  [main] 2018-07-03 00:34:42,803 StorageService.java:607 - Token metadata: 
INFO  [main] 2018-07-03 00:34:43,007 QueryProcessor.java:163 - Preloaded 0 prepared statements
INFO  [main] 2018-07-03 00:34:43,008 StorageService.java:618 - Cassandra version: 3.11.2
INFO  [main] 2018-07-03 00:34:43,009 StorageService.java:619 - Thrift API version: 20.1.0
INFO  [main] 2018-07-03 00:34:43,009 StorageService.java:620 - CQL supported versions: 3.4.4 (default: 3.4.4)
INFO  [main] 2018-07-03 00:34:43,009 StorageService.java:622 - Native protocol supported versions: 3/v3, 4/v4, 5/v5-beta (default: 4/v4)
INFO  [main] 2018-07-03 00:34:43,084 IndexSummaryManager.java:85 - Initializing index summary manager with a memory pool size of 49 MB and a resize interval of 60 minutes
INFO  [main] 2018-07-03 00:34:43,093 MessagingService.java:753 - Starting Messaging Service on /10.88.235.6:7000 (eth0)
WARN  [main] 2018-07-03 00:34:43,098 SystemKeyspace.java:1089 - No host ID found, created fb1c40e2-fde8-42c3-8d9a-8fd508d2aa11 (Note: This should happen exactly once per node).
INFO  [main] 2018-07-03 00:34:43,124 OutboundTcpConnection.java:108 - OutboundTcpConnection using coalescing strategy DISABLED
INFO  [ScheduledTasks:1] 2018-07-03 00:34:44,256 TokenMetadata.java:498 - Updating topology for all endpoints that have changed
WARN  [main] 2018-07-03 00:35:14,183 Gossiper.java:1425 - Unable to gossip with any seeds but continuing since node is in its own seed list
INFO  [main] 2018-07-03 00:35:14,196 StorageService.java:707 - Loading persisted ring state
INFO  [main] 2018-07-03 00:35:14,198 StorageService.java:825 - Starting up server gossip
INFO  [main] 2018-07-03 00:35:14,288 StorageService.java:886 - This node will not auto bootstrap because it is configured to be a seed node.
INFO  [main] 2018-07-03 00:35:14,296 BootStrapper.java:228 - Generated random tokens. tokens are [-6870680650703343278, 6572370962446540048, -1504837719455320503, -5382028017563693915, -2545465150634803700, -6439658940490789499, 6575835424458232436, 8083535470649689546, 4223602300437572478, 8848971726904200106, -2266266641731356254, -5428310385724190562, -729724288996789701, -5020057231017049637, 6072066902630979955, 5923902485267923854, -3906184936895020555, 5085685675538766174, -7725515542099892214, 3277850324608569319, 3979316321267905055, 5901585432028103902, -1095816206921821373, 4135601183207845313, 1649863528563204374, -1554741650151500323, 444255753869355665, -4415573716090900657, 2077154547499653799, 134967561086441325, -6282417994551828799, -7569018577197279575, 2761120086046494850, 4789428354690446081, -7638631797424143902, -6398060479225616680, 8033856685739006192, 9073633259555386173, 2367462949665707049, 5155747362459059755, -7649002513991102896, -4968692653017095224, 8506364926355402452, 3417760724261006498, -8759699832313440580, 411090285057562864, -8805737868188739345, 2605425145657562603, 6334733385808641056, 1436571410092599981, -7390038863717979609, 5000670362495106412, -4383478129381452525, -2433139199711881590, -3567885988430163442, 3638378220736132312, -2162094022283268690, 2158878511619744068, 979335586076719975, -6226940661831505536, 5257955442790935673, -2318721518388152018, -8893525963164249160, -1481195728677213962, -8599362890878503516, 3585312993351013155, 2933951062819803925, -9184581774142435294, -5600410222042340621, -8323103204641784006, 8862784504806807101, 4132688042035196704, -3472904194058857099, -645919341158437656, -765767665223347587, -265512667970605716, -4828996756170573920, 1938629349660247294, -2111144678131215481, -2330127664245109919, 3451322107061073076, 6250304337559570152, 4774362484843088270, -4022776060246313234, 7541331355727638853, -5196655511095905530, 1375230650377068207, -2390767016424286920, 2358717747333798004, 6214826695824021057, 8398256732833277240, -8365432318823466120, 1650693957143897399, -3343115667520502983, -2814842936907190710, 2631152506497751205, 5005788078702600696, 7321547173298412985, -8958657228630182701, 735965836848775627, -2576342961436681035, 7102593066952840063, 444807978842977869, 4523548465491628207, 4256241601078130796, -5153352392166088861, 7192795390418522040, 7634298181546164449, 385068247353646531, -983440795706825463, -6008640574281711729, -9194040688814824268, 2438619851784867076, 5239583167715022734, -3244627338929057243, 7910687326848836894, 2945690578285970185, -8813647284173588471, 3857936870910955213, -5779047125646165511, -1211892712192502822, 7971786578442304160, 8033146023931265896, 1385776852031012411, -7615876132757533663, 6251551898262231985, 1055568537683948817, -3205203174314057777, 5986490952836067535, 1998881795227150764, 1134960853485824554, 7696416718590671099, -3414346994320203393, 5275564778725598594, 2608895712101567033, -3928229739126567784, 3951096239103323624, 4782561279796335548, -7981010681444725056, -5089697931121206395, -6319232570718996706, -5433869618252584040, -6590587657682363352, -1700850599667601191, -329056223179467485, 4769836781501214614, 2856589563468233270, -5780540282851567827, -339914895835725639, 2707897636476368460, -6667494125070566887, -6520855094570515929, -7705643224437054482, -4365882435106307312, 4464208572424288847, 6178277762004010371, 1895711473535072856, 8530295966612357456, -7143951593342547407, -8822304065959895959, 2287915726862145642, 7582484389168862260, 4792543161758578291, -3222480121977790249, -4695240396184777723, 3889634848048307562, 5734258727453018432, 249347434421973118, -292554222231971426, -6687964269641751810, -1974694666942938254, 1061139548584466255, 1256859359627535848, 7570313478073885007, 8904761805569925378, -41057472640730227, 4359162378274390898, -5689394206176305676, -1030491030409611506, 7076119362506384911, 8635492570089474181, 1572115831099244031, -8100893251929189592, 170359131828508504, 2926678807285441268, 8062915094514259101, -6260216853442465238, -4679660356227104186, -372003947603074618, 5904058733626451575, -3980854414579209257, -6290037766108632731, 8050473996586152879, 5599126957658232095, 9026997190372239037, -4617472746929485657, -4344536208973175871, -8132338859012056627, 134784705099754992, -1797720633448623857, 644651922884750174, 7808795122165173918, 5160384076685520010, -6038354135419681514, 336916679489023257, 6660467024030296345, -4891143051649606642, 4132765739488145496, -766014417442464070, -2625489126608160422, 8280530660311984146, 6117632953423400638, -4900533663715906906, -3326299854566320760, 1482838157680013714, -5360009849785394247, 6414351229501177303, -2734087930245219027, -8251124871944347861, 3631949430427951554, 7536915598228346724, -5179085316791211524, 3463770983741190603, -3900218178649014967, 686411161915147452, -9115432218651653782, -8105603716568875082, -3922053008965414427, 8265829515889630687, -5150733343493362611, 1647809829735536528, -4433707452917108254, 7946786154917411769, -9191010950398085301, 3640958838581559529, 4016598125586493387, 6766355067880388874, -8237317078836429814, -3827185482101898775, -1556614321412380197, 4276693170471990805, 7903807822221630809, 2184418085617840912, 4734520456393162766, 3578925995508619773, 2862709204367032073, 6183159620185011823, 544726665323230255, 6593416457700437060, 236383348769178110, 7948681559297024761, 6586176182131701957, 8935438433965178566, -4159751760062486069, 309139936755466413, 1810885329992309609]
INFO  [main] 2018-07-03 00:35:14,299 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_traces, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=2}}, tables=[org.apache.cassandra.config.CFMetaData@5622ee64[cfId=c5e99f16-8677-3914-b17e-960613512345,ksName=system_traces,cfName=sessions,flags=[COMPOUND],params=TableParams{comment=tracing sessions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [client command coordinator duration request started_at parameters]],partitionKeyColumns=[session_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[client, command, session_id, coordinator, request, started_at, duration, parameters],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@7c7e6fa[cfId=8826e8e9-e16a-3728-8753-3bc1fc713c25,ksName=system_traces,cfName=events,flags=[COMPOUND],params=TableParams{comment=tracing events, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [activity source source_elapsed thread]],partitionKeyColumns=[session_id],clusteringColumns=[event_id],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[activity, event_id, session_id, source, thread, source_elapsed],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-03 00:35:14,619 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_traces as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-03 00:35:14,624 ColumnFamilyStore.java:411 - Initializing system_traces.events
INFO  [MigrationStage:1] 2018-07-03 00:35:14,629 ColumnFamilyStore.java:411 - Initializing system_traces.sessions
INFO  [main] 2018-07-03 00:35:14,681 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_distributed, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=3}}, tables=[org.apache.cassandra.config.CFMetaData@adf32bc[cfId=759fffad-624b-3181-80ee-fa9a52d1f627,ksName=system_distributed,cfName=repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [coordinator exception_message exception_stacktrace finished_at parent_id range_begin range_end started_at status participants]],partitionKeyColumns=[keyspace_name, columnfamily_name],clusteringColumns=[id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[status, id, coordinator, finished_at, participants, exception_stacktrace, parent_id, range_end, range_begin, exception_message, keyspace_name, started_at, columnfamily_name],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@4587f48[cfId=deabd734-b99d-3b9c-92e5-fd92eb5abf14,ksName=system_distributed,cfName=parent_repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [exception_message exception_stacktrace finished_at keyspace_name started_at columnfamily_names options requested_ranges successful_ranges]],partitionKeyColumns=[parent_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimeUUIDType,columnMetadata=[requested_ranges, exception_message, keyspace_name, successful_ranges, started_at, finished_at, options, exception_stacktrace, parent_id, columnfamily_names],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@310ffa35[cfId=5582b59f-8e4e-35e1-b913-3acada51eb04,ksName=system_distributed,cfName=view_build_status,flags=[COMPOUND],params=TableParams{comment=Materialized View build status, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UUIDType),partitionColumns=[[] | [status]],partitionKeyColumns=[keyspace_name, view_name],clusteringColumns=[host_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[view_name, status, keyspace_name, host_id],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-03 00:35:14,784 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_distributed as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-03 00:35:14,788 ColumnFamilyStore.java:411 - Initializing system_distributed.parent_repair_history
INFO  [MigrationStage:1] 2018-07-03 00:35:14,792 ColumnFamilyStore.java:411 - Initializing system_distributed.repair_history
INFO  [MigrationStage:1] 2018-07-03 00:35:14,796 ColumnFamilyStore.java:411 - Initializing system_distributed.view_build_status
INFO  [main] 2018-07-03 00:35:14,808 StorageService.java:1449 - JOINING: Finish joining ring
INFO  [main] 2018-07-03 00:35:14,919 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_auth, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=1}}, tables=[org.apache.cassandra.config.CFMetaData@6d31e70[cfId=5bc52802-de25-35ed-aeab-188eecebb090,ksName=system_auth,cfName=roles,flags=[COMPOUND],params=TableParams{comment=role definitions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [can_login is_superuser salted_hash member_of]],partitionKeyColumns=[role],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[salted_hash, member_of, role, can_login, is_superuser],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@24334acc[cfId=0ecdaa87-f8fb-3e60-88d1-74fb36fe5c0d,ksName=system_auth,cfName=role_members,flags=[COMPOUND],params=TableParams{comment=role memberships lookup table, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[role],clusteringColumns=[member],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, member],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@33218686[cfId=3afbe79f-2194-31a7-add7-f5ab90d8ec9c,ksName=system_auth,cfName=role_permissions,flags=[COMPOUND],params=TableParams{comment=permissions granted to db roles, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | [permissions]],partitionKeyColumns=[role],clusteringColumns=[resource],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, resource, permissions],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@26af2967[cfId=5f2fbdad-91f1-3946-bd25-d5da3a5c35ec,ksName=system_auth,cfName=resource_role_permissons_index,flags=[COMPOUND],params=TableParams{comment=index of db roles with permissions granted on a resource, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[resource],clusteringColumns=[role],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[resource, role],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-03 00:35:15,049 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_auth as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-03 00:35:15,052 ColumnFamilyStore.java:411 - Initializing system_auth.resource_role_permissons_index
INFO  [MigrationStage:1] 2018-07-03 00:35:15,059 ColumnFamilyStore.java:411 - Initializing system_auth.role_members
INFO  [MigrationStage:1] 2018-07-03 00:35:15,064 ColumnFamilyStore.java:411 - Initializing system_auth.role_permissions
INFO  [MigrationStage:1] 2018-07-03 00:35:15,070 ColumnFamilyStore.java:411 - Initializing system_auth.roles
INFO  [main] 2018-07-03 00:35:15,084 Gossiper.java:1670 - Waiting for gossip to settle...
INFO  [main] 2018-07-03 00:35:23,085 Gossiper.java:1701 - No gossip backlog; proceeding
INFO  [main] 2018-07-03 00:35:23,312 NativeTransportService.java:70 - Netty using native Epoll event loop
INFO  [main] 2018-07-03 00:35:23,360 Server.java:155 - Using Netty Version: [netty-buffer=netty-buffer-4.0.44.Final.452812a, netty-codec=netty-codec-4.0.44.Final.452812a, netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a, netty-codec-http=netty-codec-http-4.0.44.Final.452812a, netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a, netty-common=netty-common-4.0.44.Final.452812a, netty-handler=netty-handler-4.0.44.Final.452812a, netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb, netty-transport=netty-transport-4.0.44.Final.452812a, netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a, netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a, netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a, netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]
INFO  [main] 2018-07-03 00:35:23,361 Server.java:156 - Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...
INFO  [main] 2018-07-03 00:35:23,406 CassandraDaemon.java:529 - Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it
INFO  [OptionalTasks:1] 2018-07-03 00:35:25,306 CassandraRoleManager.java:356 - Created default superuser role 'cassandra'
INFO  [IndexSummaryManager:1] 2018-07-03 01:34:43,087 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [Service Thread] 2018-07-03 01:56:42,055 GCInspector.java:284 - ConcurrentMarkSweep GC in 323ms.  CMS Old Gen: 169531128 -> 155517720; Metaspace: 42008768 -> 42008896; Par Eden Space: 75120944 -> 84673152; 
INFO  [IndexSummaryManager:1] 2018-07-03 02:34:43,096 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 03:34:43,103 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:34] 2018-07-03 04:34:42,286 AutoSavingCache.java:394 - Saved KeyCache (12 items) in 7 ms
INFO  [IndexSummaryManager:1] 2018-07-03 04:34:43,109 IndexSummaryRedistribution.java:76 - Redistributing index summaries
WARN  [PERIODIC-COMMIT-LOG-SYNCER] 2018-07-03 04:38:10,279 NoSpamLogger.java:94 - Out of 0 commit log syncs over the past 0.00s with average duration of Infinityms, 1 have exceeded the configured commit interval by an average of 3.71ms
INFO  [IndexSummaryManager:1] 2018-07-03 05:34:43,116 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 06:34:43,178 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 07:34:43,183 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:88] 2018-07-03 08:34:42,284 AutoSavingCache.java:394 - Saved KeyCache (13 items) in 5 ms
INFO  [IndexSummaryManager:1] 2018-07-03 08:34:43,187 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 09:34:43,191 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 10:34:43,195 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 11:34:43,199 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:133] 2018-07-03 12:34:42,286 AutoSavingCache.java:394 - Saved KeyCache (14 items) in 7 ms
INFO  [IndexSummaryManager:1] 2018-07-03 12:34:43,202 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 13:34:43,208 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [Native-Transport-Requests-12] 2018-07-03 14:24:24,400 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=jaeger_v1_dc1, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.NetworkTopologyStrategy, dc1=2}}, tables=[], views=[], functions=[], types=[]}
INFO  [Native-Transport-Requests-1] 2018-07-03 14:24:24,728 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@1d54107f[cfId=c91bca70-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=traces,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.LongType, org.apache.cassandra.db.marshal.LongType),partitionColumns=[[] | [duration flags operation_name parent_id process start_time logs refs tags]],partitionKeyColumns=[trace_id],clusteringColumns=[span_id, span_hash],keyValidator=org.apache.cassandra.db.marshal.BytesType,columnMetadata=[logs, span_hash, refs, start_time, span_id, process, duration, tags, operation_name, trace_id, flags, parent_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:24,830 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.traces
INFO  [Native-Transport-Requests-2] 2018-07-03 14:24:24,896 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@6afe8f7b[cfId=c9359400-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=service_names,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | []],partitionKeyColumns=[service_name],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[service_name],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:24,974 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_names
INFO  [Native-Transport-Requests-1] 2018-07-03 14:24:24,998 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@571eb2fa[cfId=c9452460-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=operation_names,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[service_name],clusteringColumns=[operation_name],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[service_name, operation_name],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,083 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.operation_names
INFO  [Native-Transport-Requests-6] 2018-07-03 14:24:25,105 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@3302dee0[cfId=c9557810-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=service_operation_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType)),partitionColumns=[[] | [trace_id]],partitionKeyColumns=[service_name, operation_name],clusteringColumns=[start_time],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[start_time, service_name, operation_name, trace_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,202 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_operation_index
INFO  [Native-Transport-Requests-1] 2018-07-03 14:24:25,224 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@561e3f2[cfId=c967a080-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=service_name_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType)),partitionColumns=[[] | [trace_id]],partitionKeyColumns=[service_name, bucket],clusteringColumns=[start_time],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.Int32Type),columnMetadata=[start_time, service_name, bucket, trace_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,328 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_name_index
INFO  [Native-Transport-Requests-3] 2018-07-03 14:24:25,352 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@2b9e5be8[cfId=c97b2880-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=duration_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.BytesType),partitionColumns=[[] | []],partitionKeyColumns=[service_name, operation_name, bucket],clusteringColumns=[duration, start_time, trace_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.TimestampType),columnMetadata=[duration, start_time, service_name, operation_name, bucket, trace_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,431 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.duration_index
INFO  [Native-Transport-Requests-4] 2018-07-03 14:24:25,449 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@50e7ec41[cfId=c989f590-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=tag_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.BytesType, org.apache.cassandra.db.marshal.LongType),partitionColumns=[[] | []],partitionKeyColumns=[service_name, tag_key, tag_value],clusteringColumns=[start_time, trace_id, span_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[start_time, service_name, tag_key, span_id, tag_value, trace_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,536 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.tag_index
INFO  [Native-Transport-Requests-4] 2018-07-03 14:24:25,621 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@671428ae[cfId=c9a43450-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,720 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.dependencies
INFO  [Native-Transport-Requests-5] 2018-07-03 14:24:25,786 MigrationManager.java:454 - Update table 'jaeger_v1_dc1/dependencies' From org.apache.cassandra.config.CFMetaData@675ac9cf[cfId=c9a43450-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[]] To org.apache.cassandra.config.CFMetaData@344818fc[cfId=c9a43450-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[org.apache.cassandra.schema.IndexMetadata@637cc530[id=e756441e-bae6-3c1f-8e50-a5087c4f2d55,name=dependencies_ts_index_idx,kind=CUSTOM,options={mode=SPARSE, class_name=org.apache.cassandra.index.sasi.SASIIndex, target=ts_index}]]]
jpkrohling commented 6 years ago

As the schema job is idempotent converting it to a cronjob for our local use might address this if it persists for us?

There's no harm, but there's also no reason to do so :-) This job is required to be executed only once during the whole lifecycle of your installation, as it will create the tables and related structure to store the data.

tjb3 commented 6 years ago

@jpkrohling ah i see now, our cassandra datastore doesn't have persistence enabled and as such theres no pv backing. I'm sure that must be the problem.

cassandra:
  image:
    tag: 3.11
  persistence:
    # To enable persistence, please see the documentation for the Cassandra chart
    enabled: false
  config:
    cluster_name: jaeger
    cluster_size: 1
    seed_size: 1
    dc_name: dc1
    rack_name: rack1
    endpoint_snitch: GossipingPropertyFileSnitch
    max_heap_size: 1024M
    heap_new_size: 256M
  resources:
    requests.memory: 2048Mi
    requests.cpu: 0.4
    limits.memory: 2048Mi
    limits.cpu: 0.4

The reschedule killed the pod and we lost the schema, i'll update our values to enable and test rescheduling the pod to ensure it persists.

jpkrohling commented 6 years ago

Looks like it's not a problem with Jaeger itself, nor with the Kubernetes templates. I'm therefore closing this issue. If you think there's something to do as a result of this issue, please feel free to reopen it.

Ravenwater commented 5 years ago

@jpkrohling Rerunning the schema job addressed the issue, i'm able to run our test suite and tracing is visible via the query portal. I can guarantee we're not connecting to another cassandra cluster, our application stack is constrained to a single namespace in an isolated kubernetes cluster and the service endpoints our modules are using all line up as expected.

As the schema job is idempotent converting it to a cronjob for our local use might address this if it persists for us?

Attached are logs for the re-run of the schema creation and cassandra pods:

schema rerun logs:


system_traces  system_schema  system_auth  system  system_distributed

Generating the schema for the keyspace jaeger_v1_dc1 and datacenter dc1
Using template file /cassandra-schema/v001.cql.tmpl with parameters:
    mode = prod
    datacenter = dc1
    keyspace = jaeger_v1_dc1
    replication = {'class': 'NetworkTopologyStrategy', 'dc1': '2' }
    trace_ttl = 172800
    dependencies_ttl = 0

cassandra rerun logs:

CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset (Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
CompilerOracle: dontinline org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.advanceAllocatingFrom (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V
CompilerOracle: dontinline org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents ()Z
CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stop ()V
CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stopInPartition ()V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.doFlush (I)V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeExcessSlow ()V
CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeSlow (JI)V
CompilerOracle: dontinline org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
CompilerOracle: inline org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody (Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V
CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes (Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes (JJIJ[J)V
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare (Ljava/nio/ByteBuffer;[B)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare ([BLjava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/lang/Object;JI)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt (JI)[B
INFO  [main] 2018-07-03 00:34:38,627 YamlConfigurationLoader.java:89 - Configuration location: file:/etc/cassandra/cassandra.yaml
INFO  [main] 2018-07-03 00:34:38,997 Config.java:495 - Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=10.88.235.6; broadcast_rpc_address=10.88.235.6; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=0; client_encryption_options=<REDACTED>; cluster_name=jaeger; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/var/lib/cassandra/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@3e3047e6; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_materialized_views=true; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=GossipingPropertyFileSnitch; file_cache_round_up=null; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=dc; internode_recv_buff_size_in_bytes=0; internode_send_buff_size_in_bytes=0; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=10.88.235.6; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=0; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=256; otc_backlog_expiration_interval_ms=200; otc_coalescing_enough_coalesced_messages=8; otc_coalescing_strategy=DISABLED; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/var/lib/cassandra/saved_caches; seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local,}; server_encryption_options=<REDACTED>; slow_query_log_timeout_in_ms=500; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_keep_alive_period_in_secs=300; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@37e547da; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
INFO  [main] 2018-07-03 00:34:38,998 DatabaseDescriptor.java:367 - DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
INFO  [main] 2018-07-03 00:34:38,998 DatabaseDescriptor.java:425 - Global memtable on-heap threshold is enabled at 249MB
INFO  [main] 2018-07-03 00:34:38,999 DatabaseDescriptor.java:429 - Global memtable off-heap threshold is enabled at 249MB
INFO  [main] 2018-07-03 00:34:39,127 RateBasedBackPressure.java:123 - Initialized back-pressure with high ratio: 0.9, factor: 5, flow: FAST, window size: 2000.
INFO  [main] 2018-07-03 00:34:39,128 DatabaseDescriptor.java:729 - Back-pressure is disabled with strategy org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}.
INFO  [main] 2018-07-03 00:34:39,254 GossipingPropertyFileSnitch.java:64 - Loaded cassandra-topology.properties for compatibility
INFO  [main] 2018-07-03 00:34:39,386 JMXServerUtils.java:246 - Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
INFO  [main] 2018-07-03 00:34:39,392 CassandraDaemon.java:473 - Hostname: tjb-jgr-cassandra-v6-1-cassandra-0.tjb-jgr-cassandra-v6-1-cassandra.default.svc.cluster.local
INFO  [main] 2018-07-03 00:34:39,392 CassandraDaemon.java:480 - JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_171
INFO  [main] 2018-07-03 00:34:39,393 CassandraDaemon.java:481 - Heap size: 998.438MiB/998.438MiB
INFO  [main] 2018-07-03 00:34:39,394 CassandraDaemon.java:486 - Code Cache Non-heap memory: init = 2555904(2496K) used = 4594624(4486K) committed = 4653056(4544K) max = 251658240(245760K)
INFO  [main] 2018-07-03 00:34:39,395 CassandraDaemon.java:486 - Metaspace Non-heap memory: init = 0(0K) used = 18593384(18157K) committed = 19136512(18688K) max = -1(-1K)
INFO  [main] 2018-07-03 00:34:39,396 CassandraDaemon.java:486 - Compressed Class Space Non-heap memory: init = 0(0K) used = 2241720(2189K) committed = 2359296(2304K) max = 1073741824(1048576K)
INFO  [main] 2018-07-03 00:34:39,396 CassandraDaemon.java:486 - Par Eden Space Heap memory: init = 214827008(209792K) used = 111739328(109120K) committed = 214827008(209792K) max = 214827008(209792K)
INFO  [main] 2018-07-03 00:34:39,396 CassandraDaemon.java:486 - Par Survivor Space Heap memory: init = 26804224(26176K) used = 0(0K) committed = 26804224(26176K) max = 26804224(26176K)
INFO  [main] 2018-07-03 00:34:39,397 CassandraDaemon.java:486 - CMS Old Gen Heap memory: init = 805306368(786432K) used = 0(0K) committed = 805306368(786432K) max = 805306368(786432K)
INFO  [main] 2018-07-03 00:34:39,397 CassandraDaemon.java:488 - Classpath: /etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.9.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.13.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jctools-core-1.2.1.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.2.2.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/jstackjunit-0.0.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.44.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.4.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.3.jar:/usr/share/cassandra/lib/reporter-config3-3.0.3.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.11.2.jar:/usr/share/cassandra/apache-cassandra-thrift-3.11.2.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
INFO  [main] 2018-07-03 00:34:39,398 CassandraDaemon.java:490 - JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+UseNUMA, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:+CMSClassUnloadingEnabled, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Xms1024M, -Xmx1024M, -Xmn256M, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -XX:OnOutOfMemoryError=kill -9 %p, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
WARN  [main] 2018-07-03 00:34:39,443 NativeLibrary.java:187 - Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
INFO  [main] 2018-07-03 00:34:39,444 StartupChecks.java:140 - jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
WARN  [main] 2018-07-03 00:34:39,445 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
INFO  [main] 2018-07-03 00:34:39,447 SigarLibrary.java:44 - Initializing SIGAR library
INFO  [main] 2018-07-03 00:34:39,487 SigarLibrary.java:180 - Checked OS settings and found them configured for optimal performance.
WARN  [main] 2018-07-03 00:34:39,490 StartupChecks.java:311 - Maximum number of memory map areas per process (vm.max_map_count) 65530 is too low, recommended value: 1048575, you can change it with sysctl.
WARN  [main] 2018-07-03 00:34:39,501 StartupChecks.java:332 - Directory /var/lib/cassandra/data doesn't exist
WARN  [main] 2018-07-03 00:34:39,505 StartupChecks.java:332 - Directory /var/lib/cassandra/commitlog doesn't exist
WARN  [main] 2018-07-03 00:34:39,506 StartupChecks.java:332 - Directory /var/lib/cassandra/saved_caches doesn't exist
WARN  [main] 2018-07-03 00:34:39,507 StartupChecks.java:332 - Directory /var/lib/cassandra/hints doesn't exist
INFO  [main] 2018-07-03 00:34:39,553 QueryProcessor.java:116 - Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
INFO  [main] 2018-07-03 00:34:40,049 ColumnFamilyStore.java:411 - Initializing system.IndexInfo
INFO  [main] 2018-07-03 00:34:41,205 ColumnFamilyStore.java:411 - Initializing system.batches
INFO  [main] 2018-07-03 00:34:41,218 ColumnFamilyStore.java:411 - Initializing system.paxos
INFO  [main] 2018-07-03 00:34:41,235 ColumnFamilyStore.java:411 - Initializing system.local
INFO  [main] 2018-07-03 00:34:41,245 ColumnFamilyStore.java:411 - Initializing system.peers
INFO  [main] 2018-07-03 00:34:41,281 ColumnFamilyStore.java:411 - Initializing system.peer_events
INFO  [main] 2018-07-03 00:34:41,292 ColumnFamilyStore.java:411 - Initializing system.range_xfers
INFO  [main] 2018-07-03 00:34:41,302 ColumnFamilyStore.java:411 - Initializing system.compaction_history
INFO  [main] 2018-07-03 00:34:41,310 ColumnFamilyStore.java:411 - Initializing system.sstable_activity
INFO  [main] 2018-07-03 00:34:41,319 ColumnFamilyStore.java:411 - Initializing system.size_estimates
INFO  [main] 2018-07-03 00:34:41,327 ColumnFamilyStore.java:411 - Initializing system.available_ranges
INFO  [main] 2018-07-03 00:34:41,335 ColumnFamilyStore.java:411 - Initializing system.transferred_ranges
INFO  [main] 2018-07-03 00:34:41,380 ColumnFamilyStore.java:411 - Initializing system.views_builds_in_progress
INFO  [main] 2018-07-03 00:34:41,387 ColumnFamilyStore.java:411 - Initializing system.built_views
INFO  [main] 2018-07-03 00:34:41,395 ColumnFamilyStore.java:411 - Initializing system.hints
INFO  [main] 2018-07-03 00:34:41,403 ColumnFamilyStore.java:411 - Initializing system.batchlog
INFO  [main] 2018-07-03 00:34:41,411 ColumnFamilyStore.java:411 - Initializing system.prepared_statements
INFO  [main] 2018-07-03 00:34:41,418 ColumnFamilyStore.java:411 - Initializing system.schema_keyspaces
INFO  [main] 2018-07-03 00:34:41,425 ColumnFamilyStore.java:411 - Initializing system.schema_columnfamilies
INFO  [main] 2018-07-03 00:34:41,431 ColumnFamilyStore.java:411 - Initializing system.schema_columns
INFO  [main] 2018-07-03 00:34:41,478 ColumnFamilyStore.java:411 - Initializing system.schema_triggers
INFO  [main] 2018-07-03 00:34:41,485 ColumnFamilyStore.java:411 - Initializing system.schema_usertypes
INFO  [main] 2018-07-03 00:34:41,492 ColumnFamilyStore.java:411 - Initializing system.schema_functions
INFO  [main] 2018-07-03 00:34:41,500 ColumnFamilyStore.java:411 - Initializing system.schema_aggregates
INFO  [main] 2018-07-03 00:34:41,502 ViewManager.java:137 - Not submitting build tasks for views in keyspace system as storage service is not initialized
INFO  [main] 2018-07-03 00:34:41,644 ApproximateTime.java:44 - Scheduling approximate time-check task with a precision of 10 milliseconds
INFO  [main] 2018-07-03 00:34:41,693 ColumnFamilyStore.java:411 - Initializing system_schema.keyspaces
INFO  [main] 2018-07-03 00:34:41,700 ColumnFamilyStore.java:411 - Initializing system_schema.tables
INFO  [main] 2018-07-03 00:34:41,708 ColumnFamilyStore.java:411 - Initializing system_schema.columns
INFO  [main] 2018-07-03 00:34:41,714 ColumnFamilyStore.java:411 - Initializing system_schema.triggers
INFO  [main] 2018-07-03 00:34:41,718 ColumnFamilyStore.java:411 - Initializing system_schema.dropped_columns
INFO  [main] 2018-07-03 00:34:41,724 ColumnFamilyStore.java:411 - Initializing system_schema.views
INFO  [main] 2018-07-03 00:34:41,730 ColumnFamilyStore.java:411 - Initializing system_schema.types
INFO  [main] 2018-07-03 00:34:41,736 ColumnFamilyStore.java:411 - Initializing system_schema.functions
INFO  [main] 2018-07-03 00:34:41,775 ColumnFamilyStore.java:411 - Initializing system_schema.aggregates
INFO  [main] 2018-07-03 00:34:41,780 ColumnFamilyStore.java:411 - Initializing system_schema.indexes
INFO  [main] 2018-07-03 00:34:41,782 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
INFO  [MemtableFlushWriter:1] 2018-07-03 00:34:42,255 CacheService.java:112 - Initializing key cache with capacity of 49 MBs.
INFO  [MemtableFlushWriter:1] 2018-07-03 00:34:42,278 CacheService.java:134 - Initializing row cache with capacity of 0 MBs
INFO  [MemtableFlushWriter:1] 2018-07-03 00:34:42,280 CacheService.java:163 - Initializing counter cache with capacity of 24 MBs
INFO  [MemtableFlushWriter:1] 2018-07-03 00:34:42,281 CacheService.java:174 - Scheduling counter cache save to every 7200 seconds (going to save all keys).
INFO  [CompactionExecutor:2] 2018-07-03 00:34:42,436 BufferPool.java:230 - Global buffer pool is enabled, when pool is exhausted (max is 249.000MiB) it will allocate on heap
INFO  [main] 2018-07-03 00:34:42,579 StorageService.java:600 - Populating token metadata from system tables
INFO  [main] 2018-07-03 00:34:42,679 StorageService.java:607 - Token metadata: 
INFO  [pool-3-thread-1] 2018-07-03 00:34:42,713 AutoSavingCache.java:173 - Completed loading (3 ms; 5 keys) KeyCache cache
INFO  [main] 2018-07-03 00:34:42,784 CommitLog.java:152 - No commitlog files found; skipping replay
INFO  [main] 2018-07-03 00:34:42,784 StorageService.java:600 - Populating token metadata from system tables
INFO  [main] 2018-07-03 00:34:42,803 StorageService.java:607 - Token metadata: 
INFO  [main] 2018-07-03 00:34:43,007 QueryProcessor.java:163 - Preloaded 0 prepared statements
INFO  [main] 2018-07-03 00:34:43,008 StorageService.java:618 - Cassandra version: 3.11.2
INFO  [main] 2018-07-03 00:34:43,009 StorageService.java:619 - Thrift API version: 20.1.0
INFO  [main] 2018-07-03 00:34:43,009 StorageService.java:620 - CQL supported versions: 3.4.4 (default: 3.4.4)
INFO  [main] 2018-07-03 00:34:43,009 StorageService.java:622 - Native protocol supported versions: 3/v3, 4/v4, 5/v5-beta (default: 4/v4)
INFO  [main] 2018-07-03 00:34:43,084 IndexSummaryManager.java:85 - Initializing index summary manager with a memory pool size of 49 MB and a resize interval of 60 minutes
INFO  [main] 2018-07-03 00:34:43,093 MessagingService.java:753 - Starting Messaging Service on /10.88.235.6:7000 (eth0)
WARN  [main] 2018-07-03 00:34:43,098 SystemKeyspace.java:1089 - No host ID found, created fb1c40e2-fde8-42c3-8d9a-8fd508d2aa11 (Note: This should happen exactly once per node).
INFO  [main] 2018-07-03 00:34:43,124 OutboundTcpConnection.java:108 - OutboundTcpConnection using coalescing strategy DISABLED
INFO  [ScheduledTasks:1] 2018-07-03 00:34:44,256 TokenMetadata.java:498 - Updating topology for all endpoints that have changed
WARN  [main] 2018-07-03 00:35:14,183 Gossiper.java:1425 - Unable to gossip with any seeds but continuing since node is in its own seed list
INFO  [main] 2018-07-03 00:35:14,196 StorageService.java:707 - Loading persisted ring state
INFO  [main] 2018-07-03 00:35:14,198 StorageService.java:825 - Starting up server gossip
INFO  [main] 2018-07-03 00:35:14,288 StorageService.java:886 - This node will not auto bootstrap because it is configured to be a seed node.
INFO  [main] 2018-07-03 00:35:14,296 BootStrapper.java:228 - Generated random tokens. tokens are [-6870680650703343278, 6572370962446540048, -1504837719455320503, -5382028017563693915, -2545465150634803700, -6439658940490789499, 6575835424458232436, 8083535470649689546, 4223602300437572478, 8848971726904200106, -2266266641731356254, -5428310385724190562, -729724288996789701, -5020057231017049637, 6072066902630979955, 5923902485267923854, -3906184936895020555, 5085685675538766174, -7725515542099892214, 3277850324608569319, 3979316321267905055, 5901585432028103902, -1095816206921821373, 4135601183207845313, 1649863528563204374, -1554741650151500323, 444255753869355665, -4415573716090900657, 2077154547499653799, 134967561086441325, -6282417994551828799, -7569018577197279575, 2761120086046494850, 4789428354690446081, -7638631797424143902, -6398060479225616680, 8033856685739006192, 9073633259555386173, 2367462949665707049, 5155747362459059755, -7649002513991102896, -4968692653017095224, 8506364926355402452, 3417760724261006498, -8759699832313440580, 411090285057562864, -8805737868188739345, 2605425145657562603, 6334733385808641056, 1436571410092599981, -7390038863717979609, 5000670362495106412, -4383478129381452525, -2433139199711881590, -3567885988430163442, 3638378220736132312, -2162094022283268690, 2158878511619744068, 979335586076719975, -6226940661831505536, 5257955442790935673, -2318721518388152018, -8893525963164249160, -1481195728677213962, -8599362890878503516, 3585312993351013155, 2933951062819803925, -9184581774142435294, -5600410222042340621, -8323103204641784006, 8862784504806807101, 4132688042035196704, -3472904194058857099, -645919341158437656, -765767665223347587, -265512667970605716, -4828996756170573920, 1938629349660247294, -2111144678131215481, -2330127664245109919, 3451322107061073076, 6250304337559570152, 4774362484843088270, -4022776060246313234, 7541331355727638853, -5196655511095905530, 1375230650377068207, -2390767016424286920, 2358717747333798004, 6214826695824021057, 8398256732833277240, -8365432318823466120, 1650693957143897399, -3343115667520502983, -2814842936907190710, 2631152506497751205, 5005788078702600696, 7321547173298412985, -8958657228630182701, 735965836848775627, -2576342961436681035, 7102593066952840063, 444807978842977869, 4523548465491628207, 4256241601078130796, -5153352392166088861, 7192795390418522040, 7634298181546164449, 385068247353646531, -983440795706825463, -6008640574281711729, -9194040688814824268, 2438619851784867076, 5239583167715022734, -3244627338929057243, 7910687326848836894, 2945690578285970185, -8813647284173588471, 3857936870910955213, -5779047125646165511, -1211892712192502822, 7971786578442304160, 8033146023931265896, 1385776852031012411, -7615876132757533663, 6251551898262231985, 1055568537683948817, -3205203174314057777, 5986490952836067535, 1998881795227150764, 1134960853485824554, 7696416718590671099, -3414346994320203393, 5275564778725598594, 2608895712101567033, -3928229739126567784, 3951096239103323624, 4782561279796335548, -7981010681444725056, -5089697931121206395, -6319232570718996706, -5433869618252584040, -6590587657682363352, -1700850599667601191, -329056223179467485, 4769836781501214614, 2856589563468233270, -5780540282851567827, -339914895835725639, 2707897636476368460, -6667494125070566887, -6520855094570515929, -7705643224437054482, -4365882435106307312, 4464208572424288847, 6178277762004010371, 1895711473535072856, 8530295966612357456, -7143951593342547407, -8822304065959895959, 2287915726862145642, 7582484389168862260, 4792543161758578291, -3222480121977790249, -4695240396184777723, 3889634848048307562, 5734258727453018432, 249347434421973118, -292554222231971426, -6687964269641751810, -1974694666942938254, 1061139548584466255, 1256859359627535848, 7570313478073885007, 8904761805569925378, -41057472640730227, 4359162378274390898, -5689394206176305676, -1030491030409611506, 7076119362506384911, 8635492570089474181, 1572115831099244031, -8100893251929189592, 170359131828508504, 2926678807285441268, 8062915094514259101, -6260216853442465238, -4679660356227104186, -372003947603074618, 5904058733626451575, -3980854414579209257, -6290037766108632731, 8050473996586152879, 5599126957658232095, 9026997190372239037, -4617472746929485657, -4344536208973175871, -8132338859012056627, 134784705099754992, -1797720633448623857, 644651922884750174, 7808795122165173918, 5160384076685520010, -6038354135419681514, 336916679489023257, 6660467024030296345, -4891143051649606642, 4132765739488145496, -766014417442464070, -2625489126608160422, 8280530660311984146, 6117632953423400638, -4900533663715906906, -3326299854566320760, 1482838157680013714, -5360009849785394247, 6414351229501177303, -2734087930245219027, -8251124871944347861, 3631949430427951554, 7536915598228346724, -5179085316791211524, 3463770983741190603, -3900218178649014967, 686411161915147452, -9115432218651653782, -8105603716568875082, -3922053008965414427, 8265829515889630687, -5150733343493362611, 1647809829735536528, -4433707452917108254, 7946786154917411769, -9191010950398085301, 3640958838581559529, 4016598125586493387, 6766355067880388874, -8237317078836429814, -3827185482101898775, -1556614321412380197, 4276693170471990805, 7903807822221630809, 2184418085617840912, 4734520456393162766, 3578925995508619773, 2862709204367032073, 6183159620185011823, 544726665323230255, 6593416457700437060, 236383348769178110, 7948681559297024761, 6586176182131701957, 8935438433965178566, -4159751760062486069, 309139936755466413, 1810885329992309609]
INFO  [main] 2018-07-03 00:35:14,299 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_traces, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=2}}, tables=[org.apache.cassandra.config.CFMetaData@5622ee64[cfId=c5e99f16-8677-3914-b17e-960613512345,ksName=system_traces,cfName=sessions,flags=[COMPOUND],params=TableParams{comment=tracing sessions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [client command coordinator duration request started_at parameters]],partitionKeyColumns=[session_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[client, command, session_id, coordinator, request, started_at, duration, parameters],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@7c7e6fa[cfId=8826e8e9-e16a-3728-8753-3bc1fc713c25,ksName=system_traces,cfName=events,flags=[COMPOUND],params=TableParams{comment=tracing events, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=0, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [activity source source_elapsed thread]],partitionKeyColumns=[session_id],clusteringColumns=[event_id],keyValidator=org.apache.cassandra.db.marshal.UUIDType,columnMetadata=[activity, event_id, session_id, source, thread, source_elapsed],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-03 00:35:14,619 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_traces as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-03 00:35:14,624 ColumnFamilyStore.java:411 - Initializing system_traces.events
INFO  [MigrationStage:1] 2018-07-03 00:35:14,629 ColumnFamilyStore.java:411 - Initializing system_traces.sessions
INFO  [main] 2018-07-03 00:35:14,681 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_distributed, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=3}}, tables=[org.apache.cassandra.config.CFMetaData@adf32bc[cfId=759fffad-624b-3181-80ee-fa9a52d1f627,ksName=system_distributed,cfName=repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.TimeUUIDType),partitionColumns=[[] | [coordinator exception_message exception_stacktrace finished_at parent_id range_begin range_end started_at status participants]],partitionKeyColumns=[keyspace_name, columnfamily_name],clusteringColumns=[id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[status, id, coordinator, finished_at, participants, exception_stacktrace, parent_id, range_end, range_begin, exception_message, keyspace_name, started_at, columnfamily_name],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@4587f48[cfId=deabd734-b99d-3b9c-92e5-fd92eb5abf14,ksName=system_distributed,cfName=parent_repair_history,flags=[COMPOUND],params=TableParams{comment=Repair history, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [exception_message exception_stacktrace finished_at keyspace_name started_at columnfamily_names options requested_ranges successful_ranges]],partitionKeyColumns=[parent_id],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimeUUIDType,columnMetadata=[requested_ranges, exception_message, keyspace_name, successful_ranges, started_at, finished_at, options, exception_stacktrace, parent_id, columnfamily_names],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@310ffa35[cfId=5582b59f-8e4e-35e1-b913-3acada51eb04,ksName=system_distributed,cfName=view_build_status,flags=[COMPOUND],params=TableParams{comment=Materialized View build status, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UUIDType),partitionColumns=[[] | [status]],partitionKeyColumns=[keyspace_name, view_name],clusteringColumns=[host_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[view_name, status, keyspace_name, host_id],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-03 00:35:14,784 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_distributed as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-03 00:35:14,788 ColumnFamilyStore.java:411 - Initializing system_distributed.parent_repair_history
INFO  [MigrationStage:1] 2018-07-03 00:35:14,792 ColumnFamilyStore.java:411 - Initializing system_distributed.repair_history
INFO  [MigrationStage:1] 2018-07-03 00:35:14,796 ColumnFamilyStore.java:411 - Initializing system_distributed.view_build_status
INFO  [main] 2018-07-03 00:35:14,808 StorageService.java:1449 - JOINING: Finish joining ring
INFO  [main] 2018-07-03 00:35:14,919 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=system_auth, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy, replication_factor=1}}, tables=[org.apache.cassandra.config.CFMetaData@6d31e70[cfId=5bc52802-de25-35ed-aeab-188eecebb090,ksName=system_auth,cfName=roles,flags=[COMPOUND],params=TableParams{comment=role definitions, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [can_login is_superuser salted_hash member_of]],partitionKeyColumns=[role],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[salted_hash, member_of, role, can_login, is_superuser],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@24334acc[cfId=0ecdaa87-f8fb-3e60-88d1-74fb36fe5c0d,ksName=system_auth,cfName=role_members,flags=[COMPOUND],params=TableParams{comment=role memberships lookup table, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[role],clusteringColumns=[member],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, member],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@33218686[cfId=3afbe79f-2194-31a7-add7-f5ab90d8ec9c,ksName=system_auth,cfName=role_permissions,flags=[COMPOUND],params=TableParams{comment=permissions granted to db roles, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | [permissions]],partitionKeyColumns=[role],clusteringColumns=[resource],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[role, resource, permissions],droppedColumns={},triggers=[],indexes=[]], org.apache.cassandra.config.CFMetaData@26af2967[cfId=5f2fbdad-91f1-3946-bd25-d5da3a5c35ec,ksName=system_auth,cfName=resource_role_permissons_index,flags=[COMPOUND],params=TableParams{comment=index of db roles with permissions granted on a resource, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=7776000, default_time_to_live=0, memtable_flush_period_in_ms=3600000, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[resource],clusteringColumns=[role],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[resource, role],droppedColumns={},triggers=[],indexes=[]]], views=[], functions=[], types=[]}
INFO  [MigrationStage:1] 2018-07-03 00:35:15,049 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_auth as storage service is not initialized
INFO  [MigrationStage:1] 2018-07-03 00:35:15,052 ColumnFamilyStore.java:411 - Initializing system_auth.resource_role_permissons_index
INFO  [MigrationStage:1] 2018-07-03 00:35:15,059 ColumnFamilyStore.java:411 - Initializing system_auth.role_members
INFO  [MigrationStage:1] 2018-07-03 00:35:15,064 ColumnFamilyStore.java:411 - Initializing system_auth.role_permissions
INFO  [MigrationStage:1] 2018-07-03 00:35:15,070 ColumnFamilyStore.java:411 - Initializing system_auth.roles
INFO  [main] 2018-07-03 00:35:15,084 Gossiper.java:1670 - Waiting for gossip to settle...
INFO  [main] 2018-07-03 00:35:23,085 Gossiper.java:1701 - No gossip backlog; proceeding
INFO  [main] 2018-07-03 00:35:23,312 NativeTransportService.java:70 - Netty using native Epoll event loop
INFO  [main] 2018-07-03 00:35:23,360 Server.java:155 - Using Netty Version: [netty-buffer=netty-buffer-4.0.44.Final.452812a, netty-codec=netty-codec-4.0.44.Final.452812a, netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a, netty-codec-http=netty-codec-http-4.0.44.Final.452812a, netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a, netty-common=netty-common-4.0.44.Final.452812a, netty-handler=netty-handler-4.0.44.Final.452812a, netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb, netty-transport=netty-transport-4.0.44.Final.452812a, netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a, netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a, netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a, netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]
INFO  [main] 2018-07-03 00:35:23,361 Server.java:156 - Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...
INFO  [main] 2018-07-03 00:35:23,406 CassandraDaemon.java:529 - Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it
INFO  [OptionalTasks:1] 2018-07-03 00:35:25,306 CassandraRoleManager.java:356 - Created default superuser role 'cassandra'
INFO  [IndexSummaryManager:1] 2018-07-03 01:34:43,087 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [Service Thread] 2018-07-03 01:56:42,055 GCInspector.java:284 - ConcurrentMarkSweep GC in 323ms.  CMS Old Gen: 169531128 -> 155517720; Metaspace: 42008768 -> 42008896; Par Eden Space: 75120944 -> 84673152; 
INFO  [IndexSummaryManager:1] 2018-07-03 02:34:43,096 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 03:34:43,103 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:34] 2018-07-03 04:34:42,286 AutoSavingCache.java:394 - Saved KeyCache (12 items) in 7 ms
INFO  [IndexSummaryManager:1] 2018-07-03 04:34:43,109 IndexSummaryRedistribution.java:76 - Redistributing index summaries
WARN  [PERIODIC-COMMIT-LOG-SYNCER] 2018-07-03 04:38:10,279 NoSpamLogger.java:94 - Out of 0 commit log syncs over the past 0.00s with average duration of Infinityms, 1 have exceeded the configured commit interval by an average of 3.71ms
INFO  [IndexSummaryManager:1] 2018-07-03 05:34:43,116 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 06:34:43,178 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 07:34:43,183 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:88] 2018-07-03 08:34:42,284 AutoSavingCache.java:394 - Saved KeyCache (13 items) in 5 ms
INFO  [IndexSummaryManager:1] 2018-07-03 08:34:43,187 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 09:34:43,191 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 10:34:43,195 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 11:34:43,199 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [CompactionExecutor:133] 2018-07-03 12:34:42,286 AutoSavingCache.java:394 - Saved KeyCache (14 items) in 7 ms
INFO  [IndexSummaryManager:1] 2018-07-03 12:34:43,202 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [IndexSummaryManager:1] 2018-07-03 13:34:43,208 IndexSummaryRedistribution.java:76 - Redistributing index summaries
INFO  [Native-Transport-Requests-12] 2018-07-03 14:24:24,400 MigrationManager.java:331 - Create new Keyspace: KeyspaceMetadata{name=jaeger_v1_dc1, params=KeyspaceParams{durable_writes=true, replication=ReplicationParams{class=org.apache.cassandra.locator.NetworkTopologyStrategy, dc1=2}}, tables=[], views=[], functions=[], types=[]}
INFO  [Native-Transport-Requests-1] 2018-07-03 14:24:24,728 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@1d54107f[cfId=c91bca70-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=traces,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.LongType, org.apache.cassandra.db.marshal.LongType),partitionColumns=[[] | [duration flags operation_name parent_id process start_time logs refs tags]],partitionKeyColumns=[trace_id],clusteringColumns=[span_id, span_hash],keyValidator=org.apache.cassandra.db.marshal.BytesType,columnMetadata=[logs, span_hash, refs, start_time, span_id, process, duration, tags, operation_name, trace_id, flags, parent_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:24,830 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.traces
INFO  [Native-Transport-Requests-2] 2018-07-03 14:24:24,896 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@6afe8f7b[cfId=c9359400-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=service_names,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | []],partitionKeyColumns=[service_name],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[service_name],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:24,974 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_names
INFO  [Native-Transport-Requests-1] 2018-07-03 14:24:24,998 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@571eb2fa[cfId=c9452460-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=operation_names,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[] | []],partitionKeyColumns=[service_name],clusteringColumns=[operation_name],keyValidator=org.apache.cassandra.db.marshal.UTF8Type,columnMetadata=[service_name, operation_name],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,083 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.operation_names
INFO  [Native-Transport-Requests-6] 2018-07-03 14:24:25,105 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@3302dee0[cfId=c9557810-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=service_operation_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType)),partitionColumns=[[] | [trace_id]],partitionKeyColumns=[service_name, operation_name],clusteringColumns=[start_time],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[start_time, service_name, operation_name, trace_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,202 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_operation_index
INFO  [Native-Transport-Requests-1] 2018-07-03 14:24:25,224 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@561e3f2[cfId=c967a080-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=service_name_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType)),partitionColumns=[[] | [trace_id]],partitionKeyColumns=[service_name, bucket],clusteringColumns=[start_time],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.Int32Type),columnMetadata=[start_time, service_name, bucket, trace_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,328 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.service_name_index
INFO  [Native-Transport-Requests-3] 2018-07-03 14:24:25,352 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@2b9e5be8[cfId=c97b2880-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=duration_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.BytesType),partitionColumns=[[] | []],partitionKeyColumns=[service_name, operation_name, bucket],clusteringColumns=[duration, start_time, trace_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.TimestampType),columnMetadata=[duration, start_time, service_name, operation_name, bucket, trace_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,431 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.duration_index
INFO  [Native-Transport-Requests-4] 2018-07-03 14:24:25,449 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@50e7ec41[cfId=c989f590-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=tag_index,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.0, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=10800, default_time_to_live=172800, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=NONE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy, options={min_threshold=4, max_threshold=32, compaction_window_size=1, compaction_window_unit=HOURS}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.LongType), org.apache.cassandra.db.marshal.BytesType, org.apache.cassandra.db.marshal.LongType),partitionColumns=[[] | []],partitionKeyColumns=[service_name, tag_key, tag_value],clusteringColumns=[start_time, trace_id, span_id],keyValidator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UTF8Type),columnMetadata=[start_time, service_name, tag_key, span_id, tag_value, trace_id],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,536 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.tag_index
INFO  [Native-Transport-Requests-4] 2018-07-03 14:24:25,621 MigrationManager.java:376 - Create new table: org.apache.cassandra.config.CFMetaData@671428ae[cfId=c9a43450-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[]]
INFO  [MigrationStage:1] 2018-07-03 14:24:25,720 ColumnFamilyStore.java:411 - Initializing jaeger_v1_dc1.dependencies
INFO  [Native-Transport-Requests-5] 2018-07-03 14:24:25,786 MigrationManager.java:454 - Update table 'jaeger_v1_dc1/dependencies' From org.apache.cassandra.config.CFMetaData@675ac9cf[cfId=c9a43450-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[]] To org.apache.cassandra.config.CFMetaData@344818fc[cfId=c9a43450-7ecc-11e8-89a0-b71a2a7bab25,ksName=jaeger_v1_dc1,cfName=dependencies,flags=[COMPOUND],params=TableParams{comment=, read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0, gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128, max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition' : 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy, options={max_threshold=32, min_threshold=4}}, compression=org.apache.cassandra.schema.CompressionParams@65ad3eab, extensions={}, cdc=false},comparator=comparator(),partitionColumns=[[] | [ts_index dependencies]],partitionKeyColumns=[ts],clusteringColumns=[],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ts_index, ts, dependencies],droppedColumns={},triggers=[],indexes=[org.apache.cassandra.schema.IndexMetadata@637cc530[id=e756441e-bae6-3c1f-8e50-a5087c4f2d55,name=dependencies_ts_index_idx,kind=CUSTOM,options={mode=SPARSE, class_name=org.apache.cassandra.index.sasi.SASIIndex, target=ts_index}]]]

@tjb3 how did you rerun that job?

jpkrohling commented 5 years ago

@Ravenwater, you can re-create the Kubernetes job. It should have no bad side effects, as the script is meant to be idempotent.

Ravenwater commented 5 years ago

@jpkrohling I deployed the full script multiple times but it continued to fail the same way. What I ended up doing is first run the schema creation job manually, so the schema was present in the cluster, and then deploy the full script. That seemed to have solved the race condition.