uber / cadence

Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.
https://cadenceworkflow.io
MIT License
8.23k stars 792 forks source link

Issue starting up #3054

Closed darewreck54 closed 4 years ago

darewreck54 commented 4 years ago

I'm trying to run the example setup of cadence, I'm running into a lot of issues. I can't even bring up the front end at localhost: http://localhost:8088/

docker-compose up
Starting cadence_cassandra_1 ... done
Starting cadence_statsd_1    ... done
Starting cadence_cadence_1   ... done
Starting cadence_cadence-web_1 ... done
Attaching to cadence_cassandra_1, cadence_statsd_1, cadence_cadence_1, cadence_cadence-web_1
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.deserializeLargeSubset (Lorg/apache/cassandra/io/util/DataInputPlus;Lorg/apache/cassandra/db/Columns;I)Lorg/apache/cassandra/db/Columns;
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubset (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;ILorg/apache/cassandra/io/util/DataOutputPlus;)V
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/db/Columns$Serializer.serializeLargeSubsetSize (Ljava/util/Collection;ILorg/apache/cassandra/db/Columns;I)I
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/db/commitlog/AbstractCommitLogSegmentManager.advanceAllocatingFrom (Lorg/apache/cassandra/db/commitlog/CommitLogSegment;)V
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/db/transform/BaseIterator.tryGetMoreContents ()Z
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stop ()V
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/db/transform/StoppingTransformation.stopInPartition ()V
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.doFlush (I)V
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeExcessSlow ()V
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/io/util/BufferedDataOutputStreamPlus.writeSlow (JI)V
cadence_1      | + DB=cassandra
cadence_1      | + ENABLE_ES=false
cadence_1      | + ES_PORT=9200
cadence_1      | + RF=1
cadence_1      | + export KEYSPACE=cadence
cadence_1      | + KEYSPACE=cadence
cadence_1      | + export VISIBILITY_KEYSPACE=cadence_visibility
cadence_1      | + VISIBILITY_KEYSPACE=cadence_visibility
cadence_1      | + export DBNAME=cadence
cadence_1      | + DBNAME=cadence
cadence_1      | + export VISIBILITY_DBNAME=cadence_visibility
cadence_1      | + VISIBILITY_DBNAME=cadence_visibility
cadence_1      | + export DB_PORT=3306
cadence_1      | + DB_PORT=3306
cadence_1      | + wait_for_db
cadence_1      | + '[' cassandra == mysql ']'
cadence_1      | + '[' cassandra == postgres ']'
cadence_1      | + wait_for_cassandra
statsd_1       | Started runsvdir, PID is 40
statsd_1       | wait for processes to start....
cassandra_1    | CompilerOracle: dontinline org/apache/cassandra/io/util/RebufferingInputStream.readPrimitiveSlowly (I)J
cassandra_1    | CompilerOracle: inline org/apache/cassandra/db/rows/UnfilteredSerializer.serializeRowBody (Lorg/apache/cassandra/db/rows/Row;ILorg/apache/cassandra/db/SerializationHeader;Lorg/apache/cassandra/io/util/DataOutputPlus;)V
cassandra_1    | CompilerOracle: inline org/apache/cassandra/io/util/Memory.checkBounds (JJ)V
cassandra_1    | CompilerOracle: inline org/apache/cassandra/io/util/SafeMemory.checkBounds (JJ)V
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.selectBoundary (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;II)I
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/AsymmetricOrdering.strictnessOfLessThan (Lorg/apache/cassandra/utils/AsymmetricOrdering/Op;)I
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.indexes (Lorg/apache/cassandra/utils/IFilter/FilterKey;)[J
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/BloomFilter.setIndexes (JJIJ[J)V
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare (Ljava/nio/ByteBuffer;[B)I
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compare ([BLjava/nio/ByteBuffer;)I
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/ByteBufferUtil.compareUnsigned (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/lang/Object;JI)I
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/lang/Object;JILjava/nio/ByteBuffer;)I
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/FastByteOperations$UnsafeOperations.compareTo (Ljava/nio/ByteBuffer;Ljava/nio/ByteBuffer;)I
cassandra_1    | CompilerOracle: inline org/apache/cassandra/utils/vint/VIntCoding.encodeVInt (JI)[B
statsd_1       | /usr/sbin/crond 4.5 dillon's cron daemon, started with loglevel notice
statsd_1       | ok: run: nginx: (pid 53) 0s
statsd_1       | 15 Feb 16:24:33 - [66] reading config file: /opt/statsd/config/udp.js
cadence_1      | ++ echo cassandra
statsd_1       | 15 Feb 16:24:33 - server is up INFO
statsd_1       | Starting carbon-cache (instance a)
statsd_1       | 15/02/2020 16:24:34 :: [console] Using sorted write strategy for cache
statsd_1       | 15/02/2020 16:24:34 :: [console] twistd 19.7.0 (/opt/graphite/bin/python3 3.7.5) starting up.
statsd_1       | 15/02/2020 16:24:34 :: [console] reactor class: twisted.internet.epollreactor.EPollReactor.
statsd_1       | 15/02/2020 16:24:34 :: [console] ServerFactory starting on 7002
cadence_1      | ++ awk -F , '{print $1}'
cadence_1      | + server=cassandra
statsd_1       | 15/02/2020 16:24:34 :: [console] Starting factory <twisted.internet.protocol.ServerFactory object at 0x7f1a01d9bb90>
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
statsd_1       | 15/02/2020 16:24:34 :: [console] Installing SIG_IGN for SIGHUP
statsd_1       | 15/02/2020 16:24:34 :: [console] CarbonReceiverFactory starting on 2003
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
statsd_1       | 15/02/2020 16:24:34 :: [console] Starting factory <carbon.protocols.CarbonReceiverFactory object at 0x7f1a01c38950>
cadence_1      | + echo 'waiting for cassandra to start up'
statsd_1       | Starting carbon-aggregator (instance a)
statsd_1       | 15/02/2020 16:24:34 :: [console] Using sorted write strategy for cache
statsd_1       | 15/02/2020 16:24:34 :: [aggregator] reading new aggregation rules from /opt/graphite/conf/aggregation-rules.conf
statsd_1       | 15/02/2020 16:24:34 :: [aggregator] clearing aggregation buffers
statsd_1       | 15/02/2020 16:24:34 :: [clients] connecting to carbon daemon at 127.0.0.1:2004:None
statsd_1       | 15/02/2020 16:24:34 :: [console] twistd 19.7.0 (/opt/graphite/bin/python3 3.7.5) starting up.
statsd_1       | 15/02/2020 16:24:34 :: [console] reactor class: twisted.internet.epollreactor.EPollReactor.
cadence_1      | waiting for cassandra to start up
statsd_1       | 15/02/2020 16:24:34 :: [console] CarbonReceiverFactory starting on 2004
statsd_1       | 15/02/2020 16:24:34 :: [console] Starting factory <carbon.protocols.CarbonReceiverFactory object at 0x7f1a01c389d0>
cadence_1      | + sleep 1
statsd_1       | 15/02/2020 16:24:34 :: [console] Installing SIG_IGN for SIGHUP
statsd_1       | 15/02/2020 16:24:34 :: [console] Starting factory CarbonClientFactory(127.0.0.1:2004:None)
statsd_1       | 15/02/2020 16:24:34 :: [clients] CarbonClientFactory(127.0.0.1:2004:None)::startedConnecting (127.0.0.1:2004)
statsd_1       | 15/02/2020 16:24:34 :: [console] CarbonReceiverFactory starting on 2023
statsd_1       | 15/02/2020 16:24:34 :: [console] Starting factory <carbon.protocols.CarbonReceiverFactory object at 0x7f11113e8210>
statsd_1       | 15/02/2020 16:24:34 :: [console] CarbonReceiverFactory starting on 2024
statsd_1       | 15/02/2020 16:24:34 :: [console] Starting factory <carbon.protocols.CarbonReceiverFactory object at 0x7f111150d2d0>
statsd_1       | 15/02/2020 16:24:34 :: [clients] CarbonClientProtocol(127.0.0.1:2004:None)::connectionMade
statsd_1       | 15/02/2020 16:24:34 :: [clients] CarbonClientFactory(127.0.0.1:2004:None)::connectionMade (CarbonClientProtocol(127.0.0.1:2004:None))
statsd_1       | 15/02/2020 16:24:34 :: [clients] Destination is up: 127.0.0.1:2004:None
statsd_1       | 15/02/2020 16:24:34 :: [listener] MetricPickleReceiver connection with 127.0.0.1:33898 established
cassandra_1    | INFO  [main] 2020-02-15 16:24:35,370 YamlConfigurationLoader.java:89 - Configuration location: file:/etc/cassandra/cassandra.yaml
cassandra_1    | INFO  [main] 2020-02-15 16:24:35,830 Config.java:516 - Node configuration:[allocate_tokens_for_keyspace=null; authenticator=AllowAllAuthenticator; authorizer=AllowAllAuthorizer; auto_bootstrap=true; auto_snapshot=true; back_pressure_enabled=false; back_pressure_strategy=org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}; batch_size_fail_threshold_in_kb=50; batch_size_warn_threshold_in_kb=5; batchlog_replay_throttle_in_kb=1024; broadcast_address=172.18.0.3; broadcast_rpc_address=172.18.0.3; buffer_pool_use_heap_if_exhausted=true; cas_contention_timeout_in_ms=1000; cdc_enabled=false; cdc_free_space_check_interval_ms=250; cdc_raw_directory=null; cdc_total_space_in_mb=0; client_encryption_options=<REDACTED>; cluster_name=Test Cluster; column_index_cache_size_in_kb=2; column_index_size_in_kb=64; commit_failure_policy=stop; commitlog_compression=null; commitlog_directory=/var/lib/cassandra/commitlog; commitlog_max_compression_buffers_in_pool=3; commitlog_periodic_queue_size=-1; commitlog_segment_size_in_mb=32; commitlog_sync=periodic; commitlog_sync_batch_window_in_ms=NaN; commitlog_sync_period_in_ms=10000; commitlog_total_space_in_mb=null; compaction_large_partition_warning_threshold_mb=100; compaction_throughput_mb_per_sec=16; concurrent_compactors=null; concurrent_counter_writes=32; concurrent_materialized_view_writes=32; concurrent_reads=32; concurrent_replicates=null; concurrent_writes=32; counter_cache_keys_to_save=2147483647; counter_cache_save_period=7200; counter_cache_size_in_mb=null; counter_write_request_timeout_in_ms=5000; credentials_cache_max_entries=1000; credentials_update_interval_in_ms=-1; credentials_validity_in_ms=2000; cross_node_timeout=false; data_file_directories=[Ljava.lang.String;@475e586c; disk_access_mode=auto; disk_failure_policy=stop; disk_optimization_estimate_percentile=0.95; disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd; dynamic_snitch=true; dynamic_snitch_badness_threshold=0.1; dynamic_snitch_reset_interval_in_ms=600000; dynamic_snitch_update_interval_in_ms=100; enable_materialized_views=true; enable_sasi_indexes=true; enable_scripted_user_defined_functions=false; enable_user_defined_functions=false; enable_user_defined_functions_threads=true; encryption_options=null; endpoint_snitch=SimpleSnitch; file_cache_round_up=null; file_cache_size_in_mb=null; gc_log_threshold_in_ms=200; gc_warn_threshold_in_ms=1000; hinted_handoff_disabled_datacenters=[]; hinted_handoff_enabled=true; hinted_handoff_throttle_in_kb=1024; hints_compression=null; hints_directory=null; hints_flush_period_in_ms=10000; incremental_backups=false; index_interval=null; index_summary_capacity_in_mb=null; index_summary_resize_interval_in_minutes=60; initial_token=null; inter_dc_stream_throughput_outbound_megabits_per_sec=200; inter_dc_tcp_nodelay=false; internode_authenticator=null; internode_compression=dc; internode_recv_buff_size_in_bytes=0; internode_send_buff_size_in_bytes=0; key_cache_keys_to_save=2147483647; key_cache_save_period=14400; key_cache_size_in_mb=null; listen_address=172.18.0.3; listen_interface=null; listen_interface_prefer_ipv6=false; listen_on_broadcast_address=false; max_hint_window_in_ms=10800000; max_hints_delivery_threads=2; max_hints_file_size_in_mb=128; max_mutation_size_in_kb=null; max_streaming_retries=3; max_value_size_in_mb=256; memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null; memtable_flush_writers=0; memtable_heap_space_in_mb=null; memtable_offheap_space_in_mb=null; min_free_space_per_drive_in_mb=50; native_transport_flush_in_batches_legacy=true; native_transport_max_concurrent_connections=-1; native_transport_max_concurrent_connections_per_ip=-1; native_transport_max_concurrent_requests_in_bytes=-1; native_transport_max_concurrent_requests_in_bytes_per_ip=-1; native_transport_max_frame_size_in_mb=256; native_transport_max_negotiable_protocol_version=-2147483648; native_transport_max_threads=128; native_transport_port=9042; native_transport_port_ssl=null; num_tokens=256; otc_backlog_expiration_interval_ms=200; otc_coalescing_enough_coalesced_messages=8; otc_coalescing_strategy=DISABLED; otc_coalescing_window_us=200; partitioner=org.apache.cassandra.dht.Murmur3Partitioner; permissions_cache_max_entries=1000; permissions_update_interval_in_ms=-1; permissions_validity_in_ms=2000; phi_convict_threshold=8.0; prepared_statements_cache_size_mb=null; range_request_timeout_in_ms=10000; read_request_timeout_in_ms=5000; repair_session_max_tree_depth=18; request_scheduler=org.apache.cassandra.scheduler.NoScheduler; request_scheduler_id=null; request_scheduler_options=null; request_timeout_in_ms=10000; role_manager=CassandraRoleManager; roles_cache_max_entries=1000; roles_update_interval_in_ms=-1; roles_validity_in_ms=2000; row_cache_class_name=org.apache.cassandra.cache.OHCProvider; row_cache_keys_to_save=2147483647; row_cache_save_period=0; row_cache_size_in_mb=0; rpc_address=0.0.0.0; rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true; rpc_listen_backlog=50; rpc_max_threads=2147483647; rpc_min_threads=16; rpc_port=9160; rpc_recv_buff_size_in_bytes=null; rpc_send_buff_size_in_bytes=null; rpc_server_type=sync; saved_caches_directory=/var/lib/cassandra/saved_caches; seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=172.18.0.3}; server_encryption_options=<REDACTED>; slow_query_log_timeout_in_ms=500; snapshot_before_compaction=false; ssl_storage_port=7001; sstable_preemptive_open_interval_in_mb=50; start_native_transport=true; start_rpc=false; storage_port=7000; stream_throughput_outbound_megabits_per_sec=200; streaming_keep_alive_period_in_secs=300; streaming_socket_timeout_in_ms=86400000; thrift_framed_transport_size_in_mb=15; thrift_max_message_length_in_mb=16; thrift_prepared_statements_cache_size_mb=null; tombstone_failure_threshold=100000; tombstone_warn_threshold=1000; tracetype_query_ttl=86400; tracetype_repair_ttl=604800; transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@657c8ad9; trickle_fsync=false; trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=60000; unlogged_batch_across_partitions_warn_threshold=10; user_defined_function_fail_timeout=1500; user_defined_function_warn_timeout=500; user_function_timeout_policy=die; windows_timer_interval=1; write_request_timeout_in_ms=2000]
cassandra_1    | INFO  [main] 2020-02-15 16:24:35,831 DatabaseDescriptor.java:381 - DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap
cassandra_1    | INFO  [main] 2020-02-15 16:24:35,832 DatabaseDescriptor.java:439 - Global memtable on-heap threshold is enabled at 243MB
cassandra_1    | INFO  [main] 2020-02-15 16:24:35,832 DatabaseDescriptor.java:443 - Global memtable off-heap threshold is enabled at 243MB
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cassandra_1    | WARN  [main] 2020-02-15 16:24:36,033 DatabaseDescriptor.java:579 - Only 4.261GiB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,090 RateBasedBackPressure.java:123 - Initialized back-pressure with high ratio: 0.9, factor: 5, flow: FAST, window size: 2000.
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,090 DatabaseDescriptor.java:773 - Back-pressure is disabled with strategy org.apache.cassandra.net.RateBasedBackPressure{high_ratio=0.9, factor=5, flow=FAST}.
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | waiting for cassandra to start up
cadence_1      | + sleep 1
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,331 JMXServerUtils.java:246 - Configured JMX server at: service:jmx:rmi://127.0.0.1/jndi/rmi://127.0.0.1:7199/jmxrmi
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,352 CassandraDaemon.java:476 - Hostname: b3def02c66c2
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,353 CassandraDaemon.java:483 - JVM vendor/version: OpenJDK 64-Bit Server VM/1.8.0_232
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,354 CassandraDaemon.java:484 - Heap size: 975.125MiB/975.125MiB
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,356 CassandraDaemon.java:489 - Code Cache Non-heap memory: init = 2555904(2496K) used = 4440256(4336K) committed = 4456448(4352K) max = 251658240(245760K)
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,356 CassandraDaemon.java:489 - Metaspace Non-heap memory: init = 0(0K) used = 17735496(17319K) committed = 18350080(17920K) max = -1(-1K)
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,357 CassandraDaemon.java:489 - Compressed Class Space Non-heap memory: init = 0(0K) used = 2133296(2083K) committed = 2359296(2304K) max = 1073741824(1048576K)
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,361 CassandraDaemon.java:489 - Par Eden Space Heap memory: init = 208928768(204032K) used = 100308368(97957K) committed = 208928768(204032K) max = 208928768(204032K)
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,363 CassandraDaemon.java:489 - Par Survivor Space Heap memory: init = 26083328(25472K) used = 0(0K) committed = 26083328(25472K) max = 26083328(25472K)
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,363 CassandraDaemon.java:489 - CMS Old Gen Heap memory: init = 787480576(769024K) used = 0(0K) committed = 787480576(769024K) max = 787480576(769024K)
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,364 CassandraDaemon.java:491 - Classpath: /etc/cassandra:/usr/share/cassandra/lib/HdrHistogram-2.1.9.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/asm-5.0.4.jar:/usr/share/cassandra/lib/caffeine-2.2.6.jar:/usr/share/cassandra/lib/cassandra-driver-core-3.0.1-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.9.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrent-trees-2.4.0.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-18.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/hppc-0.5.4.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.13.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.13.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jctools-core-1.2.1.jar:/usr/share/cassandra/lib/jflex-1.6.0.jar:/usr/share/cassandra/lib/jna-4.2.2.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/jstackjunit-0.0.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.5.jar:/usr/share/cassandra/lib/metrics-jvm-3.1.5.jar:/usr/share/cassandra/lib/metrics-logback-3.1.5.jar:/usr/share/cassandra/lib/netty-all-4.0.44.Final.jar:/usr/share/cassandra/lib/ohc-core-0.4.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.4.4.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.3.jar:/usr/share/cassandra/lib/reporter-config3-3.0.3.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/snowball-stemmer-1.3.0.581.1.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-3.11.5.jar:/usr/share/cassandra/apache-cassandra-thrift-3.11.5.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,365 CassandraDaemon.java:493 - JVM Arguments: [-Xloggc:/var/log/cassandra/gc.log, -ea, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+AlwaysPreTouch, -XX:-UseBiasedLocking, -XX:+UseTLAB, -XX:+ResizeTLAB, -XX:+UseNUMA, -XX:+PerfDisableSharedMem, -Djava.net.preferIPv4Stack=true, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:+CMSClassUnloadingEnabled, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, -Xms999M, -Xmx999M, -Xmn249M, -XX:+UseCondCardMark, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -Dcassandra.jmx.local.port=7199, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.password.file=/etc/cassandra/jmxremote.password, -Djava.library.path=/usr/share/cassandra/lib/sigar-bin, -Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, -XX:OnOutOfMemoryError=kill -9 %p, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=/var/lib/cassandra, -Dcassandra-foreground=yes]
cassandra_1    | WARN  [main] 2020-02-15 16:24:36,469 NativeLibrary.java:189 - Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,470 StartupChecks.java:140 - jemalloc seems to be preloaded from /usr/lib/x86_64-linux-gnu/libjemalloc.so.1
cassandra_1    | WARN  [main] 2020-02-15 16:24:36,471 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,478 SigarLibrary.java:44 - Initializing SIGAR library
cassandra_1    | WARN  [main] 2020-02-15 16:24:36,516 SigarLibrary.java:174 - Cassandra server running in degraded mode. Is swap disabled? : false,  Address space adequate? : true,  nofile limit adequate? : true, nproc limit adequate? : true 
cassandra_1    | WARN  [main] 2020-02-15 16:24:36,524 StartupChecks.java:311 - Maximum number of memory map areas per process (vm.max_map_count) 262144 is too low, recommended value: 1048575, you can change it with sysctl.
cassandra_1    | INFO  [main] 2020-02-15 16:24:36,742 QueryProcessor.java:116 - Initialized prepared statement caches with 10 MB (native) and 10 MB (Thrift)
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence-web_1  | cadence-web up and listening on port 8088
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | waiting for cassandra to start up
cadence_1      | + sleep 1
cassandra_1    | INFO  [main] 2020-02-15 16:24:37,488 ColumnFamilyStore.java:426 - Initializing system.IndexInfo
statsd_1       | run: carbon: (pid 62) 5s; run: log: (pid 58) 5s
statsd_1       | run: carbon-aggregator: (pid 61) 5s; run: log: (pid 59) 5s
statsd_1       | down: carbon-relay: 1s, normally up, want up; run: log: (pid 55) 5s
statsd_1       | down: collectd: 1s, normally up, want up
statsd_1       | run: cron: (pid 51) 5s
statsd_1       | run: graphite: (pid 56) 5s
statsd_1       | run: nginx: (pid 53) 5s
statsd_1       | down: redis: 1s, normally up, want up
statsd_1       | run: statsd: (pid 66) 5s; run: log: (pid 65) 5s
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cassandra_1    | INFO  [SSTableBatchOpen:1] 2020-02-15 16:24:38,427 BufferPool.java:234 - Global buffer pool is enabled, when pool is exhausted (max is 243.000MiB) it will allocate on heap
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | waiting for cassandra to start up
cadence_1      | + sleep 1
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,728 CacheService.java:100 - Initializing key cache with capacity of 48 MBs.
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,742 CacheService.java:122 - Initializing row cache with capacity of 0 MBs
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,744 CacheService.java:151 - Initializing counter cache with capacity of 24 MBs
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,746 CacheService.java:162 - Scheduling counter cache save to every 7200 seconds (going to save all keys).
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,802 ColumnFamilyStore.java:426 - Initializing system.batches
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,815 ColumnFamilyStore.java:426 - Initializing system.paxos
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,838 ColumnFamilyStore.java:426 - Initializing system.local
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,860 ColumnFamilyStore.java:426 - Initializing system.peers
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,874 ColumnFamilyStore.java:426 - Initializing system.peer_events
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,884 ColumnFamilyStore.java:426 - Initializing system.range_xfers
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,897 ColumnFamilyStore.java:426 - Initializing system.compaction_history
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,932 ColumnFamilyStore.java:426 - Initializing system.sstable_activity
cassandra_1    | INFO  [main] 2020-02-15 16:24:38,967 ColumnFamilyStore.java:426 - Initializing system.size_estimates
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,008 ColumnFamilyStore.java:426 - Initializing system.available_ranges
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,020 ColumnFamilyStore.java:426 - Initializing system.transferred_ranges
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,033 ColumnFamilyStore.java:426 - Initializing system.views_builds_in_progress
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,042 ColumnFamilyStore.java:426 - Initializing system.built_views
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,052 ColumnFamilyStore.java:426 - Initializing system.hints
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,060 ColumnFamilyStore.java:426 - Initializing system.batchlog
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,070 ColumnFamilyStore.java:426 - Initializing system.prepared_statements
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,099 ColumnFamilyStore.java:426 - Initializing system.schema_keyspaces
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,106 ColumnFamilyStore.java:426 - Initializing system.schema_columnfamilies
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,113 ColumnFamilyStore.java:426 - Initializing system.schema_columns
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,124 ColumnFamilyStore.java:426 - Initializing system.schema_triggers
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,133 ColumnFamilyStore.java:426 - Initializing system.schema_usertypes
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,140 ColumnFamilyStore.java:426 - Initializing system.schema_functions
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,149 ColumnFamilyStore.java:426 - Initializing system.schema_aggregates
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,152 ViewManager.java:137 - Not submitting build tasks for views in keyspace system as storage service is not initialized
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,348 ApproximateTime.java:44 - Scheduling approximate time-check task with a precision of 10 milliseconds
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,419 ColumnFamilyStore.java:426 - Initializing system_schema.keyspaces
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,441 ColumnFamilyStore.java:426 - Initializing system_schema.tables
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,470 ColumnFamilyStore.java:426 - Initializing system_schema.columns
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,498 ColumnFamilyStore.java:426 - Initializing system_schema.triggers
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,527 ColumnFamilyStore.java:426 - Initializing system_schema.dropped_columns
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,558 ColumnFamilyStore.java:426 - Initializing system_schema.views
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,577 ColumnFamilyStore.java:426 - Initializing system_schema.types
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,598 ColumnFamilyStore.java:426 - Initializing system_schema.functions
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,624 ColumnFamilyStore.java:426 - Initializing system_schema.aggregates
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,654 ColumnFamilyStore.java:426 - Initializing system_schema.indexes
cassandra_1    | INFO  [main] 2020-02-15 16:24:39,695 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_schema as storage service is not initialized
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | + sleep waiting for cassandra to start up
cadence_1      | 1
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,289 StorageService.java:638 - Populating token metadata from system tables
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,319 StorageService.java:645 - Token metadata: Normal Tokens:
cassandra_1    | /172.18.0.3:[-9164505853937689668, -9140318971186185580, -9113544450604874218, -9063052614749912161, -8960219402619035128, -8775333052802257261, -8732481292028470573, -8672290831330467624, -8647678848494905870, -8601228240488988801, -8407357627008425162, -8333376979492566840, -8269227312635425943, -8241629939187331870, -8131494450422448977, -8093467086603882915, -8064670819569502621, -8037179657605678693, -7915307951670520152, -7819664151987039467, -7807391123248037087, -7794392214244124494, -7744586738941112295, -7689538238274030898, -7624189110584063103, -7580987871126460387, -7551529169608191124, -7518812517876412938, -7513034692635767129, -7460463321159994436, -7265917373683037848, -7167646315693704387, -7128574870874449558, -7112185752998405676, -7108335897869054872, -7043283542143482933, -6931920376067673760, -6850487622590697124, -6848057624744661314, -6787942147181972240, -6738576824798558073, -6669289474079636931, -6659651766067167842, -6641653338979523154, -6625543461268037061, -6540139677528492827, -6485330870429999111, -6422228688080285973, -6279145031513832019, -6157985448058849541, -6117424707478575481, -6091318709497135599, -5979377252614877957, -5949999559744236145, -5738530099754037639, -5678073753528077357, -5606442783710815751, -5586210984850748016, -5402715365714680064, -5387300360288048634, -5319259222778688110, -5166118427843740546, -5151913291375316541, -5082184762098550075, -4891330247071776745, -4877688828405231921, -4834626391101183414, -4788573060565038043, -4605340572543162783, -4507798339860963515, -4502590621395695921, -4348711382592552266, -4344078053233129829, -4281421617778802173, -4210189800785889618, -4132502096828440723, -4066662485146739147, -4037494794877747783, -3973901891367242733, -3950778251935092437, -3900529089252107559, -3882927904761471179, -3860378533311551319, -3827490800986007579, -3748361340219759991, -3709025987000141661, -3696142979770626893, -3638467115236842692, -3625190520080778439, -3603191596077653159, -3592298183244133524, -3548231463808235537, -3502950843870859309, -3374002116569215697, -3319802243351440129, -3305994981179573769, -3194475623924041768, -3167693286331532765, -3160900603694963667, -3004015093682878874, -2867024030577870428, -2832285631284234669, -2680365836874694144, -2539460863408039879, -2385663986431974251, -2169003199888325664, -2131830906011685899, -2098112509078042619, -2020473546827131176, -2007907830683954630, -1930523114060706297, -1878385461611569093, -1643245496486755343, -1601862469295823337, -1600700365271420645, -1594880435622775370, -1517208080670188132, -1512808086813952144, -1445418993608927592, -1351365771990309449, -1344132880087562245, -1257484572307285847, -1189912828349020120, -1166981555587057035, -1085001762442193647, -1059213728947193721, -956741944123895246, -873772841104376178, -810181171196319017, -810043320620128106, -774018532725708709, -707303678989184333, -618320948996830110, -612231160131486041, -551269649407636870, -431386015694620417, -364120952620031739, -308389446835905727, -307077733436199656, -253149616125815298, -195777931336518879, -189503550750703012, -179254317581871936, -135340410611787902, -94198049482137722, 34374212114722049, 127188817389878050, 132965324808210526, 303622052456337254, 329858286239927786, 366018486534491906, 406638247841567057, 479851687137388556, 670026697430376075, 752710940507410501, 767407488769810792, 851060453052476895, 868493320096444384, 936781260251079188, 1022176325614192444, 1230697997356220952, 1292957465069455562, 1325119940892912351, 1334985843885371214, 1343108093999566216, 1361406166962715214, 1378863510406235771, 1400333617546998134, 1582807585029767367, 1597215431626030651, 1749477762232355436, 1792276602923579968, 1922474206199790498, 1988052706172858418, 2127317537841102726, 2259274909912667694, 2397671054589908406, 2429099292771470560, 2533786835888428493, 2578246049355406059, 2686298223326262725, 2701717933442637254, 2807760787041561577, 2884214447668329263, 3157316500484678162, 3187716919969840056, 3270860775362644337, 3303610804117999142, 3391705420326532576, 3491091462970439509, 3706100641950710733, 3724805958006403234, 3757364129542921818, 3806036858066157266, 3895832271172542984, 3973536966711831758, 4030747908146128519, 4123902827963866009, 4160568841072448445, 4228519987033230575, 4257747407932998692, 4273551570363887293, 4274043940588997505, 4372472864976210952, 4611319075408017514, 4645600139360069230, 4750664360884816095, 4945720248950484541, 4951369330675607518, 4991207015817107215, 5076741404806976287, 5084631119909752433, 5402558636390072478, 5462611429300242125, 5553289478482844262, 5608290272328210330, 5744662837408377582, 5760902679518379616, 5848812504429212946, 5869020964144644718, 5914929288048126036, 5973397761359376924, 6031927103805483694, 6055222436683253778, 6081442801756367116, 6091868369488125021, 6133372490914863976, 6178265367129019163, 6244857225315059932, 6463710156605729092, 6576413018577432692, 6598538015126175411, 6661295267206380978, 6849808331498952045, 6909442103126738358, 7238565011966239238, 7302167330514645472, 7363533367549963358, 7384340123411537115, 7483731920713949064, 7492634448287820741, 7522235708431863509, 7559106995151888680, 7628209641055698181, 7955371823907869795, 7993663577616625534, 8318380318544429375, 8441401436675080578, 8611526084225318167, 8902580331337571042, 9000137925040549041, 9023478035160430945, 9031518461132666874, 9053250862307900413, 9096313364604515453, 9222166061143986690]
cassandra_1    | 
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,725 ColumnFamilyStore.java:426 - Initializing system_distributed.parent_repair_history
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,777 ColumnFamilyStore.java:426 - Initializing system_distributed.repair_history
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,807 ColumnFamilyStore.java:426 - Initializing system_distributed.view_build_status
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,825 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_distributed as storage service is not initialized
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,832 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,839 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions.closed_by_close_time
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,898 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions.closed_by_status
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,935 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions.closed_by_type
cassandra_1    | INFO  [main] 2020-02-15 16:24:40,967 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions.closed_by_workflow_id
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,006 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions_v2
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,016 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions_v2.closed_by_close_time_v2
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,053 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions_v2.closed_by_status_v2
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,086 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions_v2.closed_by_type_v2
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,130 ColumnFamilyStore.java:426 - Initializing cadence_visibility.closed_executions_v2.closed_by_workflow_id_v2
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,166 ColumnFamilyStore.java:426 - Initializing cadence_visibility.open_executions
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,172 ColumnFamilyStore.java:426 - Initializing cadence_visibility.open_executions.open_by_type
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | waiting for cassandra to start up
cadence_1      | + sleep 1
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,215 ColumnFamilyStore.java:426 - Initializing cadence_visibility.open_executions.open_by_workflow_id
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,255 ColumnFamilyStore.java:426 - Initializing cadence_visibility.schema_update_history
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,276 ColumnFamilyStore.java:426 - Initializing cadence_visibility.schema_version
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,286 ViewManager.java:137 - Not submitting build tasks for views in keyspace cadence_visibility as storage service is not initialized
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,292 ColumnFamilyStore.java:426 - Initializing system_auth.resource_role_permissons_index
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,302 ColumnFamilyStore.java:426 - Initializing system_auth.role_members
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,310 ColumnFamilyStore.java:426 - Initializing system_auth.role_permissions
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,316 ColumnFamilyStore.java:426 - Initializing system_auth.roles
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,328 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_auth as storage service is not initialized
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,333 ColumnFamilyStore.java:426 - Initializing system_traces.events
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,346 ColumnFamilyStore.java:426 - Initializing system_traces.sessions
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,357 ViewManager.java:137 - Not submitting build tasks for views in keyspace system_traces as storage service is not initialized
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,361 ColumnFamilyStore.java:426 - Initializing cadence.domains
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,375 ColumnFamilyStore.java:426 - Initializing cadence.domains_by_name_v2
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,381 ColumnFamilyStore.java:426 - Initializing cadence.events
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,388 ColumnFamilyStore.java:426 - Initializing cadence.executions
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,394 ColumnFamilyStore.java:426 - Initializing cadence.history_node
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,401 ColumnFamilyStore.java:426 - Initializing cadence.history_tree
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,408 ColumnFamilyStore.java:426 - Initializing cadence.schema_update_history
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,420 ColumnFamilyStore.java:426 - Initializing cadence.schema_version
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,430 ColumnFamilyStore.java:426 - Initializing cadence.tasks
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,432 ViewManager.java:137 - Not submitting build tasks for views in keyspace cadence as storage service is not initialized
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,435 AutoSavingCache.java:174 - Completed loading (1 ms; 26 keys) KeyCache cache
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,449 CommitLog.java:147 - Replaying /var/lib/cassandra/commitlog/CommitLog-6-1581783702765.log, /var/lib/cassandra/commitlog/CommitLog-6-1581783702766.log
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,471 CommitLogReader.java:105 - Skipping playback of empty log: CommitLog-6-1581783702766.log
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,943 CommitLog.java:149 - Log replay complete, 235 replayed mutations
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,944 StorageService.java:638 - Populating token metadata from system tables
cassandra_1    | INFO  [main] 2020-02-15 16:24:41,951 StorageService.java:645 - Token metadata: Normal Tokens:
cassandra_1    | /172.18.0.3:[-9164505853937689668, -9140318971186185580, -9113544450604874218, -9063052614749912161, -8960219402619035128, -8775333052802257261, -8732481292028470573, -8672290831330467624, -8647678848494905870, -8601228240488988801, -8407357627008425162, -8333376979492566840, -8269227312635425943, -8241629939187331870, -8131494450422448977, -8093467086603882915, -8064670819569502621, -8037179657605678693, -7915307951670520152, -7819664151987039467, -7807391123248037087, -7794392214244124494, -7744586738941112295, -7689538238274030898, -7624189110584063103, -7580987871126460387, -7551529169608191124, -7518812517876412938, -7513034692635767129, -7460463321159994436, -7265917373683037848, -7167646315693704387, -7128574870874449558, -7112185752998405676, -7108335897869054872, -7043283542143482933, -6931920376067673760, -6850487622590697124, -6848057624744661314, -6787942147181972240, -6738576824798558073, -6669289474079636931, -6659651766067167842, -6641653338979523154, -6625543461268037061, -6540139677528492827, -6485330870429999111, -6422228688080285973, -6279145031513832019, -6157985448058849541, -6117424707478575481, -6091318709497135599, -5979377252614877957, -5949999559744236145, -5738530099754037639, -5678073753528077357, -5606442783710815751, -5586210984850748016, -5402715365714680064, -5387300360288048634, -5319259222778688110, -5166118427843740546, -5151913291375316541, -5082184762098550075, -4891330247071776745, -4877688828405231921, -4834626391101183414, -4788573060565038043, -4605340572543162783, -4507798339860963515, -4502590621395695921, -4348711382592552266, -4344078053233129829, -4281421617778802173, -4210189800785889618, -4132502096828440723, -4066662485146739147, -4037494794877747783, -3973901891367242733, -3950778251935092437, -3900529089252107559, -3882927904761471179, -3860378533311551319, -3827490800986007579, -3748361340219759991, -3709025987000141661, -3696142979770626893, -3638467115236842692, -3625190520080778439, -3603191596077653159, -3592298183244133524, -3548231463808235537, -3502950843870859309, -3374002116569215697, -3319802243351440129, -3305994981179573769, -3194475623924041768, -3167693286331532765, -3160900603694963667, -3004015093682878874, -2867024030577870428, -2832285631284234669, -2680365836874694144, -2539460863408039879, -2385663986431974251, -2169003199888325664, -2131830906011685899, -2098112509078042619, -2020473546827131176, -2007907830683954630, -1930523114060706297, -1878385461611569093, -1643245496486755343, -1601862469295823337, -1600700365271420645, -1594880435622775370, -1517208080670188132, -1512808086813952144, -1445418993608927592, -1351365771990309449, -1344132880087562245, -1257484572307285847, -1189912828349020120, -1166981555587057035, -1085001762442193647, -1059213728947193721, -956741944123895246, -873772841104376178, -810181171196319017, -810043320620128106, -774018532725708709, -707303678989184333, -618320948996830110, -612231160131486041, -551269649407636870, -431386015694620417, -364120952620031739, -308389446835905727, -307077733436199656, -253149616125815298, -195777931336518879, -189503550750703012, -179254317581871936, -135340410611787902, -94198049482137722, 34374212114722049, 127188817389878050, 132965324808210526, 303622052456337254, 329858286239927786, 366018486534491906, 406638247841567057, 479851687137388556, 670026697430376075, 752710940507410501, 767407488769810792, 851060453052476895, 868493320096444384, 936781260251079188, 1022176325614192444, 1230697997356220952, 1292957465069455562, 1325119940892912351, 1334985843885371214, 1343108093999566216, 1361406166962715214, 1378863510406235771, 1400333617546998134, 1582807585029767367, 1597215431626030651, 1749477762232355436, 1792276602923579968, 1922474206199790498, 1988052706172858418, 2127317537841102726, 2259274909912667694, 2397671054589908406, 2429099292771470560, 2533786835888428493, 2578246049355406059, 2686298223326262725, 2701717933442637254, 2807760787041561577, 2884214447668329263, 3157316500484678162, 3187716919969840056, 3270860775362644337, 3303610804117999142, 3391705420326532576, 3491091462970439509, 3706100641950710733, 3724805958006403234, 3757364129542921818, 3806036858066157266, 3895832271172542984, 3973536966711831758, 4030747908146128519, 4123902827963866009, 4160568841072448445, 4228519987033230575, 4257747407932998692, 4273551570363887293, 4274043940588997505, 4372472864976210952, 4611319075408017514, 4645600139360069230, 4750664360884816095, 4945720248950484541, 4951369330675607518, 4991207015817107215, 5076741404806976287, 5084631119909752433, 5402558636390072478, 5462611429300242125, 5553289478482844262, 5608290272328210330, 5744662837408377582, 5760902679518379616, 5848812504429212946, 5869020964144644718, 5914929288048126036, 5973397761359376924, 6031927103805483694, 6055222436683253778, 6081442801756367116, 6091868369488125021, 6133372490914863976, 6178265367129019163, 6244857225315059932, 6463710156605729092, 6576413018577432692, 6598538015126175411, 6661295267206380978, 6849808331498952045, 6909442103126738358, 7238565011966239238, 7302167330514645472, 7363533367549963358, 7384340123411537115, 7483731920713949064, 7492634448287820741, 7522235708431863509, 7559106995151888680, 7628209641055698181, 7955371823907869795, 7993663577616625534, 8318380318544429375, 8441401436675080578, 8611526084225318167, 8902580331337571042, 9000137925040549041, 9023478035160430945, 9031518461132666874, 9053250862307900413, 9096313364604515453, 9222166061143986690]
cassandra_1    | 
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'waiting for cassandra to start up
cadence_1      | 
cadence_1      | + sleep 1
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,536 QueryProcessor.java:163 - Preloaded 147 prepared statements
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,537 StorageService.java:656 - Cassandra version: 3.11.5
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,538 StorageService.java:657 - Thrift API version: 20.1.0
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,538 StorageService.java:658 - CQL supported versions: 3.4.4 (default: 3.4.4)
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,539 StorageService.java:660 - Native protocol supported versions: 3/v3, 4/v4, 5/v5-beta (default: 4/v4)
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,587 IndexSummaryManager.java:87 - Initializing index summary manager with a memory pool size of 48 MB and a resize interval of 60 minutes
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,610 MessagingService.java:750 - Starting Messaging Service on /172.18.0.3:7000 (eth0)
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,643 StorageService.java:588 - Unable to gossip with any peers but continuing anyway since node is in its own seed list
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,659 StorageService.java:742 - Loading persisted ring state
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,660 StorageService.java:870 - Starting up server gossip
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,711 TokenMetadata.java:497 - Updating topology for /172.18.0.3
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,711 TokenMetadata.java:497 - Updating topology for /172.18.0.3
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,754 StorageService.java:1037 - Using saved tokens [-1059213728947193721, -1085001762442193647, -1166981555587057035, -1189912828349020120, -1257484572307285847, -1344132880087562245, -1351365771990309449, -135340410611787902, -1445418993608927592, -1512808086813952144, -1517208080670188132, -1594880435622775370, -1600700365271420645, -1601862469295823337, -1643245496486755343, -179254317581871936, -1878385461611569093, -189503550750703012, -1930523114060706297, -195777931336518879, -2007907830683954630, -2020473546827131176, -2098112509078042619, -2131830906011685899, -2169003199888325664, -2385663986431974251, -253149616125815298, -2539460863408039879, -2680365836874694144, -2832285631284234669, -2867024030577870428, -3004015093682878874, -307077733436199656, -308389446835905727, -3160900603694963667, -3167693286331532765, -3194475623924041768, -3305994981179573769, -3319802243351440129, -3374002116569215697, -3502950843870859309, -3548231463808235537, -3592298183244133524, -3603191596077653159, -3625190520080778439, -3638467115236842692, -364120952620031739, -3696142979770626893, -3709025987000141661, -3748361340219759991, -3827490800986007579, -3860378533311551319, -3882927904761471179, -3900529089252107559, -3950778251935092437, -3973901891367242733, -4037494794877747783, -4066662485146739147, -4132502096828440723, -4210189800785889618, -4281421617778802173, -431386015694620417, -4344078053233129829, -4348711382592552266, -4502590621395695921, -4507798339860963515, -4605340572543162783, -4788573060565038043, -4834626391101183414, -4877688828405231921, -4891330247071776745, -5082184762098550075, -5151913291375316541, -5166118427843740546, -5319259222778688110, -5387300360288048634, -5402715365714680064, -551269649407636870, -5586210984850748016, -5606442783710815751, -5678073753528077357, -5738530099754037639, -5949999559744236145, -5979377252614877957, -6091318709497135599, -6117424707478575481, -612231160131486041, -6157985448058849541, -618320948996830110, -6279145031513832019, -6422228688080285973, -6485330870429999111, -6540139677528492827, -6625543461268037061, -6641653338979523154, -6659651766067167842, -6669289474079636931, -6738576824798558073, -6787942147181972240, -6848057624744661314, -6850487622590697124, -6931920376067673760, -7043283542143482933, -707303678989184333, -7108335897869054872, -7112185752998405676, -7128574870874449558, -7167646315693704387, -7265917373683037848, -7460463321159994436, -7513034692635767129, -7518812517876412938, -7551529169608191124, -7580987871126460387, -7624189110584063103, -7689538238274030898, -774018532725708709, -7744586738941112295, -7794392214244124494, -7807391123248037087, -7819664151987039467, -7915307951670520152, -8037179657605678693, -8064670819569502621, -8093467086603882915, -810043320620128106, -810181171196319017, -8131494450422448977, -8241629939187331870, -8269227312635425943, -8333376979492566840, -8407357627008425162, -8601228240488988801, -8647678848494905870, -8672290831330467624, -8732481292028470573, -873772841104376178, -8775333052802257261, -8960219402619035128, -9063052614749912161, -9113544450604874218, -9140318971186185580, -9164505853937689668, -94198049482137722, -956741944123895246, 1022176325614192444, 1230697997356220952, 127188817389878050, 1292957465069455562, 1325119940892912351, 132965324808210526, 1334985843885371214, 1343108093999566216, 1361406166962715214, 1378863510406235771, 1400333617546998134, 1582807585029767367, 1597215431626030651, 1749477762232355436, 1792276602923579968, 1922474206199790498, 1988052706172858418, 2127317537841102726, 2259274909912667694, 2397671054589908406, 2429099292771470560, 2533786835888428493, 2578246049355406059, 2686298223326262725, 2701717933442637254, 2807760787041561577, 2884214447668329263, 303622052456337254, 3157316500484678162, 3187716919969840056, 3270860775362644337, 329858286239927786, 3303610804117999142, 3391705420326532576, 34374212114722049, 3491091462970439509, 366018486534491906, 3706100641950710733, 3724805958006403234, 3757364129542921818, 3806036858066157266, 3895832271172542984, 3973536966711831758, 4030747908146128519, 406638247841567057, 4123902827963866009, 4160568841072448445, 4228519987033230575, 4257747407932998692, 4273551570363887293, 4274043940588997505, 4372472864976210952, 4611319075408017514, 4645600139360069230, 4750664360884816095, 479851687137388556, 4945720248950484541, 4951369330675607518, 4991207015817107215, 5076741404806976287, 5084631119909752433, 5402558636390072478, 5462611429300242125, 5553289478482844262, 5608290272328210330, 5744662837408377582, 5760902679518379616, 5848812504429212946, 5869020964144644718, 5914929288048126036, 5973397761359376924, 6031927103805483694, 6055222436683253778, 6081442801756367116, 6091868369488125021, 6133372490914863976, 6178265367129019163, 6244857225315059932, 6463710156605729092, 6576413018577432692, 6598538015126175411, 6661295267206380978, 670026697430376075, 6849808331498952045, 6909442103126738358, 7238565011966239238, 7302167330514645472, 7363533367549963358, 7384340123411537115, 7483731920713949064, 7492634448287820741, 7522235708431863509, 752710940507410501, 7559106995151888680, 7628209641055698181, 767407488769810792, 7955371823907869795, 7993663577616625534, 8318380318544429375, 8441401436675080578, 851060453052476895, 8611526084225318167, 868493320096444384, 8902580331337571042, 9000137925040549041, 9023478035160430945, 9031518461132666874, 9053250862307900413, 9096313364604515453, 9222166061143986690, 936781260251079188]
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,764 StorageService.java:1521 - JOINING: Finish joining ring
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,798 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence_visibility', ColumnFamily='schema_update_history')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,803 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence_visibility', ColumnFamily='closed_executions_v2')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,804 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence_visibility', ColumnFamily='open_executions')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,805 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence_visibility', ColumnFamily='schema_version')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,806 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence_visibility', ColumnFamily='closed_executions')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,807 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='domains')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,809 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='history_node')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,809 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='history_tree')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,810 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='events')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,810 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='schema_update_history')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,811 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='domains_by_name_v2')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,813 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='schema_version')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,814 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='executions')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,814 SecondaryIndexManager.java:512 - Executing pre-join tasks for: CFS(Keyspace='cadence', ColumnFamily='tasks')
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,872 StorageService.java:2442 - Node /172.18.0.3 state jump to NORMAL
cassandra_1    | INFO  [main] 2020-02-15 16:24:42,892 Gossiper.java:1780 - Waiting for gossip to settle...
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | waiting for cassandra to start up
cadence_1      | + sleep 1
statsd_1       | 15/02/2020 16:24:44 :: [listener] MetricLineReceiver connection with 127.0.0.1:59010 established
statsd_1       | 15/02/2020 16:24:44 :: [listener] MetricLineReceiver connection with 127.0.0.1:59010 closed cleanly
statsd_1       | 15/02/2020 16:24:44 :: [tagdb] Tagging stats.statsd.graphiteStats.flush_length
statsd_1       | 15/02/2020 16:24:44 :: [tagdb] Tagged stats.statsd.graphiteStats.flush_length in 0.2237691879272461
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | waiting for cassandra to start up
cadence_1      | + sleep 1
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | + sleep 1
cadence_1      | waiting for cassandra to start up
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | + sleep 1
cadence_1      | waiting for cassandra to start up
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | waiting for cassandra to start up
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | + sleep 1
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | + sleep 1
cadence_1      | waiting for cassandra to start up
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cassandra_1    | INFO  [main] 2020-02-15 16:24:50,875 Gossiper.java:1811 - No gossip backlog; proceeding
cadence_1      | Connection error: ('Unable to connect to any servers', {'172.18.0.3:9042': error(111, "Tried connecting to [('172.18.0.3', 9042)]. Last error: Connection refused")})
cadence_1      | waiting for cassandra to start up
cadence_1      | + echo 'waiting for cassandra to start up'
cadence_1      | + sleep 1
cassandra_1    | INFO  [main] 2020-02-15 16:24:51,234 NativeTransportService.java:68 - Netty using native Epoll event loop
cassandra_1    | INFO  [main] 2020-02-15 16:24:51,373 Server.java:158 - Using Netty Version: [netty-buffer=netty-buffer-4.0.44.Final.452812a, netty-codec=netty-codec-4.0.44.Final.452812a, netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a, netty-codec-http=netty-codec-http-4.0.44.Final.452812a, netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a, netty-common=netty-common-4.0.44.Final.452812a, netty-handler=netty-handler-4.0.44.Final.452812a, netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb, netty-transport=netty-transport-4.0.44.Final.452812a, netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a, netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a, netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a, netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a]
cassandra_1    | INFO  [main] 2020-02-15 16:24:51,373 Server.java:159 - Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)...
cassandra_1    | INFO  [main] 2020-02-15 16:24:51,451 CassandraDaemon.java:556 - Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it
cadence_1      | + cqlsh --cqlversion=3.4.4 cassandra
cadence_1      | + echo 'cassandra started'
cadence_1      | cassandra started
cadence_1      | + '[' false == true ']'
cadence_1      | + '[' '' '!=' true ']'
cadence_1      | + setup_schema
cadence_1      | + '[' cassandra == mysql ']'
cadence_1      | + '[' cassandra == postgres setup cassandra schema
cadence_1      | ']'
cadence_1      | + echo 'setup cassandra schema'
cadence_1      | + setup_cassandra_schema
cadence_1      | + SCHEMA_DIR=/etc/cadence/schema/cassandra/cadence/versioned
cadence_1      | + cadence-cassandra-tool --ep cassandra create -k cadence --rf 1
cadence_1      | + cadence-cassandra-tool --ep cassandra -k cadence setup-schema -v 0.0
cadence_1      | 2020/02/15 16:24:53 Starting schema setup, config=&{SchemaFilePath: InitialVersion:0.0 Overwrite:false DisableVersioning:false}
cadence_1      | 2020/02/15 16:24:53 Setting up version tables
cadence_1      | 2020/02/15 16:24:53 Cannot add already existing table "schema_version" to keyspace "cadence"
cadence_1      | + cadence-cassandra-tool --ep cassandra -k cadence update-schema -d /etc/cadence/schema/cassandra/cadence/versioned
cadence_1      | 2020/02/15 16:24:53 UpdateSchemeTask started, config=&{DBName: TargetVersion: SchemaDir:/etc/cadence/schema/cassandra/cadence/versioned IsDryRun:false}
cadence_1      | 2020/02/15 16:24:53 ---- Executing updates for version 0.12 ----
cadence_1      | 2020/02/15 16:24:53 CREATE TYPE branch_range (branch_id uuid,end_node_id bigint, );
cadence_1      | 2020/02/15 16:24:53 error executing CQL statement:A user type of name cadence.branch_range already exists
cadence_1      | + VISIBILITY_SCHEMA_DIR=/etc/cadence/schema/cassandra/visibility/versioned
cadence_1      | + cadence-cassandra-tool --ep cassandra create -k cadence_visibility --rf 1
cadence_1      | + cadence-cassandra-tool --ep cassandra -k cadence_visibility setup-schema -v 0.0
cadence_1      | 2020/02/15 16:24:53 Starting schema setup, config=&{SchemaFilePath: InitialVersion:0.0 Overwrite:false DisableVersioning:false}
cadence_1      | 2020/02/15 16:24:53 Setting up version tables
cadence_1      | 2020/02/15 16:24:53 Cannot add already existing table "schema_version" to keyspace "cadence_visibility"
cadence_1      | + cadence-cassandra-tool --ep cassandra -k cadence_visibility update-schema -d /etc/cadence/schema/cassandra/visibility/versioned
cadence_1      | 2020/02/15 16:24:53 UpdateSchemeTask started, config=&{DBName: TargetVersion: SchemaDir:/etc/cadence/schema/cassandra/visibility/versioned IsDryRun:false}
cadence_1      | 2020/02/15 16:24:53 found zero updates from current version 0.4
cadence_1      | 2020/02/15 16:24:53 UpdateSchemeTask done
cadence_1      | + '[' false == true ']'
cadence_1      | + bash /start-cadence.sh
cadence_1      | + dockerize -template /etc/cadence/config/config_template.yaml:/etc/cadence/config/docker.yaml
cadence_1      | + exec cadence-server --root /etc/cadence --env docker start --services=history,matching,frontend,worker
cadence_1      | 2020/02/15 16:24:53 Loading config; env=docker,zone=,configDir=/etc/cadence/config
cadence_1      | 2020/02/15 16:24:53 Loading configFiles=[/etc/cadence/config/docker.yaml]
cadence_1      | 2020/02/15 16:24:53 Incompatible cassandra versions: version mismatch for keyspace/database: "cadence". Expected version: 0.24 cannot be greater than Actual version: 0.11
statsd_1       | 15/02/2020 16:24:53 :: [listener] MetricLineReceiver connection with 127.0.0.1:59072 established
statsd_1       | 15/02/2020 16:24:53 :: [listener] MetricLineReceiver connection with 127.0.0.1:59072 closed cleanly
statsd_1       | 15/02/2020 16:24:54 :: [listener] MetricLineReceiver connection with 127.0.0.1:59074 established
statsd_1       | 15/02/2020 16:24:54 :: [listener] MetricLineReceiver connection with 127.0.0.1:59074 closed cleanly
statsd_1       | 15/02/2020 16:24:54 :: [tagdb] Tagging stats.timers.view.graphite.tags.views.POST.sum_squares_90
statsd_1       | 15/02/2020 16:24:54 :: [tagdb] Tagged stats.timers.view.graphite.tags.views.POST.sum_squares_90 in 0.007393360137939453
cadence_cadence_1 exited with code 1
statsd_1       | 15/02/2020 16:25:04 :: [listener] MetricLineReceiver connection with 127.0.0.1:59078 established
statsd_1       | 15/02/2020 16:25:04 :: [listener] MetricLineReceiver connection with 127.0.0.1:59078 closed cleanly
statsd_1       | 15/02/2020 16:25:04 :: [tagdb] Tagging carbon.agents.516060f9ec79-a.whitelistRejects
statsd_1       | 15/02/2020 16:25:04 :: [tagdb] Tagged carbon.agents.516060f9ec79-a.whitelistRejects in 0.1206212043762207

When I run the next command in the instructions:

docker run --network=host --rm ubercadence/cli:master --do test-domain domain register -rd 1
Error: Register Domain operation failed.
Error Details: code:unknown message:received unknown error calling service: "cadence-frontend", procedure: "WorkflowService::RegisterDomain", err: dial tcp 127.0.0.1:7933: connect: connection refused
('export CADENCE_CLI_SHOW_STACKS=1' to see stack traces)

When i try to start up the front end UI http://localhost:8088/, I get the following error in cadence.

cadence-web_1  | { Error: getaddrinfo ENOTFOUND cadence
cadence-web_1  |     at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26)
cadence-web_1  |   errno: 'ENOTFOUND',
cadence-web_1  |   code: 'ENOTFOUND',
cadence-web_1  |   syscall: 'getaddrinfo',
cadence-web_1  |   hostname: 'cadence' }

Is there any other special configuration I need on my computer?

Any help appreciated, Thanks! Derek

darewreck54 commented 4 years ago

strange...when i re-dl the docker compose file via

curl -O https://raw.githubusercontent.com/uber/cadence/master/docker/docker-compose.ym

it works...wonder how it could have gone bad

mfateev commented 4 years ago

Unfortunately docker requires periodic cleaning and full reset due to caching. I recommend running docker-compose down and docker system prune --volumes periodically.

darewreck54 commented 4 years ago

Thanks, yeah I realized solved it.

Side question: I notice in the example it will register the interface classes. How does cadence associate it with the actual implementation.

TripBookingWorkflow trip1 = workflowClient.newWorkflowStub(TripBookingWorkflow.class); try { trip1.bookTrip("trip1"); } catch (WorkflowException e) { // Expected e.printStackTrace(); }

Say if there are multiple implementation of the trip booking workflow class. How does it know what implementation it’s using.

Thanks Derek

Get Outlook for iOShttps://aka.ms/o0ukef


From: Maxim Fateev notifications@github.com Sent: Saturday, February 15, 2020 11:07:42 AM To: uber/cadence cadence@noreply.github.com Cc: Derek d_hang10@hotmail.com; Author author@noreply.github.com Subject: Re: [uber/cadence] Issue starting up (#3054)

Unfortunately docker requires periodic cleaning and full reset due to caching. I recommend running docker-compose down and docker system prune --volumes periodically.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/uber/cadence/issues/3054?email_source=notifications&email_token=AD3MFTTNMU74VHHEJHKP22TRDA4P5A5CNFSM4KV2EOU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL3UURY#issuecomment-586631751, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD3MFTT6QCTNZK7TVQJFD43RDA4P5ANCNFSM4KV2EOUQ.

darewreck54 commented 4 years ago

Oops I mean “that solved it”*

Get Outlook for iOShttps://aka.ms/o0ukef


From: Derek Hang d_hang10@hotmail.com Sent: Saturday, February 15, 2020 11:14:26 AM To: uber/cadence reply@reply.github.com; uber/cadence cadence@noreply.github.com Cc: Author author@noreply.github.com Subject: Re: [uber/cadence] Issue starting up (#3054)

Thanks, yeah I realized solved it.

Side question: I notice in the example it will register the interface classes. How does cadence associate it with the actual implementation.

TripBookingWorkflow trip1 = workflowClient.newWorkflowStub(TripBookingWorkflow.class); try { trip1.bookTrip("trip1"); } catch (WorkflowException e) { // Expected e.printStackTrace(); }

Say if there are multiple implementation of the trip booking workflow class. How does it know what implementation it’s using.

Thanks Derek

Get Outlook for iOShttps://aka.ms/o0ukef


From: Maxim Fateev notifications@github.com Sent: Saturday, February 15, 2020 11:07:42 AM To: uber/cadence cadence@noreply.github.com Cc: Derek d_hang10@hotmail.com; Author author@noreply.github.com Subject: Re: [uber/cadence] Issue starting up (#3054)

Unfortunately docker requires periodic cleaning and full reset due to caching. I recommend running docker-compose down and docker system prune --volumes periodically.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/uber/cadence/issues/3054?email_source=notifications&email_token=AD3MFTTNMU74VHHEJHKP22TRDA4P5A5CNFSM4KV2EOU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL3UURY#issuecomment-586631751, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD3MFTT6QCTNZK7TVQJFD43RDA4P5ANCNFSM4KV2EOUQ.

mfateev commented 4 years ago

When trip1.bookTrip is called it is converted by the Java client to StartWorkflowExecution gRPC call. One of the parameters of the call is TaskList which is the name of a queue activity tasks are dispatched through. Activity worker listens a task list for activity tasks and then dispatches them to the implementation object registered with the worker. Only one implementation of the activity interface per task list is allowed. So if you need more than one implementation then you have to use different task lists to dispatch tasks to them.

Use ActivityOptions.TaskList property when creating an activity stub to override the task list name.

I recommend joining Cadence Slack Workspace to continue the conversation.

vancexu commented 4 years ago

I will close this ticket since Max helped you resolved it. Hopefully you get answers to your other questions @darewreck54, if not, it's a good idea to join the cadence slack channel for future discussion.