BallAerospace / COSMOS

Ball Aerospace COSMOS
https://ballaerospace.github.io/cosmos-website/
Other
361 stars 129 forks source link

Decom error adding key during iteration #1640

Closed ghost closed 2 years ago

ghost commented 2 years ago

I sometimes see this in the Docker logs for the operator microservice:

{"time":1652971630871585300,"@timestamp":"2022-05-19T14:47:10.871+00:00","severity":"ERROR","microservice_name":"DEFAULT__DECOM__INST","container_name":"fb01996b4427","log":"Decom error: RuntimeError : can't add a new key into hash during iteration\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/utilities/metric.rb:75:in `add_sample'\ndecom_microservice.rb:73:in `decom_packet'\ndecom_microservice.rb:46:in `block in run'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/utilities/store_autoload.rb:237:in `block (3 levels) in read_topics'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/utilities/store_autoload.rb:235:in `each'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/utilities/store_autoload.rb:235:in `block (2 levels) in read_topics'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/utilities/store_autoload.rb:234:in `each'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/utilities/store_autoload.rb:234:in `block in read_topics'\n/usr/lib/ruby/gems/3.0.0/gems/connection_pool-2.2.5/lib/connection_pool.rb:63:in `block (2 levels) in with'\n/usr/lib/ruby/gems/3.0.0/gems/connection_pool-2.2.5/lib/connection_pool.rb:62:in `handle_interrupt'\n/usr/lib/ruby/gems/3.0.0/gems/connection_pool-2.2.5/lib/connection_pool.rb:62:in `block in with'\n/usr/lib/ruby/gems/3.0.0/gems/connection_pool-2.2.5/lib/connection_pool.rb:59:in `handle_interrupt'\n/usr/lib/ruby/gems/3.0.0/gems/connection_pool-2.2.5/lib/connection_pool.rb:59:in `with'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/utilities/store_autoload.rb:230:in `read_topics'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/utilities/store_autoload.rb:225:in `read_topics'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/topics/topic.rb:29:in `read_topics'\ndecom_microservice.rb:43:in `run'\n/usr/lib/ruby/gems/3.0.0/gems/cosmos-5.0.4.pre.beta0/lib/cosmos/microservices/microservice.rb:47:in `run'\ndecom_microservice.rb:135:in `<main>'"}