[CI] `linux://rllib:test_apex_ddpg` is failing/flaky on master.

[rickyyx]

• 28c0553f8f6478795fd18be2421efb61ec6035cb FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 75a0c491e8ee4009e8828081059cfc98543c4d51 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 47aa6c432353fb22a81103c28dc343b6d9407d52 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 31347c026e985e9baa438bdaa3159fb5cd7dea6c FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 1ac75f9fea2c70f3905daccafd1ae0d595cda732 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 4652752e8620c1251b74930b512c6620ee9bc0f4 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• c5c3e6ccaccee430330c078ffb68bd12e7866e04 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 081ce2fe4e88ae5f36aa0a6b45fb7048c01947c7 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 366f2a9c687f709134c69adb0de94c3f079d2f3a FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 251903bc49b5a29a8185e0fa58d67c64fe2fc076 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 0ee478b13152254380f3dd638bd666a114fd6e67 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 29791d9acedaa4700d0b9dccecab4cddfcfd64b4 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• fd0148878d9b13449f708ae521d9bb1678c69b10 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 849afc63e55ab15e63c1ed86e08c81d449b1f7e8 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 65a8b0f8c27f0b9049b21b2b9bc9ed07782b9e81 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 91040d91bba3cdfc29475b77e8d199884469ab86 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 8486ddd1bf909449a22262c52a90f8359797fc16 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 7e939d79bcc68603b7c9a99fa6ef880dd526e931 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 4732753327109d5c33b4cf69d375730057efc631 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 68a35fceef6a47fa3e3da41041ee32dafe7f24a8 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• 23b3a599b9df8100558c477e94b0b19b1a38ac27 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• b5687a1aaa3dc72dfde9758e730607a4ab22afe4 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• e142be077f0c727ab11ba51ecaba9a98b7bfe474 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)
• b510640f15a0fa4782b83ec2ea0749386b615b15 FAILED Buildkite :brain: RLlib: Algorithm Tests (specific algos)

.... Generated from flaky test tracker. Please do not edit the signature in this section. DataCaseName-linux://rllib:test_apex_ddpg-END ....

[rickyyx]

Marking it as release blocker for now since it is showing up in the release branch CI as well: https://buildkite.com/ray-project/oss-ci-build-pr/builds/2138#0183d2a7-1715-49d7-afb9-1c1bae966d09

[gjoliver]

I looked at this test. I don't know what we can do tbh. when it passes, it passes. when it times out, pytest just hangs at "collecting tests ..." without any error logs.

[gjoliver]

I am willing to declare this non-release blocking.

[rickyyx]

@gjoliver Looks like there are some failure output w.r.t this test in this build, maybe this will help root-causing.

[gjoliver]

I don't see anything 😢 since this has been flaky for at least a quarter now, I am gonna say this is not release blocking for now. thanks man.

[rickyyx]

Oh, did you see this?

Test failure

``` =================================== FAILURES =================================== --   | ____ TestApexDDPG.test_apex_ddpg_compilation_and_per_worker_epsilon_values _____   |     | self =   |     | def test_apex_ddpg_compilation_and_per_worker_epsilon_values(self):   | """Test whether APEX-DDPG can be built on all frameworks."""   | config = (   | apex_ddpg.ApexDDPGConfig()   | .rollouts(num_rollout_workers=2)   | .reporting(min_sample_timesteps_per_iteration=100)   | .training(   | num_steps_sampled_before_learning_starts=0,   | optimizer={"num_replay_buffer_shards": 1},   | )   | .environment(env="Pendulum-v1")   | )   |     | num_iterations = 1   |     | for _ in framework_iterator(config, with_eager_tracing=True):   | trainer = config.build()   |     | # Test per-worker scale distribution.   | infos = trainer.workers.foreach_policy(   | lambda p, _: p.get_exploration_state()   | )   | scale = [i["cur_scale"] for i in infos]   | expected = [   | 0.4 ** (1 + (i + 1) / float(config.num_workers - 1) * 7)   | for i in range(config.num_workers)   | ]   | check(scale, [0.0] + expected)   |     | for _ in range(num_iterations):   | > results = trainer.train()   |     | rllib/algorithms/apex_ddpg/tests/test_apex_ddpg.py:51:   | _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _   | /ray/python/ray/tune/trainable/trainable.py:352: in train   | result = self.step()   | /ray/python/ray/rllib/algorithms/algorithm.py:772: in step   | results, train_iter_ctx = self._run_one_training_iteration()   | /ray/python/ray/rllib/algorithms/algorithm.py:2944: in _run_one_training_iteration   | results = self.training_step()   | /ray/python/ray/rllib/algorithms/apex_ddpg/apex_ddpg.py:192: in training_step   | return ApexDQN.training_step(self)   | /ray/python/ray/rllib/algorithms/apex_dqn/apex_dqn.py:457: in training_step   | return copy.deepcopy(self.learner_thread.learner_info)   | /opt/miniconda/lib/python3.7/copy.py:150: in deepcopy   | y = copier(x, memo)   | /opt/miniconda/lib/python3.7/copy.py:241: in _deepcopy_dict   | y[deepcopy(key, memo)] = deepcopy(value, memo)   | /opt/miniconda/lib/python3.7/copy.py:184: in deepcopy   | memo[d] = y   | _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _   |     | signum = 15   | frame =   |     | def sigterm_handler(signum, frame):   | > sys.exit(signum)   | E SystemExit: 15   |     | /ray/python/ray/_private/worker.py:1628: SystemExit ```

[gjoliver]

huh, thanks I didn't notice it. taking a look.

[gjoliver]

job received a sigterm from where ...

[rickyyx]

no more relevant?