run_benchalerts: 'NoneType' object is not subscriptable

[austin3dickey]

https://buildkite.com/apache-arrow/arrow-bci-schedule-and-publish/builds/50939#018979ba-0272-4b83-891a-dd1f1a3e59dc/120-178

[2023-07-21 18:37:21,971] ERROR in run_benchalerts: Error running benchalerts: TypeError("'NoneType' object is not subscriptable")
Traceback (most recent call last):
  File "/app/buildkite/schedule_and_publish/run_benchalerts.py", line 53, in run_benchalerts
    run_one_benchalerts_run(benchalerts_run)
  File "/app/buildkite/schedule_and_publish/run_benchalerts.py", line 17, in run_one_benchalerts_run
    benchalerts_run.run_benchalerts()
  File "/app/models/benchalerts_run.py", line 132, in run_benchalerts
    output = pipeline.run_pipeline()
  File "/usr/local/lib/python3.8/site-packages/benchalerts/alert_pipeline.py", line 88, in run_pipeline
    step_outputs[step.step_name] = step.run_step(
  File "/usr/local/lib/python3.8/site-packages/benchalerts/pipeline_steps/github.py", line 102, in run_step
    status=self.alerter.github_check_status(full_comparison),
  File "/app/models/benchalerts_run.py", line 244, in github_check_status
    stable_comparison, _ = self._separate_known_unstable_benchmarks(full_comparison)
  File "/app/models/benchalerts_run.py", line 227, in _separate_known_unstable_benchmarks
    run.compare_results = [
  File "/app/models/benchalerts_run.py", line 230, in <listcomp>
    if not self._is_known_unstable(result)
  File "/app/models/benchalerts_run.py", line 213, in _is_known_unstable
    return result_info["contender"]["language"] not in ["Python", "R"]
TypeError: 'NoneType' object is not subscriptable

[austin3dickey]

I accidentally committed to main the bugfix: https://github.com/voltrondata-labs/arrow-benchmarks-ci/commit/6510ecac417b8b92c9731d204dd3a71f9d0d416c

[austin3dickey]

The fix didn't work:

Traceback (most recent call last):
--
  | File "/app/buildkite/schedule_and_publish/run_benchalerts.py", line 53, in run_benchalerts
  | run_one_benchalerts_run(benchalerts_run)
  | File "/app/buildkite/schedule_and_publish/run_benchalerts.py", line 17, in run_one_benchalerts_run
  | benchalerts_run.run_benchalerts()
  | File "/app/models/benchalerts_run.py", line 132, in run_benchalerts
  | output = pipeline.run_pipeline()
  | File "/usr/local/lib/python3.8/site-packages/benchalerts/alert_pipeline.py", line 88, in run_pipeline
  | step_outputs[step.step_name] = step.run_step(
  | File "/usr/local/lib/python3.8/site-packages/benchalerts/pipeline_steps/github.py", line 102, in run_step
  | status=self.alerter.github_check_status(full_comparison),
  | File "/app/models/benchalerts_run.py", line 247, in github_check_status
  | stable_comparison, _ = self._separate_known_unstable_benchmarks(full_comparison)
  | File "/app/models/benchalerts_run.py", line 230, in _separate_known_unstable_benchmarks
  | run.compare_results = [
  | File "/app/models/benchalerts_run.py", line 233, in <listcomp>
  | if not self._is_known_unstable(result)
  | File "/app/models/benchalerts_run.py", line 213, in _is_known_unstable
  | return result_info.get("contender", {}).get("language", "") not in [
  | AttributeError: 'NoneType' object has no attribute 'get'

[alistaire47]

I've run into this annoyance before; this is because it's an explicit None so .get() still returns None; you need an or.

[austin3dickey]

I keep accidentally committing to main. Here's where we're at now: https://github.com/voltrondata-labs/arrow-benchmarks-ci/blob/c2f645777ac1e4eaea2ff51830455099ca0405d1/models/benchalerts_run.py#L213-L216

[alistaire47]

I'm surprised we don't have branch protections set up; we should

[austin3dickey]

I do not see this issue anymore.

[austin3dickey]

I'm surprised we don't have branch protections set up; we should

Just set up a protection on main that you must merge via a PR.