search

google-research / raksha

Apache License 2.0
35 stars 17 forks source link

The test //src/frontends/arcs/xform_to_datalog tests may be nondeterministic. #614

Open Cypher1 opened 2 years ago

Cypher1 commented 2 years ago

Running the following command at latest

a8442a7 Thu Jul 7 10:22:20 2022 -0700 (HEAD -> main, origin/main) Harsha Mandadi Modify antlr grammar for IR parser to allow numbers as strings (#610)
bazel test //src/frontends/arcs/xform_to_datalog/... --runs_per_test=100

Yielded

INFO: Analyzed 13 targets (0 packages loaded, 0 targets configured).
INFO: Found 8 targets and 5 test targets...
FAIL: //src/frontends/arcs/xform_to_datalog:generate_datalog_program_test (run 66 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_66_of_100/test.log)
FAIL: //src/frontends/arcs/xform_to_datalog:generate_datalog_program_test (run 44 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_44_of_100/test.log)
FAIL: //src/frontends/arcs/xform_to_datalog:generate_datalog_program_test (run 83 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_83_of_100/test.log)

FAILED: //src/frontends/arcs/xform_to_datalog:generate_datalog_program_test (Summary)
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_66_of_100/test.log
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_44_of_100/test.log
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_83_of_100/test.log
FAIL: //src/frontends/arcs/xform_to_datalog:datalog_facts_test (run 70 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/datalog_facts_test/run_70_of_100/test.log)
FAIL: //src/frontends/arcs/xform_to_datalog:authorization_logic_test (run 4 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_4_of_100/test.log)
FAIL: //src/frontends/arcs/xform_to_datalog:authorization_logic_test (run 60 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_60_of_100/test.log)
FAIL: //src/frontends/arcs/xform_to_datalog:authorization_logic_test (run 61 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_61_of_100/test.log)
FAIL: //src/frontends/arcs/xform_to_datalog:authorization_logic_test (run 18 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_18_of_100/test.log)
FAIL: //src/frontends/arcs/xform_to_datalog:authorization_logic_test (run 46 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_46_of_100/test.log)

FAILED: //src/frontends/arcs/xform_to_datalog:authorization_logic_test (Summary)
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_4_of_100/test.log
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_60_of_100/test.log
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_61_of_100/test.log
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_18_of_100/test.log
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_46_of_100/test.log
FAIL: //src/frontends/arcs/xform_to_datalog:authorization_logic_datalog_facts_test (run 19 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_datalog_facts_test/run_19_of_100/test.log)
FAIL: //src/frontends/arcs/xform_to_datalog:authorization_logic_datalog_facts_test (run 32 of 100) (see /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_datalog_facts_test/run_32_of_100/test.log)

FAILED: //src/frontends/arcs/xform_to_datalog:authorization_logic_datalog_facts_test (Summary)
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_datalog_facts_test/run_19_of_100/test.log
      /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_datalog_facts_test/run_32_of_100/test.log
INFO: Elapsed time: 3.006s, Critical Path: 0.15s
INFO: 501 processes: 602 linux-sandbox.
INFO: Build completed, 4 tests FAILED, 501 total actions
//src/frontends/arcs/xform_to_datalog:manifest_datalog_facts_test        PASSED in 0.1s
  Stats over 100 runs: max = 0.1s, min = 0.1s, avg = 0.1s, dev = 0.0s
//src/frontends/arcs/xform_to_datalog:authorization_logic_datalog_facts_test FAILED in 2 out of 100 in 0.1s
  Stats over 100 runs: max = 0.1s, min = 0.0s, avg = 0.1s, dev = 0.0s
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_datalog_facts_test/run_19_of_100/test.log
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_datalog_facts_test/run_32_of_100/test.log
//src/frontends/arcs/xform_to_datalog:authorization_logic_test           FAILED in 5 out of 100 in 0.1s
  Stats over 100 runs: max = 0.1s, min = 0.0s, avg = 0.1s, dev = 0.0s
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_4_of_100/test.log
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_60_of_100/test.log
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_61_of_100/test.log
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_18_of_100/test.log
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/authorization_logic_test/run_46_of_100/test.log
//src/frontends/arcs/xform_to_datalog:datalog_facts_test                 FAILED in 1 out of 100 in 0.1s
  Stats over 100 runs: max = 0.1s, min = 0.1s, avg = 0.1s, dev = 0.0s
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/datalog_facts_test/run_70_of_100/test.log
//src/frontends/arcs/xform_to_datalog:generate_datalog_program_test      FAILED in 3 out of 100 in 0.1s
  Stats over 100 runs: max = 0.1s, min = 0.0s, avg = 0.1s, dev = 0.0s
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_66_of_100/test.log
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_44_of_100/test.log
  /home/jopra/.cache/bazel/_bazel_jopra/ad57f266c437944f14d502fbbd0016f5/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/frontends/arcs/xform_to_datalog/generate_datalog_program_test/run_83_of_100/test.log

Executed 5 out of 5 tests: 1 test passes and 4 fail locally.
INFO: Build completed, 4 tests FAILED, 501 total actions
Cypher1 commented 2 years ago

Looking at the failures now:

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //src/frontends/arcs/xform_to_datalog:datalog_facts_test
-----------------------------------------------------------------------------
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20220708 03:21:02.609727    12 authorization_logic_datalog_facts.cc:79] Check failed: file_stream.gcount() == filesize Failure reading bytes from the result file.
*** Check failure stack trace: ***

This may be a duplicate of #464