"Global.Unexpected_None" exception thrown on first invocation of GenProg; no exception thrown on subsequent invocations

[ChrisTimperley]

$ pip3 install --upgrade bugzoo
$ bugzoo source add https://github.com/squaresLab/genprog-code
$ bugzoo source add https://github.com/squaresLab/ManyBugs
$ bugzoo source update
$ bugzoo tool build -f genprog
$ bugzoo bug build -f manybugs:python:69224-69224
$ bugzoo container launch manybugs:python:69224-69224 --with genprog
> genprog configuration-default --skip-failed-sanity-tests --allow-coverage-fail
...
...
...
test: 284
cilRep: default.cache: saved
cachingRep: default.cache: saved
faultlocRep: default.cache: saved
cilrep done serialize
cilRep: stmt_count = 1062
cilRep: stmts in weighted_path = 215
cilRep: total weight = 9.4
cilRep: stmts in weighted_path with weight >= 1.0 = 1
cilRep: Modules/selectmodule.c (code bank/base file; atoms [1,1156])
cilRep: 1 file(s) total in representation
search: genetic algorithm begins (|original| = 0.493019 MB)
cilRep: shared size: 0.558815 MB
cilRep: private size: 0.0123653 MB
search: initial population (sizeof one variant = 0.49304 MB)
genprog aborting: Global.Unexpected_None
Rep: saving test cache
Variant Test Case Queries: 0
"Test Suite Evaluations": 0

Compile Failures: 0
Wall-Clock Seconds Elapsed: 1311.11
Activity Name                     Count Seconds = Percent of Total Time
  compile                               2 104.407 = 7.97419%
  test                                570 1204.842 = 92.0208%
  TOTAL                                   1309.315 = 99.8625% (avg CPU usage)
Fatal error: exception Global.Unexpected_None

If the user executes the same command once again, then no exception is thrown and the repair process proceeds as expected.

[joerenzullo]

I got the same error with a different Python bug this afternoon - 69934-69935

[yodahaji]

I got same error with fedora 29 but on every invocation of genprog.

[pdreiter]

I ran into this same issue with the $GITROOT/tests/gcd-test [$GITROOT was directly cloned from head of this GenProg repo] I narrowed down this exact failure with the ga to when both *.cache and coverage.* files are removed. With the following Makefile text, it can be duplicated with: make repair_ga

Makefile.txt

This Makefile also uses a modified configuration that was recommended by Kevin Leach in an earlier conversation:

diff configuration configuration.brute 
5c5
< --search ga
---
> --search brute

and --search brute does work regardless of the state of file I/O.

DEBUG UPDATE: failing signature from rep.ml : 1698: let count = try TestMap.find test (fst eval_count) with _ -> 0 in

for both scenarios, !num_fitness_samples = 1 [global variable], but: In the passing scenario, TestMap.find gets an exception which renders count=0, ergo line 1700 False. In the failing scenario, count=1, ergo line 1700 True.

[pdreiter]

Identified the solution to this bug: Line 1159: [Minor Consistency issue] Ensured that Negative tests precede Positive test evaluations Line 1363: [Root-Cause of Global.Unexpected_None Exception] Ensures that 'cached information' infrastructure is reset before test_cases are evaluated. CAVEAT: There could be unintended side-effects to the complete reset of the cached information - BUT this is consistent with the brute force search, where eval_count is reset due to the calls to mutation operators. Minimal fix is just : eval_count <- TestMap.empty, 0 ; rather than self#update () ;

rep.ml.diff.txt

diff --git a/src/rep.ml b/src/rep.ml
index 11ac3c3..a07f020 100644
--- a/src/rep.ml
+++ b/src/rep.ml
@@ -1159,8 +1159,8 @@ class virtual ['gene,'code] cachingRepresentation = object (self : ('gene,'code)
           abort "cachingRepresentation: sanity check failed (compilation)\n" 
         end ;
         let tests =
-          (lmap (fun i -> (Positive i, (fun b -> not b))) (1 -- !pos_tests))
-            @ (lmap (fun i -> (Negative i, (fun b -> b))) (1 -- !neg_tests))
+          (lmap (fun i -> (Negative i, (fun b -> b))) (1 -- !neg_tests))
+            @ (lmap (fun i -> (Positive i, (fun b -> not b))) (1 -- !pos_tests))
         in
           liter (fun (t, failed) ->
             let name = test_name t in
@@ -1363,6 +1363,7 @@ class virtual ['gene,'code] cachingRepresentation = object (self : ('gene,'code)
       (e.g., [Unix.fork]), or if any of its own Unix system calls (such as
       [create_process] or [wait]) fail *)
   method test_cases tests =
+    self#updated () ;
     if !fitness_in_parallel <= 1 || (List.length tests) < 2 then
       (* If we're not going to run them in parallel, then just run them
        * sequentially in turn. *)

----- UPDATE ----- Limited testing seemed to show that this change worked.
However, during testing, I found something weird: $GITROOT/tests/gcd-test with --search ga does not return a repair solution with and without the proposed change.

[jlacomis]

Fixed by #14