search

mirage / irmin

Irmin is a distributed database that follows the same design principles as Git
https://irmin.org
ISC License
1.85k stars 157 forks source link

CI sometimes fails for irmin-git #641

Closed samoht closed 5 years ago

samoht commented 5 years ago

For instance:

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
⊘ removed   git.2.0.0
⊘ removed   duff.0.2
⊘ removed   hex.1.3.0
⊘ removed   cstruct.3.6.0
⊘ removed   irmin.dev
⊘ removed   sexplib.v0.10.0
⊘ removed   uri.2.2.0
⊘ removed   ppx_sexp_conv.v0.12.0
⊘ removed   ppxlib.0.5.0
⊘ removed   stdio.v0.12.0
⊘ removed   base.v0.12.0
⊘ removed   sexplib0.v0.12.0
∗ installed base64.3.1.0
∗ installed magic-mime.1.1.1
∗ installed mtime.1.1.0
∗ installed sexplib0.v0.11.0
∗ installed base.v0.11.1
∗ installed macaddr.3.0.0
∗ installed ipaddr.3.0.0
∗ installed parsexp.v0.11.0
∗ installed sexplib.v0.11.0
∗ installed cstruct.3.6.0
∗ installed duff.0.2
∗ installed hex.1.3.0
∗ installed stdio.v0.11.0
∗ installed ppxlib.0.5.0
∗ installed fieldslib.v0.11.0
∗ installed ppx_fields_conv.v0.11.0
∗ installed ppx_sexp_conv.v0.11.2
∗ installed uri.2.2.0
∗ installed cohttp.2.0.0
∗ installed cohttp-lwt.2.0.0
∗ installed conduit.1.4.0
∗ installed conduit-lwt.1.4.0
∗ installed conduit-lwt-unix.1.4.0
∗ installed cohttp-lwt-unix.2.0.0
∗ installed git.2.0.0
∗ installed git-http.2.0.0
∗ installed git-unix.2.0.0
∗ installed irmin.dev
∗ installed irmin-mem.dev
∗ installed uuidm.0.9.6
∗ installed alcotest.0.8.5
∗ installed irmin-test.dev
Done.
opam install irmin-git.dev "-t" "-v"
+opam install irmin-git.dev -t -v
<><> Synchronising pinned packages ><><><><><><><><><><><><><><><><><><><><><><>
[irmin-git.dev] synchronised from file:///home/travis/build/mirage/irmin
The following actions will be performed:
  ∗ install irmin-git dev*
<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+ /home/travis/.opam/opam-init/hooks/sandbox.sh "build" "dune" "subst" (CWD=/home/travis/.opam/ocaml-base-compiler.4.05.0/.opam-switch/build/irmin-git.dev)
+ /home/travis/.opam/opam-init/hooks/sandbox.sh "build" "dune" "build" "-p" "irmin-git" "-j" "1" (CWD=/home/travis/.opam/ocaml-base-compiler.4.05.0/.opam-switch/build/irmin-git.dev)
+ /home/travis/.opam/opam-init/hooks/sandbox.sh "build" "dune" "runtest" "-p" "irmin-git" (CWD=/home/travis/.opam/ocaml-base-compiler.4.05.0/.opam-switch/build/irmin-git.dev)
-         test alias test/irmin-git/runtest (exit 1)
- (cd _build/default/test/irmin-git && ./test.exe -q --color=always)
- Testing irmin-git.
- This run has ID `588A12A3-64FE-4835-B1F7-EF656F8238EA`.
[OK]                GIT                  0   Basic operations on contents.
[OK]                GIT                  1   Basic operations on nodes.
[OK]                GIT                  2   Basic operations on commits.
[OK]                GIT                  3   Basic operations on branches.
[OK]                GIT                  4   Watch callbacks and exceptions.
[OK]                GIT                  5   Basic operations on watches.
[OK]                GIT                  6   Basic merge operations.
[OK]                GIT                  7   Basic operations on slices.
[OK]                GIT                  8   Complex histories.
[OK]                GIT                  9   Empty stores.
[OK]                GIT                 10   Private node manipulation.
[OK]                GIT                 11   High-level store operations.
[OK]                GIT                 12   High-level operations on trees.
[OK]                GIT                 13   High-level store synchronisation.
[OK]                GIT                 14   High-level store merges.
[OK]                GIT                 15   Unrelated merges.
[OK]                GIT                 16   Low-level concurrency.
[OK]                GIT                 17   Concurrent updates.
[OK]                GIT                 18   with_tree strategies.
[OK]                GIT                 19   Concurrent head updates.
[OK]                GIT                 20   Concurrent merges.
[OK]                GIT.generic          0   Basic operations on contents.
[OK]                GIT.generic          1   Basic operations on nodes.
[OK]                GIT.generic          2   Basic operations on commits.
[OK]                GIT.generic          3   Basic operations on branches.
[OK]                GIT.generic          4   Watch callbacks and exceptions.
[OK]                GIT.generic          5   Basic operations on watches.
[OK]                GIT.generic          6   Basic merge operations.
[OK]                GIT.generic          7   Basic operations on slices.
[OK]                GIT.generic          8   Complex histories.
[OK]                GIT.generic          9   Empty stores.
[OK]                GIT.generic         10   Private node manipulation.
[OK]                GIT.generic         11   High-level store operations.
[OK]                GIT.generic         12   High-level operations on trees.
[OK]                GIT.generic         13   High-level store synchronisation.
[OK]                GIT.generic         14   High-level store merges.
[OK]                GIT.generic         15   Unrelated merges.
[OK]                GIT.generic         16   Low-level concurrency.
[OK]                GIT.generic         17   Concurrent updates.
[OK]                GIT.generic         18   with_tree strategies.
[OK]                GIT.generic         19   Concurrent head updates.
[OK]                GIT.generic         20   Concurrent merges.
[OK]                GIT.misc             0   Testing sort order.
[ERROR]             GIT.misc             1   Testing listing refs.
[OK]                GIT.import           0   git -> mem.
- -- GIT.misc.001 [Testing listing refs.] Failed --
- in /home/travis/.opam/ocaml-base-compiler.4.05.0/.opam-switch/build/irmin-git.dev/_build/default/test/irmin-git/_build/_tests/588A12A3-64FE-4835-B1F7-EF656F8238EA/GIT.misc.001.output:
- +2797502us git.mem    [INFO] Reset memory store
- +2797536us irmin      [DEBUG] set /test
- +2797543us irmin.git  [DEBUG] find refs/heads/master
- +2797551us git.mem    [DEBUG] Ref.read refs/heads/master
- +2797557us irmin      [DEBUG] Head.find -> 
- +2797563us irmin.tree [DEBUG] Tree.add_tree /test
- +2797570us irmin.git  [DEBUG] add toto
- +2797581us git.encoder [DEBUG] Starting to encode a Git object with a capacity = 256.
- +2797598us git.encoder [DEBUG] Ensure than we wrote exactly [shift = 11] byte(s).
- +2797607us irmin.git  [DEBUG] add [ { perm = normal; name = test;
-                                       node = b38e66e75f3535b9448e758fc0829438ecf4d6c7; } ]
- +2797636us git.encoder [DEBUG] Starting to encode a Git object with a capacity = 256.
- +2797645us git.encoder [DEBUG] Ensure than we wrote exactly [shift = 0] byte(s).
- +2797652us git.encoder [DEBUG] Ensure than we wrote exactly [shift = 32] byte(s).
- +2797658us git.encoder [DEBUG] Starting to encode a Git object with a capacity = 256.
- +2797669us git.encoder [DEBUG] Ensure than we wrote exactly [shift = 0] byte(s).
- +2797676us git.encoder [DEBUG] Ensure than we wrote exactly [shift = 40] byte(s).
- +2797682us irmin.tree [DEBUG] Tree.export -> 7a6fa9c85de9620c05654ddc4cc9be3659faa7f3
- +2797689us irmin.git  [DEBUG] add { tree = 7a6fa9c85de9620c05654ddc4cc9be3659faa7f3;
-                                     parents = [  ];
-                                     author = { name = ;
-                                                email = irmin@openmirage.org;
-                                                date = 0; };
-                                     committer = { name = ;
-                                                   email = irmin@openmirage.org;
-                                                   date = 0; }; extra = 
-                                     []; message = ; }
- +2797717us git.encoder [DEBUG] Starting to encode a Git object with a capacity = 256.
- +2797744us git.encoder [DEBUG] Ensure than we wrote exactly [shift = 127] byte(s).
- +2797751us git.encoder [DEBUG] Starting to encode a Git object with a capacity = 256.
- +2797777us git.encoder [DEBUG] Ensure than we wrote exactly [shift = 138] byte(s).
- +2797784us irmin.git  [DEBUG] test_and_set refs/heads/master
- +2797792us git.mem    [DEBUG] Ref.write refs/heads/master
- +2797799us irmin.git  [DEBUG] write_index
- +2797804us irmin.git  [DEBUG] write_index/if bare=false head=refs/heads/master
- +2797810us irmin.git  [DEBUG] write cache (refs/heads/master)
- +2797817us irmin.git  [DEBUG] find refs/heads/master
- +2797823us git.mem    [DEBUG] Ref.read refs/heads/master
- +2797829us irmin.git  [DEBUG] find 3bf0ff06cac7bc151a9797d9d9082e5729c042e8
- +2797835us irmin      [DEBUG] Head.find -> 3bf0ff06cac7bc151a9797d9d9082e5729c042e8
- +2797841us irmin.git  [DEBUG] set refs/remotes/datakit/master
- +2797848us git.mem    [DEBUG] Ref.write refs/remotes/datakit/master
- +2797854us irmin.git  [DEBUG] write_index
- +2797859us irmin.git  [DEBUG] write_index/if bare=false head=refs/remotes/datakit/master
- +2798138us irmin.git  [DEBUG] set refs/foo/bar/toto
- +2798145us git.mem    [DEBUG] Ref.write refs/foo/bar/toto
- +2798152us irmin.git  [DEBUG] write_index
- +2798157us irmin.git  [DEBUG] write_index/if bare=false head=refs/foo/bar/toto
- +2798163us irmin.git  [DEBUG] set refs/heads/foo
- +2798168us git.mem    [DEBUG] Ref.write refs/heads/foo
- +2798174us irmin.git  [DEBUG] write_index
- +2798179us irmin.git  [DEBUG] write_index/if bare=false head=refs/heads/foo
- +2798189us irmin.git  [DEBUG] list
- +2798195us git.mem    [DEBUG] Ref.list
- --------------------------------------------------------------------------------
- ASSERT raw branches
- --------------------------------------------------------------------------------
- +2798231us irmin.git  [DEBUG] list
- +2798237us git.mem    [DEBUG] Ref.list
- --------------------------------------------------------------------------------
- ASSERT filtered branches
- --------------------------------------------------------------------------------
- [failure] Error filtered branches: expecting
- [master; foo], got
- [].
- 
- The full test results are available in `/home/travis/.opam/ocaml-base-compiler.4.05.0/.opam-switch/build/irmin-git.dev/_build/default/test/irmin-git/_build/_tests/588A12A3-64FE-4835-B1F7-EF656F8238EA`.
- 1 error! in 2.400s. 45 tests run.
[ERROR] The compilation of irmin-git failed at
        "/home/travis/.opam/opam-init/hooks/sandbox.sh build dune runtest -p
        irmin-git".
#=== ERROR while compiling irmin-git.dev ======================================#
# context     2.0.3 | linux/x86_64 | ocaml-base-compiler.4.05.0 | pinned(file:///home/travis/build/mirage/irmin)
# path        ~/.opam/ocaml-base-compiler.4.05.0/.opam-switch/build/irmin-git.dev
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune runtest -p irmin-git
# exit-code   1
# env-file    ~/.opam/log/irmin-git-14616-05296d.env
# output-file ~/.opam/log/irmin-git-14616-05296d.out
### output ###
# [...]
# --------------------------------------------------------------------------------
# +2798231us irmin.git  [DEBUG] list
# +2798237us git.mem    [DEBUG] Ref.list
# --------------------------------------------------------------------------------
# ASSERT filtered branches
# --------------------------------------------------------------------------------
# [failure] Error filtered branches: expecting
# [master; foo], got
# [].
# 
# The full test results are available in `/home/travis/.opam/ocaml-base-compiler.4.05.0/.opam-switch/build/irmin-git.dev/_build/default/test/irmin-git/_build/_tests/588A12A3-64FE-4835-B1F7-EF656F8238EA`.
# 1 error! in 2.400s. 45 tests run.

Not sure how this is possible but might need some investigation.

dinosaure commented 5 years ago

If I want to find an semi-reproducible error about that, it should be here where the test does not continue to use repo, it wants to retake it from a configuration and expect that it was not deleted by the GC - and by this way checks already registered references on it.

I suspect that store was deleted and unfortunately, we create a new one with, obviously, empties references.

samoht commented 5 years ago

Good catch. We should not use ephemerons for in-memory stores /cc @pascutto