search

ivmai / bdwgc

The Boehm-Demers-Weiser conservative C/C++ Garbage Collector (bdwgc, also known as bdw-gc, boehm-gc, libgc)
https://www.hboehm.info/gc/
Other
2.86k stars 397 forks source link

gctest or threadleaktest hang sometimes on Ubuntu/x64 if --enable-gc-debug --enable-redirect-malloc #288

Open ivmai opened 5 years ago

ivmai commented 5 years ago

Build link: https://travis-ci.org/ivmai/bdwgc/jobs/551952193 Host: Linux/x64 (Xenial) Source: https://github.com/ivmai/bdwgc/commit/d7e2c143c740a93cea9acb397c2e3452327eb0ec (release-7.6) How to reproduce: configure --enable-redirect-malloc --enable-gc-debug --enable-cplusplus --enable-gc-assertions && make check

ivmai commented 4 years ago

Latest failed build: https://travis-ci.org/ivmai/bdwgc/jobs/576674642

ivmai commented 4 years ago

Unable to reproduce locally.

ivmai commented 4 years ago

Not observed on other GC versions (including master). Had not been observed before update to Travis Xenial image.

ivmai commented 4 years ago

Not observed with --disable-static

ivmai commented 1 year ago

Source: master (3f419b6) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/604837855 Host: Ubuntu 22.04.2 LTS / x64 Compiler: gcc 11.3.0 How to build: ./configure --enable-redirect-malloc --enable-gc-debug --enable-cplusplus --enable-gc-assertions && make -j check CFLAGS_EXTRA="-D _FORTIFY_SOURCE=2" Occurrence: very rare

ivmai commented 11 months ago

Source: master (b678225) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/607405872 Host: Ubuntu 22.04.2 LTS / x64

ivmai commented 10 months ago

Source: master (5839684) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/608674865 Host: Linux/amd64 Config: CFLAGS_EXTRA="-D _FORTIFY_SOURCE=2" CONF_OPTIONS="--enable-redirect-malloc --enable-gc-debug --enable-cplusplus --enable-gc-assertions"

ivmai commented 10 months ago

Hang in threadleaktest. Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/609180209 Source: master (6f8349a) Host: Ubuntu 22.04.2 LTS / amd64 (Jammy) Compiler: gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0 Config: CFLAGS_EXTRA="-D _FORTIFY_SOURCE=2" CONF_OPTIONS="--enable-redirect-malloc --enable-gc-debug --enable-cplusplus --enable-gc-assertions"

ivmai commented 9 months ago

Hang in threadleaktest. Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/611184349 Source: master (3ce1bb6) Host: Ubuntu 22.04.2 LTS / amd64 (Jammy) Compiler: gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0 Config: CFLAGS_EXTRA="-D _FORTIFY_SOURCE=2" CONF_OPTIONS="--enable-redirect-malloc --enable-gc-debug --enable-cplusplus --enable-gc-assertions"

ivmai commented 8 months ago

Source: master (43ef26f) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/611289500 same config as above

ivmai commented 8 months ago

Source: master (88e6950) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/612221909

ivmai commented 7 months ago

Source: master (f2630a4) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/613268276

ivmai commented 7 months ago

Source: master (027a48e) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/614378187

ivmai commented 6 months ago

Source: master (71d1ba4) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/615519763 Hang: threadleaktest

ivmai commented 6 months ago

Source: master (1ccba75) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/615657904 Hang: threadleaktest

ivmai commented 5 months ago

Source: master (6f6dc057f432) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/617007502

ivmai commented 4 months ago

Source: master (7fc3e07f0) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/618348900 Hang: threadleaktest

ivmai commented 4 months ago

The hang happens in GC_set_fl_marks because of a cycle in freelist. The root cause of the cycle is still unclear.

ivmai commented 4 months ago

Source: master (a85c6b186) Build: https://github.com/ivmai/bdwgc/actions/runs/8132977608/job/22224018072#step:5:476 Output: Assertion failure: /home/runner/work/bdwgc/bdwgc/extra/../alloc.c:1021

ivmai commented 4 months ago

Source: master (0c88daa55ac8) Reproduce locally (in threadleaktest): ./configure --enable-redirect-malloc --enable-gc-debug --enable-cplusplus --enable-gc-assertions && make -j check Occurrence: ~1/180

Also reproduced with GC_MARKERS=2 (but not 1): ./configure --enable-werror --enable-gc-assertions --disable-disclaim --disable-atomic-uncolectable --disable-handle-fork --disable-gcj-support --disable-thread-local-alloc --enable-gc-debug --enable-redirect-malloc && make -j check CFLAGS_EXTRA="-O0 -g -D GC_DISABLE_INCREMENTAL" GC_LOOP_ON_ABORT=1 GC_MARKERS=2 ./threadleaktest Occurrence: ~1/10

ivmai commented 3 months ago

Reproduced locally even after commit b37ca67cd3

ivmai commented 3 months ago

Source: master (b37ca67cd) Build: https://github.com/ivmai/bdwgc/actions/runs/8339128168/job/22820588748 Command: zig/zig build -DBUILD_SHARED_LIBS=true -Ddisable_handle_fork=false -Denable_gc_assertions=true -Denable_gc_debug=true -Denable_large_config=false -Denable_munmap=true -Denable_parallel_mark=true -Denable_redirect_malloc=true -Denable_rwlock=true -Denable_thread_local_alloc=true -Denable_threads=true -Denable_werror test

ivmai commented 2 months ago

Source: master (df1a787) Build: https://app.travis-ci.com/github/ivmai/bdwgc/jobs/620625409

ivmai commented 1 month ago

Identification of places to examine:

diff --cc malloc.c
index e3e17e76,e3e17e76..3f8bcaa5
--- a/malloc.c
+++ b/malloc.c
@@@ -193,7 -193,7 +193,7 @@@
        if (NULL == op) return NULL;
      }
    }
--  *opp = obj_link(op);
++  *opp = obj_link(op); //??
    obj_link(op) = NULL;
    GC_bytes_allocd += GRANULES_TO_BYTES((word)lg);
    return op;
@@@ -347,7 -347,7 +347,7 @@@ GC_INNER void * GC_malloc_kind_aligned_
              GC_ASSERT(PTRFREE == k || NULL == obj_link(op)
                        || ((word)obj_link(op) < GC_greatest_real_heap_addr
                            && (word)obj_link(op) > GC_least_real_heap_addr));
--            *opp = obj_link(op);
++            *opp = obj_link(op); //??
              if (k != PTRFREE)
                  obj_link(op) = NULL;
              GC_bytes_allocd += GRANULES_TO_BYTES((word)lg);
@@@ -407,8 -407,8 +407,8 @@@ GC_API GC_ATTR_MALLOC void * GC_CALL GC
          opp = &GC_obj_kinds[k].ok_freelist[lg];
          op = *opp;
          if (EXPECT(op != NULL, TRUE)) {
--            *opp = obj_link(op);
--            obj_link(op) = 0;
++            *opp = obj_link(op); //??
++            obj_link(op) = NULL;
              GC_bytes_allocd += GRANULES_TO_BYTES((word)lg);
              /* Mark bit was already set on free list.  It will be       */
              /* cleared only temporarily during a collection, as a       */
@@@ -630,7 -630,7 +630,7 @@@ static void free_internal(void *p, hdr 
      }

      flh = &(ok -> ok_freelist[lg]);
--    obj_link(p) = *flh;
++    obj_link(p) = *flh; //???
      *flh = (ptr_t)p;
    } else {
      if (lb > HBLKSIZE) {
diff --cc mallocx.c
index 8f142786,8f142786..f8576b13
--- a/mallocx.c
+++ b/mallocx.c
@@@ -321,12 -321,12 +321,12 @@@ GC_API void GC_CALL GC_generic_malloc_m
                    GC_release_mark_lock();
                }
  #           endif
--            op = GC_reclaim_generic(hbp, hhdr, lb_adjusted,
++            op = GC_reclaim_generic(hbp, hhdr, lb_adjusted,  //???
                                      ok -> ok_init, 0, &my_bytes_allocd);
--            if (op != 0) {
++            if (op != NULL) {
  #             ifdef PARALLEL_MARK
                  if (GC_parallel) {
--                  *result = op;
++                  *result = op; //?? should be holding lock?
                    (void)AO_fetch_and_add(&GC_bytes_allocd_tmp,
                                           (AO_t)my_bytes_allocd);
                    GC_acquire_mark_lock();
@@@ -380,7 -380,7 +380,7 @@@
          for (p = op; p != NULL; p = obj_link(p)) {
            my_bytes_allocd += lb_adjusted;
            if ((word)my_bytes_allocd >= HBLKSIZE) {
--            *opp = obj_link(p);
++            *opp = obj_link(p); //??
              obj_link(p) = NULL;
              break;
            }
@@@ -403,11 -403,11 +403,9 @@@
                ++GC_fl_builder_count;
                UNLOCK();
                GC_release_mark_lock();
--
--              op = GC_build_fl(h, lw,
++              op = GC_build_fl(h, lw, //???
                                 ok -> ok_init || GC_debugging_started, 0);
--
--              *result = op;
++              *result = op; //?? should be hold lock during result write?
                GC_acquire_mark_lock();
                --GC_fl_builder_count;
                if (GC_fl_builder_count == 0) GC_notify_all_builder();
@@@ -427,7 -427,7 +425,7 @@@
      if (op != NULL) obj_link(op) = NULL;

    out:
--    *result = op;
++    *result = op; /* store the pointer while holding the allocator lock */
      UNLOCK();
      (void)GC_clear_stack(0);
  }
diff --cc new_hblk.c
index 23ffdf11,23ffdf11..d24e74bc
--- a/new_hblk.c
+++ b/new_hblk.c
@@@ -142,7 -142,7 +142,7 @@@ GC_INNER ptr_t GC_build_fl(struct hblk 
    /* Make a list of all objects in *h with head as last object. */
      for (p = prev + lw; (word)p <= (word)last_object; p += lw) {
        /* current object's link points to last object */
--        obj_link(p) = (ptr_t)prev;
++        obj_link(p) = (ptr_t)prev; //???
          prev = p;
      }
      p -= lw;    /* p now points to last object */
@@@ -168,7 -168,7 +168,7 @@@ GC_INNER void GC_new_hblk(size_t lg, in
    if (IS_UNCOLLECTABLE(k)) GC_set_hdr_marks(HDR(h));

    /* Build the free list.       */
--  GC_obj_kinds[k].ok_freelist[lg] =
++  GC_obj_kinds[k].ok_freelist[lg] = //???
          GC_build_fl(h, GRANULES_TO_WORDS(lg),
                      GC_debugging_started || GC_obj_kinds[k].ok_init,
                      (ptr_t)GC_obj_kinds[k].ok_freelist[lg]);
diff --cc reclaim.c
index 9f30a43e,9f30a43e..5bc594ac
--- a/reclaim.c
+++ b/reclaim.c
@@@ -197,7 -197,7 +197,7 @@@ STATIC ptr_t GC_reclaim_clear(struct hb
              p += sz;
          } else {
              /* The object is available - put it on list. */
--            obj_link(p) = list;
++            obj_link(p) = list; //??
              list = p;
              FREE_PROFILER_HOOK(p);
              p = (ptr_t)GC_clear_block((word *)p, sz, pcount);
@@@ -226,7 -226,7 +226,7 @@@ STATIC ptr_t GC_reclaim_uninit(struct h
          if (!mark_bit_from_hdr(hhdr, bit_no)) {
              n_bytes_found += sz;
              /* The object is available - put it on list. */
--            obj_link(p) = list;
++            obj_link(p) = list; //??
              list = p;
              FREE_PROFILER_HOOK(p);
          }
@@@ -350,8 -350,8 +350,7 @@@ STATIC void GC_reclaim_small_nonempty_b
      } else {
          struct obj_kind *ok = &GC_obj_kinds[hhdr -> hb_obj_kind];
          void **flh = &(ok -> ok_freelist[BYTES_TO_GRANULES(sz)]);
--
--        *flh = GC_reclaim_generic(hbp, hhdr, sz, ok -> ok_init,
++        *flh = GC_reclaim_generic(hbp, hhdr, sz, ok -> ok_init, //??
                                    (ptr_t)(*flh), (word *)&GC_bytes_found);
      }
  }
ivmai commented 1 month ago

Source: master (8915b63d) Build: https://github.com/ivmai/bdwgc/actions/runs/9357520555/job/25768616825#step:4:511 Output (threadleaktest): Assertion failure: /home/runner/work/bdwgc/bdwgc/extra/../alloc.c:1025