search

JuliaCI / julia-buildbot

Buildbot configuration for build.julialang.org
MIT License
18 stars 14 forks source link

Request: MEMDEBUG/Valgrind buildbot #53

Closed garrison closed 2 years ago

garrison commented 8 years ago

@staticfloat

It would be great to have a buildbot that builds with MEMDEBUG and MEMDEBUG2 and runs the tests under Valgrind, and possibly other sanitizers (separately) as well. Since this takes a long time, it should perhaps run infrequently (once a day or even less). There are already instructions for running julia under valgrind but I know less about the LLVM sanitizers. Hopefully this can help catch build errors with MEMDEBUG2 before they are nearly a year old (c.f. JuliaLang/julia#18536), and (more importantly) also help recognize when changes introduce things that valgrind considers to be memory errors.

tkelman commented 8 years ago

we have @yuyichao's gc-debug buildbot, how different would this be?

garrison commented 8 years ago

I've forgotten precisely how gc-debug works, but I can't imagine it's going to pick up everything that valgrind would. I think of gc-debug and MEMDEBUG/valgrind as being orthogonal to each other, and I believe it would be helpful for both buildbots to exist.

yuyichao commented 8 years ago

The gc-debug buildbots only makes sure the code compiles. Running the test with aggressive GC currently is still only done by myself locally. Such tests usually takes 10x-10000x longer to run and I'm not yet sure what's the most systematic way to do that automatically yet (We almost certainly need some fuzz test for this).

The two options are not exactly orthogonal with each other can have non-trivial interference in certain cases (i.e. there's cases where either option makes the other one more or less efficient) so it is useful to keep them both working.

DilumAluthge commented 2 years ago

On Buildkite, we now run some sanitizers, including asan and tsan.

PRs to add more sanitizers are welcome. The PRs should be made to the JuliaLang/julia repository. Look at e.g. https://github.com/JuliaLang/julia/blob/0ddba1572f70e39505c291e6ca682cbc75123de1/.buildkite/pipelines/main/misc/sanitizers.yml for an example.

garrison commented 2 years ago

The PRs should be made to the JuliaLang/julia repository.

I'm a little bit confused by the current status. With https://github.com/JuliaLang/julia/pull/43053 pending, should PRs be made to JuliaCI/julia-buildkite instead? Or should I wait for https://github.com/JuliaLang/julia/pull/43053 to be merged before submitting PRs to the julia-buildkite repository?

DilumAluthge commented 2 years ago

Sorry about the confusion! That's on me; I'm currently working on that transition, so things are in a state of flux.

You can go ahead and make a PR to JuliaCI/julia-buildkite, and we can review and merge it even before I finish up https://github.com/JuliaLang/julia/pull/43053.

Worst case, if something breaks during the transition process, we can revert your PR, fix the breakage, and then re-land your PR.