Closed PEZ closed 2 years ago
Here's a blog post where I try to summarize the issue with the performance-flicker: https://blog.agical.se/en/posts/java-bitset-performance-mystery
I've figured out where the flickery performance of the bitset version came from. Or, well, not I, but with some help over at HN it became clear that sometimes the JVM process and the memory heap allocation happens on different NUMA nodes when run in docker on the drag-racing machines. This costs a lot in performance. I've updated the run script to try pin the JVM and the heap allocation to the same node. This will only work if the docker container is run in privileged mode, which I don't know if it does?
How the containers are run is technically a matter of public record, in the sense that it's documented in the source code of the tools/src
directory within the repo. Without looking. my guess would be that the containers are not run in privileged mode. That would be the logical default coming from this project's "security first" premise.
Checked the source now. You seem to assume correctly. I can't see the --priviledged
option being added.
That will mean that the performance of this solution's bitset variant will flicker between it's supposed levels and 0.5X of that, because the numactl
command will fail.
I'll bring this issue up in the Discussions forum, maybe we can come up with a solution.
@rbergen I might have a solution that does not require privileged mode. Putting this in draft mode until I have tested it.
@PEZ, do you have an expectation concerning the time it will take you to test this? The reason I'm asking is that generally speaking, we don't use draft PRs in this project. If work still needs to be done that blocks the processing of a PR that's been opened, we prefer to close it and then have it reopened when it's ready (again).
Roger that, @rbergen. This is ready for your review now.
Description
Adds a solution using a regular
java.util.BitSet
for storage.It exhibits a strange behaviour when run in Docker on Linux, where some of the runs perform at only 0.5X. (Same as I've seen it happen with Clojure solutions using Java BitSet.)
Contributing requirements
drag-race
as the target branch.