PlummersSoftwareLLC / Primes

Prime Number Projects in C#/C++/Python
https://plummerssoftwarellc.github.io/PrimeView/
2.46k stars 573 forks source link

Java solution 1 - add BitSet variant #824

Closed PEZ closed 2 years ago

PEZ commented 2 years ago

Description

Adds a solution using a regularjava.util.BitSet for storage.

It exhibits a strange behaviour when run in Docker on Linux, where some of the runs perform at only 0.5X. (Same as I've seen it happen with Clojure solutions using Java BitSet.)

Contributing requirements

PEZ commented 2 years ago

Here's a blog post where I try to summarize the issue with the performance-flicker: https://blog.agical.se/en/posts/java-bitset-performance-mystery

PEZ commented 2 years ago

I've figured out where the flickery performance of the bitset version came from. Or, well, not I, but with some help over at HN it became clear that sometimes the JVM process and the memory heap allocation happens on different NUMA nodes when run in docker on the drag-racing machines. This costs a lot in performance. I've updated the run script to try pin the JVM and the heap allocation to the same node. This will only work if the docker container is run in privileged mode, which I don't know if it does?

rbergen commented 2 years ago

How the containers are run is technically a matter of public record, in the sense that it's documented in the source code of the tools/src directory within the repo. Without looking. my guess would be that the containers are not run in privileged mode. That would be the logical default coming from this project's "security first" premise.

PEZ commented 2 years ago

Checked the source now. You seem to assume correctly. I can't see the --priviledged option being added.

That will mean that the performance of this solution's bitset variant will flicker between it's supposed levels and 0.5X of that, because the numactl command will fail.

I'll bring this issue up in the Discussions forum, maybe we can come up with a solution.

PEZ commented 2 years ago

@rbergen I might have a solution that does not require privileged mode. Putting this in draft mode until I have tested it.

rbergen commented 2 years ago

@PEZ, do you have an expectation concerning the time it will take you to test this? The reason I'm asking is that generally speaking, we don't use draft PRs in this project. If work still needs to be done that blocks the processing of a PR that's been opened, we prefer to close it and then have it reopened when it's ready (again).

PEZ commented 2 years ago

Roger that, @rbergen. This is ready for your review now.