Benchmark Rules violation — not using bit per number?

[PaulCharlton]

I thought the requirements stated an array of bits, along with requisite bit-whacking needed. The code for zig referenced below is using u8 instead of u1 …

https://github.com/PlummersSoftwareLLC/Primes/blob/e8a223d567932e026b268c4d0a4de28a43e7052b/PrimeZig/src/prime.zig#L9

[rbergen]

@PaulCharlton You're commenting on a file in the original branch. The drag race rules apply to the solutions in the drag-race branch, which has been the default branch for about 3 years now.

Even within the boundaries of those rules (which, again, don't apply to the file in question) it's possible/allowed to use more than a single bit for prime candidate storage, as long as the solution documents it does so.

The previous makes that I'm going to close this issue as "won't fix".

[PaulCharlton]

got it. thank you. The reason I wrote is that I just saw the video, in which I recall Dave mentioning in the video the 1-bit per flag rule, and requiring that for all implementations -- seems like an absolute requirement for apples-to-apples comparison as bit-whacking is more computationally intense than just read/write a byte. After the video's proclamation that Zig was the absolute hands down winner, I went looking for the code, and the shift to a byte from a bit is an obvious reason for performance gain that was not mentioned in Dave's video.

In any event, after your note, I did see the following, which makes it overall less interesting unless the reported stats are also clustered by "flag size".

https://github.com/PlummersSoftwareLLC/Primes/blob/6980b4d355deaaa16dedd82b2069a03920a58f6a/CONTRIBUTING.md?plain=1#L317

[rbergen]

If you take a look at one of the daily benchmark reports (like this one), you'll see that the results are registered together with the corresponding implementation's key properties, and can be filtered using them:

• Single-threaded or multi-threaded
• Variety of algorithm - base, wheel or other
• Faithful or not - as per the definition in the contributing guidelines
• Number of bits per prime candidate

The link I included also expands the filter presets panel, which shows that only the solutions that are faithful, implement the base algorithm and really use 1 bit per prime candidate are considered for the leader boards. Of which there are two: one for single-threaded and one for multi-threaded solutions.

So yes, the comparisons that are discussed in Dave's Garage videos do consider the number of bits per prime candidate as well as some other relevant characteristics.