codewars / content-issues

Higher level issue tracker for the Codewars content.
15 stars 1 forks source link

Harden tests of Triangular Treasure #194

Open hobovsky opened 1 year ago

hobovsky commented 1 year ago

As a follow-up of the deduplication issue #188, the kata Triangular Treasure should have its tests updated to allow only O(1) solutions.

List of languages to change:

monadius commented 1 year ago

I am not sure how to update C, C#, NASM, Rust. They all require 32-bit integer results. It is not reasonable to enforce O(n) solutions with this return type (more than 200_000 tests are required in NASM to eliminate O(n) solutions). Java already uses long for results. I think 64-bit integers should be used in all statically typed languages. Do you agree?

hobovsky commented 1 year ago

Yes, I think switching to 64 bit type is OK. For a brief moment i even considered testing for inputs which would trigger overflow if multiplied first and divided later, but I dont know if this would be still fair for 7 kyu.

akar-0 commented 11 months ago

Unless I am doing something bad or missing something, it seems impossible to make iterative solutions fail in Rust, even using i64, probably due to some compiler optimization, see this fork (random test with n = i32::MAX as i64, reference solution is iterative).

monadius commented 11 months ago

The Rust compiler knows how to compute the sum of a range efficiently. So the compiled code does not contain any loops and your solution is not iterative. Naive solutions (with for or while loops) are iterative and they will fail your tests if the exponent p is increased in random tests. See this Godbolt examples: https://godbolt.org/z/jYv38bs6Y

I forked your kumite and ran some tests. It seems that the range sum is not optimized on Codewars (which is strange because the docs show opt-level = 1). So it is enough to generate large values of n to fail iterative solutions:

let n = rng.gen_range(0..10_i64.pow(9));
akar-0 commented 11 months ago

Ok, thanks, I didn't know that.

KayleighWasTaken commented 10 months ago

Updated CoffeeScript tests.