Include documentation on how to prevent monkey patching for a given language

[10XL]

This is for Ruby but I'm guessing it also applies to Python/Javascript. I've noticed a user overwriting comparison methods(and methods related to the return type of the method) to bypass test cases. I tested this myself and was able to pass the few katas I tried simply by monkey patching Object or NilClass methods. For some katas it's as simple as

class Object
  def ==(o)
    true
  end
end

or

class Object
  def !=(o)
    false
  end
end

See cw-2.rb#L112, related kumite Maybe core classes should be frozen like the Test class is, though that could interfere with katas that rely on the user monkey patching. A better solution might be to use untouched/unmodified comparison methods.

[kazk]

If I'm not misunderstanding, this is discussed in Codewars/codewars.com#214.

[DonaldKellett]

I've noticed a user overwriting comparison methods ... to bypass test cases.

@10XL How often is this user using the cheat you just mentioned? If he/she has recently been exploiting this loophole frequently for personal rank/honor gain, you may want to consider reporting the user in the Codewars.com repo like in the Issue @kazk just referenced.

[10XL]

@DonaldKellett I'm not sure how often but I've completed several katas around the top of the /kata list and I've seen the same user with this type of solution. I don't want to target the user so I'm not going to put their username here.

[kazk]

Do people consider cheating major issue?

From the author/translator's point of view, I do understand that it's frustrating to see a user bypassing all the work you put in. But I don't think it's realistic to keep patching for every cheat found. I know many ways to cheat because I've read most of the codebase, and anyone can do the same.

So I think detecting/flagging a cheater should be done by the community instead. I think those cheaters using multiple accounts are handled this way.

If the user is doing this repeatedly, maybe it's best to report the user privately through email for now.

[jhoffner]

I 100% agree with @kazk.

Trying to hunt down every way in which a cheat can be performed is going to be a never ending cat and mouse game. If anything we should open up a discussion about the best way to handle reporting of cheaters in an official manner. The platform already has the ability to mark specific solutions as cheats, and for users who have clearly cheated a lot, we can just mark them as a cheater, which results in them never being able to see any solutions at all plus their honor/rank is always 0/8kyu. Edit: Both of these features however are admin only functions.

In terms of things that should be fixed. My main concern is making sure things that are too easy to come across while NOT cheating are fixed, assuming they are someone common. The last thing we need are users accidentally passing challenges all of the time.

In this case, I think it's safe to assume that someone would be meaning to cheat.

[smiks]

I'm apologizing if you have already discussed this. I just joined the chat. I wrote 3 katas (2 still in beta) in python. I overwrote python functions and modules that could stop program from executing and make user pass kata. I most likely did not cover all possible ways but player who kept on "hacking" my kata stopped doing it after I did that. Is there a way to have this code (code which overwrites functions/modules) preloaded by default? This way number of cheaters would reduce.

[OverZealous]

I think that as we expand our ability to include external libraries, it would be nice to start putting together some open-source libraries that can be community maintained, which can be included as part of the test.

We haven't fully finalized how we want to support external libraries, but even now, it would be pretty easy to paste a shared library into the preloaded section.

This way, rather than requiring updating the code runner every time a new workaround or hack is discovered, the "cheat prevention utilities" libraries can be updated.

It also sort of leans toward an opt-in approach, so if you want to allow some behavior, you can, but we could even offer utility methods for locking down specific classes or functions that might only be used on a few types of challenges. (For example, locking out Array in JavaScript is common on some of the challenges that want you to recreate some part of Array's functionality, but obviously that's undesirable in most other kata.)

There's actually other features we offer within the code runner that aren't easily discovered, and this would allow us to expose those through easy-to-access methods as well.

[10XL]

Sometimes 'cheating' is the aim of a kata and still can be educational if it isn't. Normal(maybe an unlocked privilege) users should have an option to "vote cheat" a solution that deducts a point.

I might be straying off from a CLI issue here; I remember code being automatically submitted after pressing submit, presuming it passed. This was probably a bug but users weren't able to hide their cheating solution. However, now if you pass a kata and don't click submit final, you still unlock /solutions. A user could derive a 'legit' solution after looking at /solutions, submit that, and avoid his cheated solution from being displayed. This could be prevented by unlocking /solutions after code is actually submitted instead of only passing the tests.

[jhoffner]

A user could derive a 'legit' solution after looking at /solutions

I had locked this down so that you can't view solutions until you submit final. Is there a regression?

[10XL]

I just tested with an easy kata and was able to view solutions by going to it's url. The unlock solutions button didn't change to view solutions.

[smiks]

What about opt-out option when creating kata? This way it would be possible to have anit-cheat library (as OverZealous mentioned) and way to make 'cheating' kata (as 10XL mentioned). Simple check-box (selected by default) which marks if anti-cheat library is preloaded or not.

[smiks]

I tried solving 8kyu kata (invert values) (I'm 3kyu) for the first time. Before I solved it, I was not able to look at solutions. I solved it but have not yet submitted final version. After first submit I was able to see solutions. After that I submitted it and checked solutions again. I was still able to see them (expected) but I couldn't see my solution. Instead I got message: "There are no solutions yet for this kata.".

[kazk]

I don't think including this in the editor is a good idea and opt-out is problematic.

Simple check-box (selected by default) which marks if anti-cheat library is preloaded or not.

Sounds easy, but this requires knowing the state (exists/functioning) of the anti-cheat libraries for languages (currently 24) and adjust (hide/warn) depending on that.

---

@jhoffner can @config support specifying "Preloaded" code from downloaded github-repo/gist? I haven't looked into the code handling them yet.

[jhoffner]

@kazk preloaded code can't be downloaded from github/gist, but the files can which can then be referenced via the preloaded code.

[anter69]

Just dropping in here with my 2 cents: I came across the below cheat in python, and was wondering if such "simple" hacks could be avoided? Reading the thread I see some good ideas.

def blabla(*args, **kwargs):
    test.expect(True)
test.assert_equals = blabla

Is it possible to somehow protect or lock the test class? I would vote for an opt-out button in the kata editor, as mentioned above. Also, probably some other functions could be disabled by default, like exec() in python (also mentioned somewhere already).

Worst case, some anti-cheat measurements should be published in some easily accessible page(s), like the CW codex of @bkaestner

[smiks]

I put part of code in all my katas to work as "protection" against cheats.

This is what I use

""" hacker protection """
import sys
del sys.modules['sys']
sys.exit = lambda *x: print("Lame"); 
exit = lambda *x: print("Lame");
sys = None
assert_equals_123fffffffffffffffffffffffff = Test.assert_equals
expect_123fffffffffffffffffffffff` = Test.expect

[smiks]

I wrote new snippet to prevent cheating. I'm testing it here.