time limit per turn?

[LaTrissTitude]

title

[nobody5050]

Most likely no, since once you go beyond about 15 strategies, the simulation starts taking a while to complete anyways. I’d assume as long as you don’t intentionally timeout the simulation it should be fine

[LaTrissTitude]

yep, but it's important for predictive and simulating strategies to avoid taking too long to know what is too long

[jherndon8]

From the website:

-I will read through each .py file before running it, and I will run each .py file alongside a dummy player before entering it in the arena. This is to protect against infinite loops and malicious exploits. -If I have any suspicion that you're trying to break the rules in any way, by using disallowed libraries, or trying to read or write any files, or being intentionally inefficient with loops or sleep() commands, I reserve the right to disqualify you from the tournament.

Looks like if it's intentionally slow then you'll be disqualified, but if there's a rhyme or reason to why you're doing a bunch of calculations it should be fine.

[LaTrissTitude]

that doesn't take into account simulation based strategies, which do use loops but can be stopped after a specific amount of iterations or a maximum timeout

[nobody5050]

Those strategies would probably be frowned along unless they used statistics. See #11

[LaTrissTitude]

? Why would they, those kind of strategies do not use any magical knowledge or langage specific knowledge like #11 would use. I'm talking rather classical adversarial algorithms (min max, monte carlo, whatever), they only use statistics, they just generate some using predictive strategies

[nobody5050]

oh

[jherndon8]

I'd probably set what you think would be a reasonable iteration number or timeout for going against a few hundred strategies and leave a comment in your code saying "Edit this parameter if it's taking too long."

[zelosos]

Since the compute time of the hole tournament increases quadratic by amount of strats. A limit of 2 sec per turn should be more then enough. Remember he has to run this multiple (100 or more) times to get a scientific result.

[LaTrissTitude]

oh I'm more around around 2ms per round right now. Btw, if the dev is reading this, think about parallelizing the matches using the multiprocessing base library, you will reduce drastically the computation time. the tqdm library can also give a pretty useful loading bar to watch the progress and an ETA :p

[carykh]

Oh shoot, this is a good discussion topic, and not one I'd thought enough about before! Yeah, if you're at 2ms per round, then you should be fine. I guess I would be concerned if you went over 10ms or so?

Also, I really should consider parallelizing matches. If it starts to take a while a 15 submissions, and it's O(n^2) to run a tournament, and there's 400 submissions now, that is a bit of a problem! But multiple threads should help out with that.

[TimStanglow]

Honestly, what would anybody write that takes 2ms per call? Even a way too large NN for this purpose runs in 3.584e-05 s per call (18x faster than 2 ms).

[nobody5050]

If it starts to take a while a 15 submissions, and it's O(n^2) to run a tournament

by “a while” I mean about 5 seconds. Not a super long time but it could really increase once you have a few hundred

[LaTrissTitude]

Honestly, what would anybody write that takes 2ms per call? Even a way too large NN for this purpose runs in 3.584e-05 s per call (18x faster than 2 ms).

Won't go into the details and I optimized my strategy near .8 ms ( 8e-04 s ) since then, hence approx 0.2s for a 200 rounds game.

But tbh the comparison is not fair, neural networks are pretty fast to run, of course a simple feed forward calculation won't need much time to run. Their only real expensive time cost being the training time.

If you need examples of time constrained strategy calculation algorithms, take a look at the way chess is dealt with using monte carlo simulations or genetic algorithms, codingame has a few pretty cool problems you can solve with those approaches

[LaTrissTitude]

Oh shoot, this is a good discussion topic, and not one I'd thought enough about before! Yeah, if you're at 2ms per round, then you should be fine. I guess I would be concerned if you went over 10ms or so?

Also, I really should consider parallelizing matches. If it starts to take a while a 15 submissions, and it's O(n^2) to run a tournament, and there's 400 submissions now, that is a bit of a problem! But multiple threads should help out with that.

Also threads won't do much, this is CPU bound, threads in base python cannot be parallelized due to the GIL, you will need multiple processes in order to accelerate the computations.

see https://docs.python.org/3/library/multiprocessing.html