Open GoogleCodeExporter opened 9 years ago
Maxime: I love when people attach a patch with a change request. You truly rock.
I strongly support using CPU time instead of wall time. If we can make it work,
it would be great. I have a few questions:
(1) Are you sure that this patch does limit the bots according to CPU time?
It's not clear to me that's what's going on. There are two processes involved
here: the engine and the bot. Whose CPU time is actually being measured in the
patch, the engine's or the bot's? We have to make sure that we're not limiting
the bot to 1 second of the engine's CPU time, as that would produce seemingly
arbitrary time limits.
(2) There are two engines in the codebase. One is the Java engine that is used
by the contestants to test their code. The other is located at
planet_wars/backend/engine.py and is used server-side to run the tournaments
amongst all the submitted bots. Time control is not very important in the Java
engine, it is more important in the Python engine. I don't know it it's
possible to accurately measure the CPU time being consumed by a Python
subprocess. I will have to look into this.
If somebody (probably Maxime) can produce a proof-of-concept Python script that
accurately measures the CPU time being consumed by a Python subprocess and cuts
it off after one second, then I think we can go ahead and switch to CPU time
instead of wall time.
Original comment by cameron.jp@gmail.com
on 29 Aug 2010 at 5:42
I have added some experimental Python code under
experimental/python_subprocess_cpu_time. master.py starts up client.py as a
subprocess. After 1 second of the master's CPU time, the master shuts down the
subprocess then terminates. The trick here is to change this so that the master
shuts down after 1 second of the client process' CPU time WITHOUT modifying
client.py.
Can it be done? Probably. I just don't know how yet. There is probably some way
to query how much CPU time has been used by a process based on its PID. Within
master.py, you can get the client's PID using "process.pid". If there is no
Python API function for doing this, there is probably a shell command for it.
You can use the subprocess module to launch that shell command with the
appropriate arguments, then parse the response. It's dirty, but it just might
work.
Original comment by cameron.jp@gmail.com
on 29 Aug 2010 at 7:38
I just thought of a potential problem with using CPU time to limit the bots.
Someone could attack the contest by submitting a bot that does this:
while (1) {
sleep(999);
}
AFAIK this bot would run pretty much forever while using a negligible amount of
CPU time. In order to guard against this we would have to keep the wall-time
limit in addition to the CPU time limit. However, we could loosen the wall-time
limit.
Original comment by cameron.jp@gmail.com
on 29 Aug 2010 at 7:43
[deleted comment]
May I make a small suggestion?
The current code in experimental/python_subprocess_cpu_time/master.py relies on
forking, which is expensive. The code attached to this post does the same but
without any forking at all. It relies on two libraries: twisted (For the async
process management) and psutil (Used to manage the processes and get the cpu
time).
It spawns the process and then polls the cpu time (user+kernel time) every 0.1
seconds. This is preferable to a "while time.clock()" type loop because it is a
lot more efficient, and the current code can manage any number of processes.
You can grab psutil here: http://code.google.com/p/psutil/ and twisted here:
http://twistedmatrix.com/ - or just use pip/easy_install.
Another advantage to this is you can use psutil to poll the total memory the
program is using, and perhaps kill it if it is too excessive.
Original comment by sharpbla...@gmail.com
on 14 Sep 2010 at 11:56
Attachments:
sharpblade1: that looks pretty awesome. I see you're quite active on the
project in the last day or so. Keep it up! I have to fix other issues ATM, but
I will get back to this one, promise!
Original comment by cameron.jp@gmail.com
on 14 Sep 2010 at 11:47
My concern with CPU time instead of wall-time is that I want to be able to
craft my bot so that it opportunistically does more work if more time is
available, or at the very least to have some way to interrupt work if it is
taking too long for some reason.
How would I be able to tell how much time I have used/remaining?
Original comment by hage...@gmail.com
on 28 Sep 2010 at 11:34
would it be possible to make sure each bot gets a separate core and then wall
time can be used to limit them fairly?
Original comment by ben...@gmail.com
on 10 Oct 2010 at 10:45
Please fix this defect. Because of it bots lose battles with timeout.
My bot finishes the turn in 0.8 sec and that's still not enough!
When this was set to 0.9 I lost %90 of battles with timeout.
With 0.8 sec I lose 10% of battles with timeout.
I use gettimeofday() function to measure time.
Original comment by buratin....@gmail.com
on 29 Oct 2010 at 7:16
Original issue reported on code.google.com by
Maxim...@gmail.com
on 27 Aug 2010 at 1:14Attachments: