search

barneygale / mark2

minecraft server wrapper, written in python with twisted
Other
105 stars 27 forks source link

Twistd taking up excessive CPU #56

Closed vemacs closed 11 years ago

vemacs commented 11 years ago 
nullblock@ftb:~$ top
top - 03:47:27 up 1 day,  6:51,  1 user,  load average: 2.15, 1.97, 1.90
Tasks:  23 total,   2 running,  21 sleeping,   0 stopped,   0 zombie
Cpu(s): 27.5%us,  5.4%sy,  0.0%ni, 60.2%id,  0.0%wa,  0.0%hi,  0.0%si,  6.9%st
Mem:   2097152k total,  1496580k used,   600572k free,        0k buffers
Swap:        0k total,        0k used,        0k free,    74704k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
6433 nullbloc  20   0 1975m 1.3g 5432 S   71 64.2 158:05.14 java
20979 nullbloc  20   0 22952  10m 1784 R   64  0.5 737:37.59 twistd

Nothing much to say, except that twistd uses a lot of CPU, when mark2 is running.

edk0 commented 11 years ago

Are there any errors logged in /tmp/mark2/.log?

barneygale commented 11 years ago

Will attempt to reproduce today.

vemacs commented 11 years ago

@edk141 Nope. Looked thru it, all it mentions is the server starting and stopping.

This occurs on both our VPSes, one which is KVM, one which is OpenVZ, they're both running Ubuntu 12.04 LTS x64, one is running MCPC+, one is Spigot.

barneygale commented 11 years ago

Which reactor class does twisted use? Should be third line of log.

vemacs commented 11 years ago

@barneygale 

2013-04-14 21:35:14+0400 [-] reactor class: twisted.internet.pollreactor.PollReactor.

Is it possible to avoid enabling twisted-enabled components, if per-se you have IRC disabled?

Here's an uptime checking implementation without twisted: https://github.com/savi3000/redalert

barneygale commented 11 years ago

Twisted is used throughout - to wrap the MC process, to ping it, to provide the UI client and server, etc.

What OS is the VM running? Default for Linux is the epoll reactor which should be much faster.

vemacs commented 11 years ago

@BarneyGale I'm using Ubuntu 12.04.2 LTS x32.

edk0 commented 11 years ago

I doubt the OS is the cause then - plenty of people use mark2 on ubuntu without issue. what version of twisted do you have installed, and how did you get it? if it was through pip I might try removing it and installing it with apt-get instead. If not, I'll try to acquire that version of twisted and reproduce the issue locally.

fwiw, you don't need to mention people in replies, we get notifications for them anyway :)

vemacs commented 11 years ago

I ran:

sudo apt-get install python-dev python-twisted-core python-twisted-web python-twisted-words

during startup. I could try the pip install.

edk0 commented 11 years ago

what version of twisted are you running? my VPS (Xen) uses almost no CPU for either pip or apt-get versions, it's ubuntu 12.04 LTS. you can find out the twisted version with something like $ python -c "import twisted; print twisted.version".

barneygale commented 11 years ago

@edk141 Try forcing twisted to use the poll reactor rather than epoll - see if you can reproduce the problem. Twisted selecting poll is a bit weird.

edk0 commented 11 years ago 
$ twistd --reactor poll --pidfile /tmp/mark2/test.pid --logfile /tmp/mark2/test.log mark2 --shared-path /tmp/mark2 --server-name test --server-path /home/edk/test
$ ps aux | grep twistd
edk       3340  0.2  4.0  97552 20088 ?        S    14:53   0:00 /usr/bin/python /usr/local/bin/twistd --reactor poll --pidfile /tmp/mark2/test.pid --logfile /tmp/mark2/test.log mark2 --shared-path /tmp/mark2 --server-name test --server-path /home/edk/test

worse, but not 64%.

vemacs commented 11 years ago 
nullblock@ftb:~/660$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.2 LTS
Release:        12.04
Codename:       precise
nullblock@ftb:~/660$ python -c "import twisted; print twisted.version"
[twisted, version 11.1.0]
barneygale commented 11 years ago

Try upgrading to 12 or 13 (where epoll is default on linux)

vemacs commented 11 years ago

I've upgraded to 12.1.0 from the Twisted Ubuntu PPA, and the issue still occurs.

nullblock@ftb:~/660$ python -c "import twisted; print twisted.version"
[twisted, version 12.1.0]
vemacs commented 11 years ago

Per @barneygale , here is the binary profile I generated:

http://cl.ly/ORJk/profile.txt

Here are some strange profiles that replace the binary profile after a few seconds:

http://paste.ubuntu.com/5724993/

vemacs commented 11 years ago

Fixed it by adding:

mark2.service.query.enabled=false

In mark2.properties for the server folder. It's still a bug in mark2, so I'm not closing this.