Solve pyhton3 full CPU

[G4br1elsec]

At the beginning of the game, the python3 service will take up the entire CPU, resulting in slow access, please modify the code to use multi-cores to ensure fast running

[eljeffeg]

Can you explain what is happening - what pages are being loaded at what frequency, how many users? That's abnormal behavior and I can't reproduce it.

[eljeffeg]

Also, is this the docker container or full install. What DB are you using?

[G4br1elsec]

I don't know what happened, but when the game started, the CPU was running full load, with 20 users, docker container, sqlist database。Use git clone for this project , docker one-button installation

[G4br1elsec]

您能解释发生了什么吗？以什么频率加载了哪些页面，有多少用户？那是异常行为，我无法重现。

Excuse me, I have reported this situation, can you help me solve this problem? Because now do external projects with your source code。using docker

[eljeffeg]

I'm going to need more information to help. Let's start with how you're monitoring as it looks like top. Is that run inside the docker container? If you're running it on the host, that's not likely anything to do with RootTheBox. To check the stats in a docker container, you'll want to run: docker stats CONTAINER

docker top CONTAINER [ps OPTIONS]

top will show docker-compose

As far as I know, Python runs in its own image within docker. So I want to make sure we're actually troubleshooting RTB and not some other Python process. Can you match the PID with ps ax to get the full command? Or maybe do a kill PID on your Python process to see what happens.

Next, for troubleshooting, I'd suggest adding "--debug", to the Dockerfile. That will give a little more information if something is going wrong. Then when we start the docker, when does the CPU spike? Is it soon as it starts or does it require some interaction with a page? The log will display what pages are being queried in real time.

[G4br1elsec]

I know this is the reason for python's GIL lock, so single-process multithreading can't take advantage of multi-cores

我需要更多信息来帮助。让我们从您的监视方式开始top。那是在docker容器中运行吗？如果要在主机上运行它，则与RootTheBox无关。要检查Docker容器中的统计信息，您需要运行：将显示docker-compose docker stats CONTAINER

docker top CONTAINER [ps OPTIONS]

top

据我所知，Python在docker中以自己的映像运行。因此，我想确保我们实际上是在对RTB进行故障排除，而不是其他一些Python流程。您可以将PID与之匹配ps ax以获得完整命令吗？或者，也许kill PID在您的Python进程上进行操作以查看会发生什么。

接下来，为进行故障排除，建议将添加"--debug",到中Dockerfile。如果出现问题，这将提供更多信息。然后，当我们启动docker时，CPU何时达到峰值？它是刚开始还是需要与页面进行一些交互？该日志将实时显示正在查询的页面。

[eljeffeg]

Weird that you're getting Error: Database dialect not supported. It should be sqlite if it's the docker.

[eljeffeg]

One option that you might be able to play with is switching to PyPy, which claims to be on average 4.4 times faster than CPython. It just hasn't been tested as much on RootTheBox. To use PyPy with docker, just change your Dockerfile to this:

FROM pypy:3

RUN mkdir /opt/rtb
ADD . /opt/rtb

RUN apt-get update
RUN apt-get install build-essential zlib1g-dev -y
RUN apt-get install python3-pycurl sqlite3 libsqlite3-dev -y

ADD ./setup/requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt --upgrade

VOLUME ["/opt/rtb/files"]
ENTRYPOINT ["pypy3", "/opt/rtb/rootthebox.py", "--setup=docker", "--sql_dialect=sqlite"]

[packetgeek]

Just wanted to add... I'm also seeing the CPU loading issue. I run RTB in the usual two Docker containers (for memcached and the web app). The database is an instance of SQLite3, running in the one container. We're using RTB to game-ify a college course for approximately 70 students. When we restart the containers, the CPU load will shoot up to 100% and remain there for the first few minutes. However, it does (eventually) come back down to 0% (when no one is logged in). Over time (weeks) the CPU loading will creep back up, forcing a container restart every 3 weeks or so. It's not a major problem as it just requires some minor hands-on maintenance.