長期間動作させるとpage crashdが発生する問題

長時間の動作でpage crashedが発生する問題がある。これは、chromeが消費する一時ファイルの容量がdockerが通常に比べて非常に小さいのが原因

(https://github.com/puppeteer/puppeteer/blob/master/docs/troubleshooting.md#running-puppeteer-in-docker)

Tips By default, Docker runs a container with a /dev/shm shared memory space 64MB. This is typically too small for Chrome and will cause Chrome to crash when rendering large pages. To fix, run the container with docker run --shm-size=1gb to increase the size of /dev/shm. Since Chrome 65, this is no longer necessary. Instead, launch the browser with the --disable-dev-shm-usage flag:

const browser = await puppeteer.launch({ args: ['--disable-dev-shm-usage'] }); This will write shared memory files into /tmp instead of /dev/shm. See crbug.com/736452 for more details.

Seeing other weird errors when launching Chrome? Try running your container with docker run --cap-add=SYS_ADMIN when developing locally. Since the Dockerfile adds a pptr user as a non-privileged user, it may not have all the necessary privileges.

dumb-init is worth checking out if you're experiencing a lot of zombies Chrome processes sticking around. There's special treatment for processes with PID=1, which makes it hard to terminate Chrome properly in some cases (e.g. in Docker).

ymmtr6 / TDeck-Scraping

長期間動作させるとpage crashdが発生する問題 #1