MemoryLeak

[mmeles]

websocket を使用するようになってからエラーが生じる

[mmeles]

コードを手元（自PC の docker ）で実行したところ、実行して終了する際（ctrl+C を押下したとき）下記エラーが生じました。 これは想定される動作でしょうか？

vscode ➜ /workspaces/mem_leak $ python mem_leak.py
^CTraceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/workspaces/mem_leak/mem_leak.py", line 23, in main
    await store.orderbook.wait()
  File "/home/vscode/.local/lib/python3.12/site-packages/pybotters/store.py", line 258, in wait
    await event.wait()
  File "/usr/local/lib/python3.12/asyncio/locks.py", line 212, in wait
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspaces/mem_leak/mem_leak.py", line 27, in <module>
    asyncio.run(main())
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 123, in run
    raise KeyboardInterrupt()
KeyboardInterrupt

[mmeles]

'# print(orderbook)' のコメントアウト外したらちゃんとprint されていたので大丈夫そうかなと思ってます！

[mmeles]

30分ほど実行し、ps auxf でプロセスが使用しているメモリを見てみたのですが、メモリリークはしてないように見えました。（はじめてつかうコマンドなのでよくわかってないですが...） ctrl+C で終了しても特にメモリの消費量に差はなかったです。 下記をそれぞれ記載します。

• 30分ほど実行した際の ps auxf の結果
• ctrl+C でキャンセルした後の ps auxf の結果

[mmeles]

30分ほど実行した際の ps auxf の結果

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
vscode     272  0.0  0.3 597916 52232 ?        Ssl  04:28   0:01 /home/vscode/.vscode-server/bin/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node -e  ????const net = require('net'); ????const fs = require('fs'); ????process.stdin.pause(); ????const client = net.createConnection({ host: '127.0.0.1', port: 41521 }, () => { ?????console.error('Connection established'); ?????client.pipe(process.stdout); ?????process.stdin.pipe(client); ????}); ????client.on('close', function (hadError) { ?????console.error(hadError ? 'Remote close with error' : 'Remote close'); ?????process.exit(hadError ? 1 : 0); ????}); ????client.on('error', function (err) { ?????process.stderr.write(err && (err.stack || err.message) || String(err)); ????}); ????process.stdin.on('close', function (hadError) { ?????console.error(hadError ? 'Remote stdin close with error' : 'Remote stdin close'); ?????process.exit(hadError ? 1 : 0); ????}); ????process.on('uncaughtException', function (err) { ?????fs.writeSync(process.stderr.fd, `Uncaught Exception: ${String(err && (err.stack || err.message) || err)}\n`); ????}); ???
vscode     249  0.0  0.3 603036 54204 ?        Ssl  04:28   0:01 /home/vscode/.vscode-server/bin/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node -e  ????const net = require('net'); ????const fs = require('fs'); ????process.stdin.pause(); ????const client = net.createConnection({ host: '127.0.0.1', port: 41521 }, () => { ?????console.error('Connection established'); ?????client.pipe(process.stdout); ?????process.stdin.pipe(client); ????}); ????client.on('close', function (hadError) { ?????console.error(hadError ? 'Remote close with error' : 'Remote close'); ?????process.exit(hadError ? 1 : 0); ????}); ????client.on('error', function (err) { ?????process.stderr.write(err && (err.stack || err.message) || String(err)); ????}); ????process.stdin.on('close', function (hadError) { ?????console.error(hadError ? 'Remote stdin close with error' : 'Remote stdin close'); ?????process.exit(hadError ? 1 : 0); ????}); ????process.on('uncaughtException', function (err) { ?????fs.writeSync(process.stderr.fd, `Uncaught Exception: ${String(err && (err.stack || err.message) || err)}\n`); ????}); ???
vscode     178  0.0  0.0   2480   508 ?        Ss   04:28   0:00 sh /home/vscode/.vscode-server/bin/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/bin/code-server --log debug --force-disable-user-env --server-data-dir /home/vscode/.vscode-server --use-host-proxy --telemetry-level all --accept-server-license-terms --host 127.0.0.1 --port 0 --connection-token-file /home/vscode/.vscode-server/data/Machine/.connection-token-1a5daa3a0231a0fbba4f14db7ec463cf99d7768e --extensions-download-dir /home/vscode/.vscode-server/extensionsCache --install-extension dbaeumer.vscode-eslint --install-extension ms-python.python --install-extension ms-python.vscode-pylance --start-server --disable-websocket-compression
vscode     187  0.4  0.7 982228 120956 ?       Sl   04:28   0:12  \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/server-main.js --log debug --force-disable-user-env --server-data-dir /home/vscode/.vscode-server --use-host-proxy --telemetry-level all --accept-server-license-terms --host 127.0.0.1 --port 0 --connection-token-file /home/vscode/.vscode-server/data/Machine/.connection-token-1a5daa3a0231a0fbba4f14db7ec463cf99d7768e --extensions-download-dir /home/vscode/.vscode-server/extensionsCache --install-extension dbaeumer.vscode-eslint --install-extension ms-python.python --install-extension ms-python.vscode-pylance --start-server --disable-websocket-compression
vscode     303  0.0  0.3 914452 55492 ?        Sl   04:28   0:00      \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/bootstrap-fork --type=fileWatcher
vscode     326  1.2  1.1 1058052 195288 ?      Sl   04:28   0:35      \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node --dns-result-order=ipv4first /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/bootstrap-fork --type=extensionHost --transformURIs --useHostProxy=true
vscode     364  0.0  0.3 602804 55388 ?        Sl   04:28   0:00      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/json-language-features/server/dist/node/jsonServerMain --node-ipc --clientProcessId=326
vscode     922  0.1  1.4 1040668 231176 ?      Sl   04:29   0:05      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /home/vscode/.vscode-server/extensions/ms-python.vscode-pylance-2023.11.10/dist/server.bundle.js --cancellationReceive=file:2d54a24b698eca97aad63a0cdecfcb0dd0bd80de07 --node-ipc --clientProcessId=326
vscode   16333  0.0  0.3 601860 49180 ?        Sl   05:13   0:00      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /home/vscode/.vscode-server/extensions/dbaeumer.vscode-eslint-2.4.2/server/out/eslintServer.js --node-ipc --clientProcessId=326
vscode   16341  0.1  0.4 636728 79476 ?        Sl   05:13   0:00      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/html-language-features/server/dist/node/htmlServerMain --node-ipc --clientProcessId=326
vscode   16725  0.6  0.8 697148 141248 ?       Sl   05:13   0:01      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node --max-old-space-size=3072 /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/node_modules/typescript/lib/tsserver.js --serverMode partialSemantic --useInferredProjectPerProjectRoot --disableAutomaticTypingAcquisition --cancellationPipeName /tmp/vscode-typescript1000/dd4d0ebbebf4ef817eb6/tscancellation-88b86aa67af86c58b18f.tmp* --locale en --noGetErrOnBackgroundUpdate --validateDefaultNpmLocation --useNodeIpc
vscode   16726  1.1  0.9 746108 160124 ?       Sl   05:13   0:02      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node --max-old-space-size=3072 /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/node_modules/typescript/lib/tsserver.js --useInferredProjectPerProjectRoot --enableTelemetry --cancellationPipeName /tmp/vscode-typescript1000/dd4d0ebbebf4ef817eb6/tscancellation-c33fb68b941710b3d939.tmp* --locale en --noGetErrOnBackgroundUpdate --validateDefaultNpmLocation --useNodeIpc
vscode   16750  0.0  0.3 608848 61580 ?        Sl   05:13   0:00      |       \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/node_modules/typescript/lib/typingsInstaller.js --globalTypingsCacheLocation /home/vscode/.cache/typescript/5.2 --enableTelemetry --typesMapLocation /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/node_modules/typescript/lib/typesMap.json --validateDefaultNpmLocation
vscode     496  0.3  0.4 694752 75680 ?        Sl   04:28   0:10      \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/bootstrap-fork --type=ptyHost --logsPath /home/vscode/.vscode-server/data/logs/20231203T042819
vscode     761  0.0  0.0  12456  7312 pts/1    Ss   04:29   0:00          \_ /bin/bash --init-file /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/vs/workbench/contrib/terminal/browser/media/shellIntegration-bash.sh
vscode    1538  3.6  0.2 115448 33880 pts/1    Sl+  04:30   1:42          |   \_ python mem_leak.py
vscode    2950  0.0  0.0  12456  7348 pts/2    Ss   04:34   0:00          \_ /bin/bash --init-file /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/vs/workbench/contrib/terminal/browser/media/shellIntegration-bash.sh
vscode    3031  0.2  0.0   8952  4548 pts/2    S+   04:35   0:05          |   \_ htop
vscode   17499  0.0  0.0  12456  7408 pts/3    Ss   05:16   0:00          \_ /bin/bash --init-file /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/vs/workbench/contrib/terminal/browser/media/shellIntegration-bash.sh
vscode   17930  0.0  0.0  11620  3356 pts/3    R+   05:17   0:00              \_ ps auxf
vscode     158  0.0  0.0   2480   508 ?        Ss   04:28   0:00 /bin/sh
vscode     177  0.0  0.2 632156 46260 ?        Sl   04:28   0:00  \_ /home/vscode/.vscode-server/bin/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /tmp/vscode-remote-containers-server-15a0b9ac-e713-4eb8-9065-796943bca722.js
root        76  0.0  0.0   2480   564 ?        Ss   04:28   0:00 /bin/sh
vscode      42  0.0  0.0   2480  1608 ?        Ss   04:28   0:00 /bin/sh
root        34  0.0  0.0   2480   508 ?        Ss   04:28   0:00 /bin/sh -c echo "New container started. Keep-alive process started." ; export VSCODE_REMOTE_CONTAINERS_SESSION=23ad5ef2-a037-46dc-96b6-0b05482277ea1701577694278 ; /bin/sh
root        40  0.0  0.0   2480   504 ?        S    04:28   0:00  \_ /bin/sh
root         1  0.0  0.0   2480  1508 ?        Ss   04:28   0:00 /bin/sh -c echo Container started trap "exit 0" 15  exec "$@" while sleep 1 & wait $!; do :; done -
root     17928  0.0  0.0   7256   556 ?        S    05:17   0:00 sleep 1

[mmeles]

ctrl+C でキャンセルした後の ps auxf の結果

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
vscode     272  0.0  0.3 597916 52232 ?        Ssl  04:28   0:01 /home/vscode/.vscode-server/bin/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node -e  ????const net = require('net'); ????const fs = require('fs'); ????process.stdin.pause(); ????const client = net.createConnection({ host: '127.0.0.1', port: 41521 }, () => { ?????console.error('Connection established'); ?????client.pipe(process.stdout); ?????process.stdin.pipe(client); ????}); ????client.on('close', function (hadError) { ?????console.error(hadError ? 'Remote close with error' : 'Remote close'); ?????process.exit(hadError ? 1 : 0); ????}); ????client.on('error', function (err) { ?????process.stderr.write(err && (err.stack || err.message) || String(err)); ????}); ????process.stdin.on('close', function (hadError) { ?????console.error(hadError ? 'Remote stdin close with error' : 'Remote stdin close'); ?????process.exit(hadError ? 1 : 0); ????}); ????process.on('uncaughtException', function (err) { ?????fs.writeSync(process.stderr.fd, `Uncaught Exception: ${String(err && (err.stack || err.message) || err)}\n`); ????}); ???
vscode     249  0.0  0.3 603036 54468 ?        Ssl  04:28   0:01 /home/vscode/.vscode-server/bin/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node -e  ????const net = require('net'); ????const fs = require('fs'); ????process.stdin.pause(); ????const client = net.createConnection({ host: '127.0.0.1', port: 41521 }, () => { ?????console.error('Connection established'); ?????client.pipe(process.stdout); ?????process.stdin.pipe(client); ????}); ????client.on('close', function (hadError) { ?????console.error(hadError ? 'Remote close with error' : 'Remote close'); ?????process.exit(hadError ? 1 : 0); ????}); ????client.on('error', function (err) { ?????process.stderr.write(err && (err.stack || err.message) || String(err)); ????}); ????process.stdin.on('close', function (hadError) { ?????console.error(hadError ? 'Remote stdin close with error' : 'Remote stdin close'); ?????process.exit(hadError ? 1 : 0); ????}); ????process.on('uncaughtException', function (err) { ?????fs.writeSync(process.stderr.fd, `Uncaught Exception: ${String(err && (err.stack || err.message) || err)}\n`); ????}); ???
vscode     178  0.0  0.0   2480   508 ?        Ss   04:28   0:00 sh /home/vscode/.vscode-server/bin/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/bin/code-server --log debug --force-disable-user-env --server-data-dir /home/vscode/.vscode-server --use-host-proxy --telemetry-level all --accept-server-license-terms --host 127.0.0.1 --port 0 --connection-token-file /home/vscode/.vscode-server/data/Machine/.connection-token-1a5daa3a0231a0fbba4f14db7ec463cf99d7768e --extensions-download-dir /home/vscode/.vscode-server/extensionsCache --install-extension dbaeumer.vscode-eslint --install-extension ms-python.python --install-extension ms-python.vscode-pylance --start-server --disable-websocket-compression
vscode     187  0.4  0.7 982228 121560 ?       Sl   04:28   0:12  \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/server-main.js --log debug --force-disable-user-env --server-data-dir /home/vscode/.vscode-server --use-host-proxy --telemetry-level all --accept-server-license-terms --host 127.0.0.1 --port 0 --connection-token-file /home/vscode/.vscode-server/data/Machine/.connection-token-1a5daa3a0231a0fbba4f14db7ec463cf99d7768e --extensions-download-dir /home/vscode/.vscode-server/extensionsCache --install-extension dbaeumer.vscode-eslint --install-extension ms-python.python --install-extension ms-python.vscode-pylance --start-server --disable-websocket-compression
vscode     303  0.0  0.3 914452 55492 ?        Sl   04:28   0:00      \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/bootstrap-fork --type=fileWatcher
vscode     326  1.2  1.2 1058052 196112 ?      Sl   04:28   0:36      \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node --dns-result-order=ipv4first /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/bootstrap-fork --type=extensionHost --transformURIs --useHostProxy=true
vscode     364  0.0  0.3 602804 55388 ?        Sl   04:28   0:00      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/json-language-features/server/dist/node/jsonServerMain --node-ipc --clientProcessId=326
vscode     922  0.1  1.4 1040668 231176 ?      Sl   04:29   0:05      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /home/vscode/.vscode-server/extensions/ms-python.vscode-pylance-2023.11.10/dist/server.bundle.js --cancellationReceive=file:2d54a24b698eca97aad63a0cdecfcb0dd0bd80de07 --node-ipc --clientProcessId=326
vscode   16333  0.0  0.3 601860 49180 ?        Sl   05:13   0:00      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /home/vscode/.vscode-server/extensions/dbaeumer.vscode-eslint-2.4.2/server/out/eslintServer.js --node-ipc --clientProcessId=326
vscode   16341  0.1  0.4 636728 79476 ?        Sl   05:13   0:00      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/html-language-features/server/dist/node/htmlServerMain --node-ipc --clientProcessId=326
vscode   16725  0.5  0.8 697148 141248 ?       Sl   05:13   0:01      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node --max-old-space-size=3072 /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/node_modules/typescript/lib/tsserver.js --serverMode partialSemantic --useInferredProjectPerProjectRoot --disableAutomaticTypingAcquisition --cancellationPipeName /tmp/vscode-typescript1000/dd4d0ebbebf4ef817eb6/tscancellation-88b86aa67af86c58b18f.tmp* --locale en --noGetErrOnBackgroundUpdate --validateDefaultNpmLocation --useNodeIpc
vscode   16726  1.0  0.9 746108 160124 ?       Sl   05:13   0:02      |   \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node --max-old-space-size=3072 /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/node_modules/typescript/lib/tsserver.js --useInferredProjectPerProjectRoot --enableTelemetry --cancellationPipeName /tmp/vscode-typescript1000/dd4d0ebbebf4ef817eb6/tscancellation-c33fb68b941710b3d939.tmp* --locale en --noGetErrOnBackgroundUpdate --validateDefaultNpmLocation --useNodeIpc
vscode   16750  0.0  0.3 608848 61580 ?        Sl   05:13   0:00      |       \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/node_modules/typescript/lib/typingsInstaller.js --globalTypingsCacheLocation /home/vscode/.cache/typescript/5.2 --enableTelemetry --typesMapLocation /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/extensions/node_modules/typescript/lib/typesMap.json --validateDefaultNpmLocation
vscode     496  0.3  0.4 694752 75604 ?        Sl   04:28   0:10      \_ /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/bootstrap-fork --type=ptyHost --logsPath /home/vscode/.vscode-server/data/logs/20231203T042819
vscode     761  0.0  0.0  12456  7312 pts/1    Ss+  04:29   0:00          \_ /bin/bash --init-file /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/vs/workbench/contrib/terminal/browser/media/shellIntegration-bash.sh
vscode    2950  0.0  0.0  12456  7348 pts/2    Ss   04:34   0:00          \_ /bin/bash --init-file /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/vs/workbench/contrib/terminal/browser/media/shellIntegration-bash.sh
vscode    3031  0.2  0.0   8952  4548 pts/2    S+   04:35   0:05          |   \_ htop
vscode   17499  0.0  0.0  12456  7408 pts/3    Ss   05:16   0:00          \_ /bin/bash --init-file /vscode/vscode-server/bin/linux-x64/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/out/vs/workbench/contrib/terminal/browser/media/shellIntegration-bash.sh
vscode   18191  0.0  0.0  11620  3388 pts/3    R+   05:17   0:00              \_ ps auxf
vscode     158  0.0  0.0   2480   508 ?        Ss   04:28   0:00 /bin/sh
vscode     177  0.0  0.2 632156 46260 ?        Sl   04:28   0:00  \_ /home/vscode/.vscode-server/bin/1a5daa3a0231a0fbba4f14db7ec463cf99d7768e/node /tmp/vscode-remote-containers-server-15a0b9ac-e713-4eb8-9065-796943bca722.js
root        76  0.0  0.0   2480   564 ?        Ss   04:28   0:00 /bin/sh
vscode      42  0.0  0.0   2480  1608 ?        Ss   04:28   0:00 /bin/sh
root        34  0.0  0.0   2480   508 ?        Ss   04:28   0:00 /bin/sh -c echo "New container started. Keep-alive process started." ; export VSCODE_REMOTE_CONTAINERS_SESSION=23ad5ef2-a037-46dc-96b6-0b05482277ea1701577694278 ; /bin/sh
root        40  0.0  0.0   2480   504 ?        S    04:28   0:00  \_ /bin/sh
root         1  0.0  0.0   2480  1508 ?        Ss   04:28   0:00 /bin/sh -c echo Container started trap "exit 0" 15  exec "$@" while sleep 1 & wait $!; do :; done -
root     18189  0.0  0.0   7256   500 ?        S    05:17   0:00 sleep 1

[mmeles]

下記が考えられると思います。

• 実行環境の差異
• コードの差異

なので、

• 実行環境の詳細
• サンプルコードで事象が発生するかどうか（+ 可能であれば ps auxf 等でプロセスのメモリ消費量を確認した結果）

などが気になります。 もしかしたら docker を使ったのが微妙かもしれない...

[denytu]

memory_profilerで各行ごとに調べてみたものになります。ループ回数で違っていました。ループ数100のほう

Filename: test_program.py

Line # Mem usage Increment Occurrences Line Contents

 5     30.6 MiB     30.6 MiB           1   @profile
 6                                         async def main():
 7     30.6 MiB      0.0 MiB           1       async with pybotters.Client() as client:
 8     30.6 MiB      0.0 MiB           1           store = pybotters.BybitV5DataStore()
 9     30.6 MiB      0.0 MiB           1           ws = await client.ws_connect(
10     30.6 MiB      0.0 MiB           1               'wss://stream.bybit.com/v5/public/linear',
11                                                     send_json={
12     30.6 MiB      0.0 MiB           1                   'op': 'subscribe',
13     30.6 MiB      0.0 MiB           1                   'args': ['orderbook.50.BTCUSDT'],
14                                                     },
15     32.8 MiB      2.2 MiB           2               hdlr_json=store.onmessage,
16                                                 )
17     33.0 MiB      0.0 MiB           2           while not all([
18     33.0 MiB      0.0 MiB           2               len(store.orderbook),
19                                                 ]):
20     33.0 MiB      0.2 MiB           2               await store.wait()
21                                                 
22     33.0 MiB      0.0 MiB         102           for _ in range(100):
23     33.0 MiB      0.0 MiB         200               await store.orderbook.wait()
24     33.0 MiB      0.0 MiB         100               orderbook = store.orderbook.sorted({'symbol': 'BTCUSDT'})

[denytu]

ループ数1000にした方。23行目が異なる。 記述されてない上4行は以下のモジュールの読み込み+改行 import asyncio import pybotters from memory_profiler import profile

Filename: test_program.py

Line # Mem usage Increment Occurrences Line Contents

 5     30.8 MiB     30.8 MiB           1   @profile
 6                                         async def main():
 7     30.8 MiB      0.0 MiB           1       async with pybotters.Client() as client:
 8     30.8 MiB      0.0 MiB           1           store = pybotters.BybitV5DataStore()
 9     30.8 MiB      0.0 MiB           1           ws = await client.ws_connect(
10     30.8 MiB      0.0 MiB           1               'wss://stream.bybit.com/v5/public/linear',
11                                                     send_json={
12     30.8 MiB      0.0 MiB           1                   'op': 'subscribe',
13     30.8 MiB      0.0 MiB           1                   'args': ['orderbook.50.BTCUSDT'],
14                                                     },
15     32.8 MiB      2.0 MiB           2               hdlr_json=store.onmessage,
16                                                 )
17     33.1 MiB      0.0 MiB           2           while not all([
18     33.1 MiB      0.0 MiB           2               len(store.orderbook),
19                                                 ]):
20     33.1 MiB      0.2 MiB           2               await store.wait()
21                                                 
22     33.4 MiB      0.0 MiB        1002           for _ in range(1000):
23     33.4 MiB      0.3 MiB        2000               await store.orderbook.wait()
24     33.4 MiB      0.0 MiB        1000               orderbook = store.orderbook.sorted({'symbol': 'BTCUSDT'})

[mmeles]

ループ数変えてやってみたところ、23行目に記載されたメモリは下記の通りでした

100:0.2
1000:0.8
10000:0.7
100000:2.0
(MiB)

確かにメモリの消費量が増える傾向ありそうです。 謎も多いです。

• 1000と10000で逆転してる
• 増加傾向は同じでも値がいただいた結果とだいぶ違う

次は500000で回してみます(時間かかりすぎて諦めるかも…)

---

環境はひとまず下記でやっています。

• Python3.7.16
• Pybotters0.17

[mmeles]

500000は何回やってもファイルが残らないので、200000で実行したらよくわからない結果になってました... とりあえず結果だけ貼ります。時間があるときにまた調べます。

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     5     30.6 MiB     30.6 MiB           1   @profile
     6                                         async def main():
     7     30.8 MiB      0.2 MiB           1       async with pybotters.Client() as client:
     8     30.8 MiB      0.0 MiB           1           store = pybotters.BybitV5DataStore()
     9     30.8 MiB      0.0 MiB           1           ws = await http://client.ws_connect(
    10     30.8 MiB      0.0 MiB           1               'wss://stream.bybit.com/v5/public/linear',
    11                                                     send_json={
    12     30.8 MiB      0.0 MiB           1                   'op': 'subscribe',
    13     30.8 MiB      0.0 MiB           1                   'args': ['orderbook.50.BTCUSDT'],
    14                                                     },
    15     32.7 MiB      1.9 MiB           2               hdlr_json=store.onmessage,
    16                                                 )
    17     32.7 MiB      0.0 MiB           2           while not all([
    18     32.7 MiB      0.0 MiB           2               len(store.orderbook),
    19                                                 ]):
    20     32.7 MiB      0.0 MiB           2               await store.wait()
    21                                                 # 以下取得したordebookを元に発注を繰り返す。
    22     37.7 MiB -22580.4 MiB      200002           for _ in range (200000):
    23     37.7 MiB -45154.2 MiB      400000               await store.orderbook.wait()
    24     37.7 MiB -22579.9 MiB      200000               orderbook = store.orderbook.sorted({'symbol': 'BTCUSDT'})

[denytu]

ありがと～、なんだこれはｗ

[mmeles]

きちんと記録取れていないですが、同じループ回数でもメモリ消費量が変わる場合がある気がします… 調査に役立つかもしれないので残しておきます。

[mmeles]

コード追ってみたのですが、メモリリークの箇所はわかりませんでした...

わかったところだけ下記に記載します

• 下記のように bybit_v5 .py に sleep を入れると事象が再現しやすい（ループ数 3 回とかでもメモリ使ってくれます。時間は環境によって変えたほうが良さそうです） def _onmessage(self, msg: Item, topic_ext: list[str]) -> None: time.sleep(0.04) operation = {"delete": [], "update": [], "insert": []}

• メモリの消費は下記箇所の★の行で起こっている(loop のたびに呼ばれて、その累積値が最終的な増分値になります) 。 ここから先は await event.wait() の時にどの処理が動いているのか追いきれませんでした... async def wait(self) -> list[Item]: event = asyncio.Event() ret = [] self._events[event] = ret await event.wait() ★★ del self._events[event] return ret

• @profile をライブラリ内に色々と仕込んで動かしたところ、下記が怪しそうな箇所の一つでした。ただ、他にもリークしてそうなところがあり数値は合わなかったので正直参考程度です...どんな処理をしているのかは追えていません。 Filename: /usr/local/lib/python3.7/site-packages/pybotters/models/bybit_v5.py

Line # Mem usage Increment Occurrences Line Contents

68     38.7 MiB     38.7 MiB           1       @profile
69                                             def _onmessage(self, msg: Item, ws: ClientWebSocketResponse) -> None:
70     38.7 MiB      0.0 MiB           1           if "success" in msg:
71                                                     if not msg["success"]:
72                                                         logger.warning(msg)
73                                         
74     38.7 MiB      0.0 MiB           1           if "topic" in msg:
75     38.7 MiB      0.0 MiB           1               dot_topic: str = msg["topic"]
76     39.3 MiB      0.5 MiB           1               topic, *topic_ext = dot_topic.split(".") ★★
77                                         
78     39.3 MiB      0.0 MiB           1               if topic in self:
79     39.3 MiB      0.0 MiB           1                   getattr(self[topic], "_onmessage")(msg, topic_ext) 

Line # Mem usage Increment Occurrences Line Contents

68     35.8 MiB     35.8 MiB           1       @profile
69                                             def _onmessage(self, msg: Item, ws: ClientWebSocketResponse) -> None:
70     35.8 MiB      0.0 MiB           1           if "success" in msg:
71                                                     if not msg["success"]:
72                                                         logger.warning(msg)
73                                         
74     35.8 MiB      0.0 MiB           1           if "topic" in msg:
75     35.8 MiB      0.0 MiB           1               dot_topic: str = msg["topic"]
76     35.8 MiB      0.0 MiB           1               topic, *topic_ext = dot_topic.split(".")
77                                         
78     35.8 MiB      0.0 MiB           1               if topic in self:
79     36.2 MiB      0.3 MiB           1                   getattr(self[topic], "_onmessage")(msg, topic_ext) ★★

[mmeles]

いつか時間があればさらに調査したいですが、当面は時間を取るのが難しそうです…