memory leaks

[caobug]

Hello, the ssmanager on master used a lot of memory, maybe have some leaks?

[zonyitoo]

That's strange. How to reproduce? How many servers are you running in this process?

Memory won't be recycled even if you close some of the servers?

[zonyitoo]

I have tested a whole night with this script:

#!/bin/bash

while true; do
    echo 'add: {"server_port":8388,"password":"hello-kitty"}' | nc -uc '127.0.0.1' '6100'
    echo 'remove: {"server_port":8388}' | nc -uc '127.0.0.1' '6100'
    sleep 0.1
done

[caobug]

Hello, I use your command created 20000 servers, and random sent numerous normal and error connect request to these ports.

memory:

[zonyitoo]

Yes, I can reproduce that.

It doesn't "leak", but use tremendous VIRT memory.

[caobug]

Yes, I can reproduce that.

It doesn't "leak", but use tremendous VIRT memory.

Thanks for your answers.

Yes, 69.9G is VIRT.

This 9.6G physical memory (RES) is normal?

[zonyitoo]

Can't tell. My ssserver, running on Debian 9, requires about 156M VIRT and 10168 RES.

Let's see why ssmanager consumes so many memories.

[zonyitoo]

I tested ss-manager (shadowsocks-libev) with the same script.

Every ss-server uses about 4200M VIRT and 1150 RES after server started without processing any connections.

So, if you start 20000 servers, it may uses a lot more VIRT memories then ssmanager in this project, but less RES, because ss-server are separated processes.

ssserver running on the same laptop uses:

4284M VIRT and 4660 RES.

It is normal, probably. ssserver have a multi-threaded runtime, but ss-server is a single process program.

[caobug]

I have limited the buffer for each thread to 8K, so I don't think this is the problem.

I listed the memory usage details through the command pmap -x pid:

RSS:

69720,48264,45536,40232,39340,35200,34124,33292,30544,29728,29448,27344,24856,24612,23100,21640,19636,19520,19192,18788,16808,16512,16048,15740,15424,14944,14916,14448,14324,14300,14104,14100,14032,13836,13720,13536,13468,13456,13416,13224,13156,12892,12880,12872,12832,12668,12600,12480,12320,12312,12284,12248,12228,12020,11972,11968,11964,11912,11716,11672,11644,11580,11576,11524,11440,11396,11372,11244,11200,11172,11164,11132,11116,11100,11052,11040,10976,10960,10904,10904,10888,10860,10832,10824,10820,10808,10796,10780,10736,10736,10728,10724,10664,10600,10588,10564,10556,10548,10512,10496,10488,10480,10448,10428,10404,10388,10340,10332,10324,10300,10280,10252,10244,10228,10228,10224,10212,10196,10192,10140,10056,10044,10016,9992,9976,9968,9964,9952,9948,9948,9944,9940,9936,9912,9892,9868,9780,9780,9768,9768,9744,9716,9708,9700,9692,9688,9688,9684,9636,9632,9556,9508,9468,9396,9380,9376,9360,9360,9352,9348,9348,9348,9316,9288,9288,9256,9240,9216,9212,9208,9200,9180,9152,9148,9144,9144,9136,9092,9088,9080,9080,9076,9068,9064,9060,9052,9048,9044,9024,9016,8988,8976,8956,8948,8888,8852,8820,8816,8800,8800,8788,8784,8776,8772,8764,8736,8716,8656,8632,8624,8600,8560,8552,8548,8548,8536,8536,8532,8532,8528,8492,8480,8476,8476,8472,8472,8468,8464,8456,8400,8392,8368,8348,8340,8284,8280,8236,8232,8220,8220,8212,8212,8208,8192,8188,8188,8188,8184,8160,8144,8100,8080,8076,8064,8056,8048,8028,7976,7960,7952,7944,7924,7920,7920,7904,7900,7832,7812,7808,7796,7780,7768,7748,7744,7676,7648,7648,7636,7632,7632,7632,7628,7624,7624,7624,7624,7624,7624,7620,7616,7616,7612,7612,7608,7604,7604,7600,7600,7588,7588,7584,7580,7560,7556,7556,7552,7528,7528,7516,7504,7500,7484,7484,7472,7472,7472,7472,7472,7468,7460,7460,7432,7416,7416,7396,7372,7372,7348,7348,7328,7292,7288,7272,7268,7252,7248,7220,7208,7196,7188,7180,7160,7160,7144,7124,7116,7112,7112,7104,7096,7088,7080,7076,7064,7060,7060,7060,7048,7032,7028,7020,7008,7000,6980,6980,6976,6964,6952,6924,6924,6884,6872,6860,6840,6832,6808,6800,6788,6788,6780,6780,6780,6776,6772,6768,6768,6764,6760,6752,6748,6744,6740,6740,6732,6732,6728,6724,6724,6716,6712,6712,6700,6684,6672,6668,6660,6628,6612,6608,6592,6588,6588,6560,6524,6508,6504,6496,6496,6496,6480,6476,6468,6464,6464,6460,6460,6456,6456,6456,6456,6452,6452,6452,6448,6448,6444,6444,6444,6436,6432,6428,6416,6416,6412,6380,6372,6368,6356,6352,6340,6332,6328,6312,6292,6268,6260,6252,6248,6216,6216,6208,6208,6200,6192,6184,6180,6180,6176,6176,6176,6176,6172,6172,6172,6172,6164,6164,6164,6164,6164,6156,6156,6156,6148,6136,6120,6116,6116,6116,6108,6104,6076,6072,6064,6040,6032,6024,6024,6016,6012,6008,5992,5980,5976,5968,5964,5964,5956,5952,5948,5908,5896,5896,5892,5892,5892,5892,5892,5888,5888,5888,5884,5884,5884,5884,5884,5884,5884,5880,5880,5880,5880,5880,5880,5876,5876,5876,5876,5876,5876,5876,5876,5872,5872,5872,5872,5868,5868,5844,5840,5836,5800,5788,5788,5780,5776,5772,5756,5748,5732,5724,5720,5720,5720,5720,5720,5716,5716,5716,5716,5712,5712,5708,5704,5692,5672,5664,5660,5656,5648,5636,5620,5616,5580,5576,5560,5556,5552,5548,5544,5532,5532,5520,5508,5500,5492,5484,5460,5448,5444,5428,5392,5364,5348,5348,5348,5340,5328,5280,5268,5264,5260,5260,5244,5220,5208,5184,5168,5148,5140,5140,5116,5080,5060,5056,5052,5048,5048,5032,5032,5032,5032,5020,5016,5016,5016,5016,5008,5000,5000,5000,4996,4996,4996,4992,4988,4988,4984,4984,4984,4976,4976,4972,4972,4968,4968,4968,4960,4952,4952,4928,4888,4884,4872,4864,4860,4856,4852,4848,4840,4812,4808,4800,4788,4784,4780,4776,4760,4756,4752,4748,4744,4744,4740,4736,4736,4736,4728,4728,4720,4704,4704,4704,4704,4700,4700,4696,4696,4692,4692,4688,4688,4688,4688,4684,4680,4680,4680,4672,4664,4660,4656,4652,4648,4620,4604,4564,4536,4532,4532,4520,4516,4512,4500,4496,4496,4464,4460,4460,4456,4456,4456,4456,4452,4444,4440,4436,4432,4432,4420,4420,4420,4420,4416,4416,4412,4412,4412,4412,4408,4408,4404,4400,4292,4280,4280,4268,4252,4244,4240,4220,4216,4212,4208,4208,4200,4196,4192,4184,4172,4156,4148,4148,4144,4144,4144,4144,4144,4140,4140,4140,4140,4140,4140,4136,4136,4136,4136,4136,4136,4136,4136,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4132,4128,4128,4128,4128,4128,4128,4128,4128,4128,4128,4128,4124,4124,4124,4124,4120,4120,4120,4120,4120,4116,4116,4116,4108,4100,4096

Greater than 4k: 6GB Greater than 8k: 2.8GB RSS total: 7GB

Maybe you can reduce these buffer size.

[lotusired]

Is it not uncommon to use a VPS with 128G memory to circumvent GFW?

[caobug]

Yes. I was curious and tested it.

[zonyitoo]

Very busy these days. Hope you guys have found the reason why it consumed so many memories.

[zonyitoo]

@caobug I found a bug that may be related to this issue. Please try the nightly build and see if the problem have been resolved.