Is there a way to detect memory leak in Mapnik?

zdila commented 4 years ago

Hello,

We use latest mapnik for node (4.3.1). We observe there some memory leak. During rendering the memory usage rises until kernel kills the process (and it gets restarted by systemd). Our application uses 8 mapnik instances from the pool, destroying instance after it was used 1000 times.

If rendering is at 100% (when we re-render whole map) the process can eat in a cca two hours around 40 GB of RAM.

Rising memory consumption was observed from the beginning since we started to use the library but since last update of styles and maybe also library version (not sure about it) it is even higher.

Can we somehow analyze process dump to find where is it leaking (something that could help mapnik developers)?

We can dump it with https://serverfault.com/a/408929. According to this most memory is taken by heap and most of this heap is occupied by data simmilar to this:

27A0BC6909C B3 88 FF │ 9B B2 88 FF │ 9B B2 88 FF │ 9B B2 88 FF │ 9A B1 87 FF │ 9A B1 87 FF │ 9A B1 87 FF │ 9A B1 87 FF │ 9A B1 87 FF │ 9A B1 87 FF │ 9A B1 87 FF │ 9A B1 87 FF  ................................................
27A0BC6C09A B1 87 FF │ 9A B1 87 FF │ 99 B0 86 FF │ 99 B0 87 FF │ 99 B0 87 FF │ 99 B0 87 FF │ 99 B0 87 FF │ 99 B0 87 FF │ 99 B0 87 FF │ 99 B0 87 FF │ 99 B0 87 FF │ 99 B0 87 FF  ................................................
27A0BC6F099 AF 87 FF │ 99 AF 87 FF │ 99 AF 87 FF │ 99 AF 87 FF │ 99 AF 87 FF │ 99 AF 87 FF │ 99 AF 87 FF │ 99 AF 87 FF │ 98 AE 86 FF │ 98 AE 86 FF │ 97 AE 86 FF │ 97 AE 86 FF  ................................................
27A0BC72097 AE 86 FF │ 97 AE 86 FF │ 97 AE 86 FF │ 93 AA 82 FF │ 8A 9F 79 FF │ 80 93 6F FF │ 7D 90 6C FF │ 7F 92 6E FF │ 88 9D 77 FF │ 94 AA 83 FF │ 96 AE 85 FF │ 96 AE 85 FF  ..................y...o.}.l...n...w.............
27A0BC75096 AE 85 FF │ 96 AE 85 FF │ 96 AE 85 FF │ 96 AD 85 FF │ 95 AD 85 FF │ 95 AD 85 FF │ 94 AC 84 FF │ 94 AC 84 FF │ 94 AC 84 FF │ 94 AC 84 FF │ 94 AC 84 FF │ 94 AC 84 FF  ................................................
27A0BC78094 AC 84 FF │ 94 AC 84 FF │ 94 AC 84 FF │ 94 AC 84 FF │ 94 AB 84 FF │ 94 AB 84 FF │ 94 AB 84 FF │ 94 AB 84 FF │ 94 AB 84 FF │ 94 AB 84 FF │ 94 AB 84 FF │ 94 AB 84 FF  ................................................
27A0BC7B094 AB 84 FF │ 94 AB 84 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF  ................................................
27A0BC7E093 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF  ................................................
27A0BC81093 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 A9 83 FF │ 93 A9 83 FF │ 93 A9 83 FF │ 93 A9 83 FF │ 93 AA 83 FF │ 93 AA 83 FF │ 93 AA 83 FF  ................................................
27A0BC84093 AA 83 FF │ 93 AA 83 FF │ 92 A9 82 FF │ 92 A9 82 FF │ 92 A9 82 FF │ 92 A9 82 FF │ 92 A9 82 FF │ 92 A9 83 FF │ 92 A9 83 FF │ 92 A9 83 FF │ 92 A9 83 FF │ 92 A9 83 FF  ................................................
27A0BC87092 A9 83 FF │ 92 A9 83 FF │ 91 A8 82 FF │ 91 A8 82 FF │ 91 A8 82 FF │ 91 A8 82 FF │ 91 A8 82 FF │ 91 A8 82 FF │ B9 D5 A1 FF │ B9 D5 A1 FF │ BA D5 A1 FF │ BA D5 A1 FF  .................................ա ..ա ..ա ..ա .
27A0BC8A0BA D5 A1 FF │ BA D5 A1 FF │ B9 D5 A0 FF │ B9 D5 A0 FF │ B9 D6 A0 FF │ B9 D6 A0 FF │ B9 D6 A0 FF │ B9 D6 A0 FF │ B9 D6 A0 FF │ B9 D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF  .ա ..ա ..ՠ ..ՠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .
27A0BC8D0BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF  . ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .
27A0BC900BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ B9 D5 9F FF  . ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ ..՟ .
27A0BC930B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF │ B9 D5 9F FF  .՟ ..՟ ..՟ ..՟ ..՟ ..՟ ..՟ ..՟ ..՟ ..՟ ..՟ ..՟ .
27A0BC960AE C7 94 FF │ A5 BD 8B FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF │ BA D6 A0 FF  .ǔ ...... ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .. ֠ .
27A0BC990BA D6 A0 FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF  . ֠ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .
27A0BC9C0BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF  . ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .
27A0BC9F0BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ 7A 57 3D FF │ 63 3E 25 FF │ 63 3E 25 FF │ 7B 5A 3F FF │ BA D6 9F FF │ BA D6 9F FF  . ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .zW=.c>%.c>%.{Z?.. ֟ .. ֟ .
27A0BCA20BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF  . ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .. ֟ .
27A0BCA50BA D6 9F FF │ BA D6 9F FF │ BA D6 9F FF │ BA D5 9F FF │ BA D5 9F FF │ BA D5 9F FF │ BA D5 9F FF │ BA D5 9F FF │ B9 D4 9E FF │ 97 AB 7C FF │ B8 D3 9D FF │ BA D5 9F FF  . ֟ .. ֟ .. ֟ ..՟ ..՟ ..՟ ..՟ ..՟ ..Ԟ ...|..ӝ ..՟ .
27A0BCA80BA D5 9F FF │ BA D5 9F FF │ BA D5 9F FF │ B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF  .՟ ..՟ ..՟ ..՞ ..՞ ..՞ ..՞ ..՞ ..՞ ..՞ ..՞ ..՞ .
27A0BCAB0B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF │ B9 D5 9E FF │ B8 D3 9E FF │ B8 D3 9E FF │ B8 D3 9E FF │ B8 D3 9E FF │ B8 D3 9E FF │ B8 D3 9E FF │ B8 D3 9E FF │ B8 D3 9E FF  .՞ ..՞ ..՞ ..՞ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ .
27A0BCAE0B8 D3 9E FF │ B8 D3 9E FF │ B8 D3 9E FF │ B8 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF  .Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ .
27A0BCB10B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B7 D3 9E FF │ B6 D2 9D FF │ B6 D2 9D FF │ B6 D2 9D FF │ B6 D2 9D FF │ B7 D3 9D FF │ B7 D3 9D FF │ B7 D2 9D FF  .Ӟ ..Ӟ ..Ӟ ..Ӟ ..Ӟ ..ҝ ..ҝ ..ҝ ..ҝ ..ӝ ..ӝ ..ҝ .
27A0BCB40B7 D2 9D FF │ B7 D2 9D FF │ B7 D2 9D FF │ B7 D2 9D FF │ B6 D1 9C FF │ B7 D2 9C FF │ B7 D2 9C FF │ B7 D2 9C FF │ B7 D2 9C FF │ B7 D2 9C FF │ B7 D2 9C FF │ B7 D2 9C FF  .ҝ ..ҝ ..ҝ ..ҝ ..ќ ..Ҝ ..Ҝ ..Ҝ ..Ҝ ..Ҝ ..Ҝ ..Ҝ .
27A0BCB70B7 D2 9C FF │ B7 D2 9C FF │ B7 D3 9C FF │ B6 D2 9B FF │ B6 D2 9B FF │ B6 D2 9B FF │ B6 D2 9B FF │ B6 D2 9B FF │ B6 D2 9B FF │ B5 D1 9A FF │ B6 D2 9A FF │ B6 D2 9A FF  .Ҝ ..Ҝ ..Ӝ ..қ ..қ ..қ ..қ ..қ ..қ ..њ ..Қ ..Қ .
27A0BCBA0B6 D2 9A FF │ B6 D2 9A FF │ B6 D2 9A FF │ B6 D1 9A FF │ B6 D1 9A FF │ B0 C9 93 FF │ 98 AD 7B FF │ B6 D1 99 FF │ B7 D1 9A FF │ B7 D1 9A FF │ B7 D1 9A FF │ B7 D1 9A FF  .Қ ..Қ ..Қ ..њ ..њ ..ɓ ...{..љ ..њ ..њ ..њ ..њ .
27A0BCBD0B8 D1 9A FF │ B8 D1 9A FF │ B8 D1 9A FF │ B8 D2 9A FF │ B7 D2 99 FF │ B7 D2 99 FF │ B7 D2 99 FF │ B8 D2 99 FF │ B8 D2 99 FF │ B7 D1 98 FF │ B7 D2 98 FF │ B7 D2 98 FF  .њ ..њ ..њ ..Қ ..ҙ ..ҙ ..ҙ ..ҙ ..ҙ ..ј ..Ҙ ..Ҙ .
27A0BCC00B7 D2 98 FF │ B6 D1 97 FF │ B6 D1 97 FF │ B6 D1 97 FF │ B6 D1 97 FF │ B7 D1 97 FF │ B7 D1 97 FF │ B7 D1 97 FF │ B7 D1 97 FF │ B7 D2 97 FF │ B8 D2 97 FF │ B7 D1 96 FF  .Ҙ ..ї ..ї ..ї ..ї ..ї ..ї ..ї ..ї ..җ ..җ ..і .
27A0BCC30B7 D1 96 FF │ B7 D1 96 FF │ B7 D1 96 FF │ B7 D0 95 FF │ B7 D0 95 FF │ B7 D0 95 FF │ B7 D0 95 FF │ B6 CF 94 FF │ B7 D0 94 FF │ B7 D0 94 FF │ B7 CF 94 FF │ B7 CF 94 FF  .і ..і ..і ..Е ..Е ..Е ..Е ..ϔ ..Д ..Д ..ϔ ..ϔ .
27A0BCC60B7 CF 94 FF │ B7 CF 94 FF │ B7 D0 94 FF │ B7 D0 94 FF │ B7 D0 94 FF │ B7 D0 94 FF │ B6 CF 93 FF │ B7 D0 93 FF │ B7 D0 93 FF │ B7 D0 93 FF │ B6 D0 92 FF │ B6 D0 92 FF  .ϔ ..ϔ ..Д ..Д ..Д ..Д ..ϓ ..Г ..Г ..Г ..В ..В .
27A0BCC90B6 D0 92 FF │ B7 D1 92 FF │ B6 D0 91 FF │ B6 D0 91 FF │ B6 D0 91 FF │ B6 CF 91 FF │ 97 AA 72 FF │ B3 CA 8D FF │ B7 CF 91 FF │ B7 CF 91 FF │ B7 CF 91 FF │ B7 CF 91 FF  .В ..ђ ..Б ..Б ..Б ..ϑ ...r..ʍ ..ϑ ..ϑ ..ϑ ..ϑ .
27A0BCCC0B7 CF 91 FF │ B7 CF 90 FF │ B7 CF 90 FF │ B7 CF 90 FF │ B7 CF 90 FF │ B6 CE 8F FF │ B6 CE 8F FF │ B6 CE 8F FF │ B6 CF 8F FF │ B6 CE 8F FF │ B6 CE 8F FF │ B6 CE 8F FF  .ϑ ..ϐ ..ϐ ..ϐ ..ϐ ..Ώ ..Ώ ..Ώ ..Ϗ ..Ώ ..Ώ ..Ώ .
27A0BCCF0B6 CE 8F FF │ B5 CD 8E FF │ B5 CD 8E FF │ B6 CE 8E FF │ B6 CE 8E FF │ B6 CE 8E FF │ B6 CE 8E FF │ B6 CE 8E FF │ B6 CE 8E FF │ B5 CD 8D FF │ B5 CD 8D FF │ B6 CE 8D FF  .Ώ .. ͎ .. ͎ ..Ύ ..Ύ ..Ύ ..Ύ ..Ύ ..Ύ .. ͍ .. ͍ ... .
27A0BCD20B5 CD 8C FF │ B5 CD 8C FF │ B5 CD 8C FF │ B5 CD 8C FF │ B5 CD 8C FF │ B6 CD 8C FF │ B6 CD 8C FF │ B6 CD 8C FF │ B5 CC 8B FF │ B5 CC 8B FF │ B5 CC 8B FF │ B6 CC 8B FF  . ͌ .. ͌ .. ͌ .. ͌ .. ͌ .. ͌ .. ͌ .. ͌ .. ̋ .. ̋ .. ̋ .. ̋ .
27A0BCD50B6 CC 8B FF │ B6 CD 8B FF │ B6 CD 8B FF │ B6 CD 8B FF │ 9B AC 70 FF │ AD C2 82 FF │ B5 CC 8A FF │ B5 CC 8A FF │ B4 CC 89 FF │ B4 CC 89 FF │ B4 CC 89 FF │ B4 CC 88 FF  . ̋ .. ͋ .. ͋ .. ͋ ...p... .. ̊ .. ̊ .. ̉ .. ̉ .. ̉ .. ̈ .
27A0BCD80B5 CB 89 FF │ B5 CB 89 FF │ B5 CB 89 FF │ B5 CB 89 FF │ B5 CB 89 FF │ B5 CB 89 FF │ B4 CA 88 FF │ B4 CA 88 FF │ B4 CA 88 FF │ B4 CA 88 FF │ B4 CA 88 FF │ B4 CA 88 FF  .ˉ ..ˉ ..ˉ ..ˉ ..ˉ ..ˉ ..ʈ ..ʈ ..ʈ ..ʈ ..ʈ ..ʈ .
27A0BCDB0B4 CB 88 FF │ B4 CB 88 FF │ B4 CB 88 FF │ B4 CB 88 FF │ B5 CB 88 FF │ B5 CB 88 FF │ B4 CA 87 FF │ B4 CA 87 FF │ B4 CA 87 FF │ B4 CA 87 FF │ B4 CA 87 FF │ B4 CA 87 FF  .ˈ ..ˈ ..ˈ ..ˈ ..ˈ ..ˈ ..ʇ ..ʇ ..ʇ ..ʇ ..ʇ ..ʇ .
27A0BCDE0B4 CA 87 FF │ B3 C9 86 FF │ B3 C9 86 FF │ B3 C9 86 FF │ B3 C9 86 FF │ B3 C9 86 FF │ B3 C9 86 FF │ B3 C9 86 FF │ AB BF 7E FF │ 99 AA 6C FF │ B3 C9 86 FF │ B3 C8 86 FF  .ʇ ..Ɇ ..Ɇ ..Ɇ ..Ɇ ..Ɇ ..Ɇ ..Ɇ ...~...l..Ɇ ..Ȇ .
27A0BCE10B3 C8 86 FF │ B3 C8 86 FF │ B3 C8 86 FF │ B3 C8 86 FF │ B3 C8 86 FF │ B3 C8 86 FF │ B3 C8 86 FF │ B3 C8 86 FF │ B3 C8 86 FF │ B2 C8 85 FF │ B2 C8 85 FF │ B2 C8 85 FF  .Ȇ ..Ȇ ..Ȇ ..Ȇ ..Ȇ ..Ȇ ..Ȇ ..Ȇ ..Ȇ ..ȅ ..ȅ ..ȅ .
27A0BCE40B2 C8 85 FF │ B2 C8 85 FF │ B1 C8 86 FF │ B1 C8 86 FF │ B1 C8 86 FF │ B1 C8 86 FF │ B1 C8 86 FF │ B1 C8 86 FF │ B1 C7 86 FF │ B1 C7 86 FF │ B1 C7 86 FF │ B1 C7 86 FF  .ȅ ..ȅ ..Ȇ ..Ȇ ..Ȇ ..Ȇ ..Ȇ ..Ȇ ..ǆ ..ǆ ..ǆ ..ǆ .
27A0BCE70B0 C7 86 FF │ B0 C7 86 FF │ B0 C7 86 FF │ B0 C7 86 FF │ B0 C7 86 FF │ B0 C7 86 FF │ B0 C7 86 FF │ B0 C7 86 FF │ B0 C7 86 FF │ B0 C6 86 FF │ B0 C6 86 FF │ B0 C6 86 FF  .ǆ ..ǆ ..ǆ ..ǆ ..ǆ ..ǆ ..ǆ ..ǆ ..ǆ ..Ɔ ..Ɔ ..Ɔ .
27A0BCEA0B0 C6 86 FF │ AF C5 85 FF │ 96 A7 6C FF │ A5 B9 7B FF │ B0 C6 86 FF │ B0 C6 86 FF │ B0 C6 86 FF │ B0 C6 86 FF │ AF C6 86 FF │ AF C5 86 FF │ AF C5 86 FF │ AF C5 86 FF  .Ɔ ..Ņ ...l...{..Ɔ ..Ɔ ..Ɔ ..Ɔ ..Ɔ ..ņ ..ņ ..ņ .
27A0BCED0AF C5 86 FF │ AF C5 86 FF │ AF C5 86 FF │ AF C5 86 FF │ AF C5 86 FF │ AF C5 86 FF │ AE C5 86 FF │ AE C5 86 FF │ AE C5 86 FF │ AE C5 86 FF │ AE C5 86 FF │ AE C5 86 FF  .ņ ..ņ ..ņ ..ņ ..ņ ..ņ ..ņ ..ņ ..ņ ..ņ ..ņ ..ņ .
27A0BCF00AE C5 86 FF │ AE C5 86 FF │ AE C5 86 FF │ AE C5 86 FF │ AD C5 87 FF │ AD C5 87 FF │ AD C5 87 FF │ AD C5 87 FF │ AD C4 87 FF │ AD C4 87 FF │ AD C4 87 FF │ AC C4 87 FF  .ņ ..ņ ..ņ ..ņ ..Ň ..Ň ..Ň ..Ň ..ć ..ć ..ć ..ć .
27A0BCF30AC C4 87 FF │ AC C4 87 FF │ AC C4 87 FF │ AC C3 87 FF │ AC C3 87 FF │ AC C3 87 FF │ AC C3 87 FF │ AC C3 87 FF │ AC C3 87 FF │ AC C3 87 FF │ AC C2 87 FF │ AC C2 87 FF  .ć ..ć ..ć ..Ç ..Ç ..Ç ..Ç ..Ç ..Ç ..Ç ... ... .
27A0BCF60AC C2 87 FF │ AC C2 87 FF │ AB C2 87 FF │ A8 BE 84 FF │ 92 A4 6E FF │ 97 A9 74 FF │ AA C0 87 FF │ AB C1 88 FF │ AB C1 88 FF │ AA C1 88 FF │ AA C1 88 FF │ AA C1 88 FF  .. ... ... .......n...t.........................

talaj commented 4 years ago

Are you using GDAL? Do you limit GDAL_CACHEMAX? It takes 5% of RAM by default which can easily take 50 GB of memory with 10 workers on a machine with 100 GB RAM.

I was profiling memory use of Mapnik several times and it was always GDAL who took the memory. I was using valgrind massif heap profiler.

zdila commented 4 years ago

Yes, for shading we use gdal. Thank you for the information. I will try it and close the issue if it helps :).

zdila commented 4 years ago

BTW we use two processes to render map. One is for pre-rendering, running with low priority (nice) and other is for on-demand rendering. Both use 8 mapnik instances. Server has 64 GB RAM. 64 0.05 (8 + 8) = 51.2 so from you describe it can easily get OOM. BTW can't it be configured so that all mapnik instances would share GDAL cache?

talaj commented 4 years ago

can't it be configured so that all mapnik instances would share GDAL cache?

I think the cache is per process, so if you will render in threads, then the cache will be shared. But aren't you using node-mapnik so it should already be shared?

I'm setting it by an environment variable to 256 MB (GDAL_CACHEMAX=256) but I think you can use even smaller values. There is also linux disk cache so the cache is duplicated partially.

zdila commented 4 years ago

Env. var GDAL_CACHEMAX=256 doesn't seem to help in our case.

zdila commented 4 years ago

Problem seems to be caused by https://github.com/mapnik/node-mapnik/issues/950.

zdila commented 3 years ago

Note that https://github.com/mapnik/node-mapnik/issues/950#issuecomment-674031716 fixed lot of memory leak but there is still some.

mapnik / mapnik-support

Is there a way to detect memory leak in Mapnik? #138