search

HaxeFoundation / haxe

Haxe - The Cross-Platform Toolkit
https://haxe.org
6.16k stars 656 forks source link

Compilation server memory leak #4033

Closed waneck closed 7 years ago

waneck commented 9 years ago

At first I thought it was something wrong with Java/C# (and I think indeed it was - it had to do with I might be doing something wrong, and I think the libs weren't flushing, which indeed made the situation worst (btw, -swf-lib might have the same issue). But as I solved this (on e16c481474ed58ab58bd843b6d60db6ad9fd1b57 ), it still leaks memory.

I've also tried reverting to a revision before ca3a08430fc1cb11194e9eda95d5fa0a1c60ea6c , but sadly it still happens:

wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:    38480 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   237868 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   317964 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   382108 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   443296 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   501924 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   545472 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   631144 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   657132 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   672884 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   672884 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   734912 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   837372 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   867384 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:   949648 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1009828 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1074844 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1142940 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1193872 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1247816 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % haxe --display memory                                                                                                                                                                                                       
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1325780 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1325780 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1369328 kB
wnk-comtacti :: haxe/tests/unit ‹8d6351f*› % cat /proc/17282/status | grep VmSize && haxe compile-neko.hxml                                                                                                                                                              
VmSize:  1532188 kB

As you can see, the memory after a few runs is already occupying 1.5GB, even after haxe --display memory is invoked. --display memory shows the following result:

Total Allocated Memory 0 B
Free Memory 0 B
Total cache size 56.1 MB
  haxelib 176 B
  parsed ast 18.4 MB (700 files stored)
  typed modules 43.4 MB (644 modules stored)
    --- CONFIG 518d3b0bab658da4da1b82f3a2f71cbe ----------------------------
    unit.MyInterface : 1016 B
    haxe.io.BytesData : 1 KB
    EnumValue : 1 KB
    Class : 1 KB
    Enum : 1 KB
    unit.RBNode_unit_MyData : 1 KB
    unit.MyGeneric2_unit_MyRandomClass : 1 KB
    unit.MyUsing1 : 1 KB
    unit.RBTree_unit_MyData : 2 KB
    unit.MyGeneric2_unit_MyGeneric_unit_MyRandomClass : 2 KB
    sys.io.FileSeek : 2 KB
    haxe.io.Eof : 2 KB
    haxe.ds.Option : 2 KB
    unit.SimpleEnum : 2 KB
    haxe.PosInfos : 2 KB
    haxe.macro.Tools : 2 KB
    unit.MyEnum : 2 KB
    unit.MyGeneric_unit_MyRandomClass : 2 KB
    unit.MyGeneric_String : 2 KB
    haxe.io.Error : 2 KB
    unit.MyGeneric_Int : 2 KB
    unit.MyUsing2 : 3 KB
    unit.MySubClass : 3 KB
    sys.FileStat : 3 KB
    sys.db.Connection : 4 KB
    sys.db.ResultSet : 4 KB
    IntIterator : 5 KB
    haxe.Constraints : 5 KB
    haxe.Json : 6 KB
    neko.NativeString : 6 KB
    unit.issues.misc.Issue2720Macro : 7 KB
    haxe.Log : 8 KB
    haxe.ds.WeakMap : 8 KB
    StdTypes : 8 KB
    sys.db.Object : 9 KB
    unit.TestOrder : 9 KB
    haxe.Timer : 10 KB
    haxe.crypto.Md5 : 10 KB
    neko.vm.Tls : 11 KB
    haxe.io.BytesOutput : 11 KB
    unit.issues.Issue2766 : 11 KB
    haxe.rtti.Meta : 12 KB
    unit.issues.Issue2558 : 12 KB
    unit.issues.Issue2982 : 14 KB
    unit.issues.Issue2958 : 14 KB
    unit.issues.Issue3387 : 15 KB
    unit.issues.Issue2640 : 16 KB
    neko.NativeArray : 16 KB
    haxe.Resource : 17 KB
    sys.db.RecordInfos : 17 KB
    Issue1497DefinedClass : 17 KB
    unit.issues.Issue2521 : 18 KB
    unit.issues.misc.Issue3089Macro : 22 KB
    unit.MyMacro : 22 KB
    haxe.io.BytesInput : 23 KB
    StringBuf : 24 KB
    haxe.EnumTools : 24 KB
    haxe.ds.EnumValueMap : 25 KB
    haxe.ds.GenericStack : 25 KB
    haxe.macro.ComplexTypeTools : 25 KB
    haxe.ds.HashMap : 25 KB
    unit.issues.Issue3731 : 25 KB
    sys.io.FileOutput : 26 KB
    haxe.io.BytesBuffer : 27 KB
    haxe.ds.IntMap : 28 KB
    sys.io.FileInput : 29 KB
    haxe.ds.StringMap : 29 KB
    sys.io.File : 30 KB
    haxe.crypto.BaseCode : 30 KB
    unit.issues.Issue2634 : 32 KB
    unit.TestGeneric : 32 KB
    haxe.macro.PositionTools : 33 KB
    unit.issues.Issue3198 : 34 KB
    unit.TestIssues : 35 KB
    Map : 36 KB
    unit.issues.misc.Issue2003Macro : 36 KB
    unit.issues.Issue3914 : 36 KB
    haxe.ds.ObjectMap : 39 KB
    unit.TestSpecification : 39 KB
    unit.issues.Issue3365 : 39 KB
    haxe.macro.MacroStringTools : 40 KB
    Std : 41 KB
    unit.issues.Issue2629 : 41 KB
    unit.issues.Issue3921 : 42 KB
    haxe.CallStack : 44 KB
    unit.issues.Issue3346 : 44 KB
    unit.TestResource : 48 KB
    haxe.Utf8 : 49 KB
    Lambda : 50 KB
    unit.TestGADT : 51 KB
    Date : 51 KB
    haxe.io.FPHelper : 54 KB
    List : 57 KB
    StringTools : 58 KB
    sys.FileSystem : 61 KB
    haxe.io.Output : 64 KB
    haxe.xml.Printer : 64 KB
    unit.issues.misc.Issue1497Macro : 65 KB
    Math : 66 KB
    haxe.macro.Expr : 66 KB
    neko.Lib : 67 KB
    neko.Boot : 72 KB
    haxe.crypto.Sha1 : 74 KB
    haxe.io.Path : 75 KB
    haxe.macro.Type : 75 KB
    String : 76 KB
    haxe.Int32 : 77 KB
    haxe.io.Input : 79 KB
    DateTools : 82 KB
    unit.issues.Issue2990 : 85 KB
    Sys : 85 KB
    haxe.io.Bytes : 87 KB
    Xml : 95 KB
    EReg : 96 KB
    haxe.ds.BalancedTree : 102 KB
    unit.MyClass : 106 KB
    UInt : 106 KB
    unit.TestLocals : 112 KB
    unit.issues.misc.Issue3183Macro : 115 KB
    haxe.macro.Context : 122 KB
    haxe.format.JsonParser : 123 KB
    unit.TestMeta : 128 KB
    unit.TestJson : 131 KB
    Array : 150 KB
    unit.TestOps : 151 KB
    haxe.macro.TypeTools : 151 KB
    Reflect : 158 KB
    unit.TestBytes : 161 KB
    haxe.Unserializer : 161 KB
    unit.TestDCE : 163 KB
    Type : 164 KB
    haxe.format.JsonPrinter : 171 KB
    haxe.xml.Parser : 174 KB
    neko.Web : 177 KB
    unit.TestEReg : 196 KB
    unit.MyAbstract : 196 KB
    unit.TestIO : 201 KB
    unit.TestSerialize : 228 KB
    unit.Test : 234 KB
    haxe.Serializer : 235 KB
    haxe.Template : 269 KB
    haxe.macro.ExprTools : 271 KB
    unit.TestXML : 287 KB
    unit.UnitBuilder : 296 KB
    haxe.macro.TypedExprTools : 305 KB
    haxe.Int64 : 319 KB
    haxe.macro.Printer : 408 KB
    unit.TestReflect : 467 KB
    unit.TestMisc : 507 KB
    sys.db.Manager : 648 KB
    unit.TestBasetypes : 663 KB
    unit.TestMatch : 800 KB
    unit.TestInt64 : 941 KB
    unit.TestType : 1.3 MB
    sys.db.RecordMacros : 1.4 MB
    --- CONFIG fe8633a4fc98b718b0e1c60e2d419c03 ----------------------------
    haxe.io.BytesData : 1 KB
    EnumValue : 1 KB
    unit.MyInterface : 1 KB
    Issue1497DefinedClass : 1 KB
    unit.issues.misc.Issue1497Macro : 1 KB
    Class : 1 KB
    unit.issues.misc.Issue2720Macro : 1 KB
    unit.TestIssues : 1 KB
    Enum : 1 KB
    unit.issues.Issue3383 : 1 KB
    unit.issues.Issue3226 : 1 KB
    unit.issues.Issue2856 : 1 KB
    unit.issues.Issue2648 : 1 KB
    unit.issues.Issue3084 : 1 KB
    unit.issues.Issue2772 : 1 KB
    unit.issues.Issue2980 : 1 KB
    unit.issues.Issue3171 : 1 KB
    unit.issues.Issue3558 : 1 KB
    unit.issues.Issue3306 : 1 KB
    unit.issues.Issue1925 : 1 KB
    unit.issues.Issue2049 : 1 KB
    unit.issues.Issue3842 : 1 KB
    unit.issues.Issue2927 : 1 KB
    unit.issues.Issue3400 : 1 KB
    unit.issues.Issue3946 : 1 KB
    unit.issues.Issue3173 : 1 KB
    unit.MyUsing1 : 2 KB
    unit.MyMacro : 2 KB
    unit.issues._Issue2085.Node_unit_issues__Issue2085_Object_unit_issues__Issue2085_Ar_unit_issues__Issue2085_Object : 2 KB
    sys.io.FileSeek : 2 KB
    unit.issues._Issue2581.Arrayable_unit_issues__Issue2581_KVPair_String_String : 2 KB
    unit.RBNode_unit_MyData : 2 KB
    unit.issues._Issue3557.Generic_unit_issues__Issue3557_MyBaseClass : 2 KB
    unit.issues.Issue3585 : 2 KB
    unit.issues.misc.Issue2152Class : 2 KB
    unit.MyGeneric2_unit_MyRandomClass : 2 KB
    unit.issues._Issue3155.A_Int : 2 KB
    unit.issues._Issue2016.Gen_String : 2 KB
    unit.issues._Issue2044.Gen_String : 2 KB
    unit.issues._Issue2044.Gen_Int : 2 KB
    unit.issues._Issue2016.Gen_Int : 2 KB
    unit.issues._Issue2044.Gen_Bool : 2 KB
    unit.issues._Issue3615.BaseClass_Int : 2 KB
    unit.issues.misc.Issue3089Macro : 2 KB
    unit.issues.Issue3979 : 2 KB
    unit.issues.Issue3451 : 2 KB
    unit.issues.Issue3771 : 2 KB
    unit.issues.Issue3894 : 2 KB
    unit.issues.Issue3861 : 2 KB
    unit.issues.Issue3480 : 2 KB
    unit.RBTree_unit_MyData : 2 KB
    unit.issues._Issue3615.BaseClass_String : 2 KB
    unit.SimpleEnum : 2 KB
    haxe.ds.Option : 2 KB
    unit.issues.misc.Issue3133Class : 2 KB
    unit.MyGeneric2_unit_MyGeneric_unit_MyRandomClass : 2 KB
    unit.issues.misc.Issue3008Class : 2 KB
    haxe.PosInfos : 2 KB
    haxe.io.Eof : 2 KB
    unit.MyEnum : 2 KB
    haxe.zip.FlushMode : 2 KB
    unit.issues._Issue2581.KVPair_String_String : 2 KB
    haxe.extern.EitherType : 3 KB
    unit.issues._Issue3109.C1_unit_issues_TT : 3 KB
    haxe.io.Error : 3 KB
    unit.issues._Issue2085.Ar_unit_issues__Issue2085_Object : 3 KB
    unit.MyGeneric_unit_MyRandomClass : 3 KB
    unit.MyGeneric_String : 3 KB
    unit.MyGeneric_Int : 3 KB
    unit.issues._Issue3005.B_String : 3 KB
    unit.issues.Issue3008 : 3 KB
    unit.MyUsing2 : 3 KB
    unit.issues._Issue3005.A_String : 3 KB
    unit.issues._Issue3109.C2_unit_issues__Issue3109_SC1_unit_issues_TT : 3 KB
    sys.FileStat : 3 KB
    unit.MySubClass : 4 KB
    unit.issues.MyTuple_2 : 4 KB
    unit.issues._Issue3109.SC2_unit_issues_TT : 4 KB
    unit.issues._Issue2653.BasicTexture_Int : 4 KB
    unit.issues.misc.Issue2003Macro : 4 KB
    unit.issues._Issue2086.Foo_unit_issues__Issue2086_StringNode : 4 KB
    unit.issues.Issue2881 : 4 KB
    unit.issues.Issue3150 : 4 KB
    unit.issues.Issue3545 : 4 KB
    unit.issues.Issue2844 : 4 KB
    unit.issues.Issue2720 : 4 KB
    unit.issues.misc.Issue2497Class : 4 KB
    unit.issues.Issue3531 : 4 KB
    unit.issues._Issue3090.A_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer_haxe_xml_Printer : 4 KB
    unit.issues._Issue2902.G_String : 4 KB
    unit.issues.Issue1497 : 4 KB
    unit.issues.Issue3391 : 4 KB
    unit.issues.Issue3388 : 5 KB
    unit.issues.Issue2982 : 5 KB
    unit.issues._Issue2902.G_Int : 5 KB
    sys.db.Connection : 5 KB
    unit.issues.Issue3719 : 5 KB
    unit.issues.Issue2725 : 5 KB
    unit.issues.Issue3950 : 5 KB
    unit.issues.Issue3387 : 5 KB
    unit.issues.Issue2558 : 5 KB
    sys.db.ResultSet : 5 KB
    unit.issues.Issue2271 : 5 KB
    unit.issues.Issue2668 : 5 KB
    unit.issues.Issue3380 : 5 KB
    unit.issues.Issue2845 : 5 KB
    unit.issues.Issue3844 : 5 KB
    unit.issues.Issue2048 : 5 KB
    unit.issues.misc.Issue3679Abstract : 5 KB
    unit.issues.Issue3921 : 5 KB
    unit.issues.Issue3687 : 5 KB
    unit.issues.Issue2958 : 5 KB
    unit.issues.Issue3138 : 5 KB
    unit.issues.Issue3110 : 6 KB
    unit.issues.Issue3133 : 6 KB
    unit.issues.Issue2989 : 6 KB
    unit.issues.Issue3039 : 6 KB
    unit.issues.Issue3181 : 6 KB
    unit.issues._Issue2086.Node_unit_issues__Issue2086_StringNode : 6 KB
    haxe.Json : 6 KB
    neko.NativeString : 6 KB
    IntIterator : 6 KB
    unit.issues.Issue2766 : 6 KB
    unit.issues.Issue2785 : 6 KB
    unit.issues.Issue3679 : 6 KB
    unit.issues.Issue3132 : 6 KB
    unit.issues.Issue2735 : 6 KB
    unit.issues._Issue2581.MMap_String_String : 7 KB
    unit.issues.Issue3440 : 7 KB
    haxe.Constraints : 7 KB
    unit.issues.Issue2960 : 7 KB
    unit.issues.Issue3577 : 7 KB
    unit.issues.Issue3799 : 7 KB
    unit.issues.Issue3547 : 7 KB
    unit.issues.Issue3486 : 7 KB
    unit.issues.Issue3030 : 7 KB
    unit.issues.Issue3217 : 7 KB
    unit.issues.Issue2085 : 7 KB
    unit.issues.Issue3461 : 7 KB
    unit.issues.Issue3303 : 7 KB
    unit.hxcpp_issues.Issue173 : 7 KB
    unit.issues.Issue2380 : 7 KB
    unit.issues.Issue2401 : 7 KB
    unit.issues.Issue3365 : 7 KB
    unit.issues.Issue3615 : 7 KB
    unit.issues.Issue3447 : 7 KB
    unit.issues.Issue3588 : 7 KB
    unit.issues.Issue2282 : 7 KB
    unit.issues.Issue3557 : 7 KB
    unit.issues.Issue2809 : 8 KB
    unit.issues.Issue2941 : 8 KB
    unit.issues.Issue3732 : 8 KB
    unit.issues.Issue3344 : 8 KB
    unit.issues.Issue3731 : 8 KB
    unit.issues.Issue2722 : 8 KB
    unit.issues.Issue3024 : 8 KB
    unit.issues.Issue3364 : 8 KB
    haxe.Log : 8 KB
    unit.issues.Issue2640 : 8 KB
    unit.issues.Issue3788 : 8 KB
    unit.issues.Issue3005 : 8 KB
    unit.issues.Issue3094 : 8 KB
    unit.issues.Issue3665 : 8 KB
    unit.issues.Issue3543 : 8 KB
    unit.issues.Issue2696 : 8 KB
    unit.issues.Issue2003 : 8 KB
    haxe.ds.WeakMap : 8 KB
    unit.issues.Issue3448 : 8 KB
    unit.issues.Issue2634 : 9 KB
    unit.issues.Issue2607 : 9 KB
    unit.issues.Issue3115 : 9 KB
    unit.issues.Issue2497 : 9 KB
    unit.issues.Issue2871 : 9 KB
    unit.issues.Issue2521 : 9 KB
    unit.issues.Issue2752 : 9 KB
    unit.issues.Issue2254 : 9 KB
    unit.issues.Issue3155 : 9 KB
    unit.issues.Issue3390 : 9 KB
    unit.issues.Issue2859 : 9 KB
    StdTypes : 9 KB
    unit.issues.Issue4002 : 9 KB
    unit.issues.Issue3198 : 9 KB
    unit.issues.Issue3212 : 9 KB
    unit.issues.Issue3431 : 9 KB
    unit.issues.Issue2152 : 9 KB
    unit.issues.Issue3554 : 9 KB
    unit.issues.Issue3127 : 9 KB
    unit.issues.Issue3892 : 9 KB
    unit.issues.Issue3586 : 9 KB
    unit.issues.Issue4001 : 9 KB
    unit.issues.Issue3777 : 9 KB
    unit.issues.Issue3090 : 9 KB
    unit.issues.Issue3054 : 9 KB
    unit.issues.Issue2675 : 10 KB
    unit.issues.Issue2409 : 10 KB
    unit.issues.Issue2900 : 10 KB
    unit.issues.Issue2728 : 10 KB
    unit.issues.Issue3218 : 10 KB
    unit.issues.Issue2957 : 10 KB
    sys.db.Object : 10 KB
    unit.issues.Issue2712 : 10 KB
    unit.issues.Issue3333 : 10 KB
    unit.issues.Issue3089 : 10 KB
    unit.issues.Issue2846 : 10 KB
    unit.issues.Issue3713 : 10 KB
    unit.issues.Issue2304 : 10 KB
    unit.issues.Issue3042 : 10 KB
    unit.issues.Issue3846 : 10 KB
    unit.issues.Issue2343 : 10 KB
    unit.issues.Issue2184 : 10 KB
    unit.issues.Issue3280 : 10 KB
    unit.issues.Issue2988 : 10 KB
    unit.issues.Issue3578 : 11 KB
    unit.issues.Issue2823 : 11 KB
    unit.issues.Issue3742 : 11 KB
    unit.issues.Issue3338 : 11 KB
    unit.issues.Issue2716 : 11 KB
    unit.issues.Issue2813 : 11 KB
    unit.issues.Issue2755 : 11 KB
    unit.issues.Issue2939 : 11 KB
    unit.issues.Issue3203 : 11 KB
    unit.issues.Issue2540 : 11 KB
    unit.TestOrder : 11 KB
    unit.issues.Issue3118 : 11 KB
    unit.issues.Issue2914 : 11 KB
    unit.issues.Issue2235 : 12 KB
    unit.issues.Issue2581 : 12 KB
    unit.issues.Issue2645 : 12 KB
    unit.issues.Issue2979 : 12 KB
    unit.issues.Issue2570 : 12 KB
    unit.issues.Issue3346 : 12 KB
    haxe.io.BytesOutput : 12 KB
    unit.issues.Issue2478 : 12 KB
    haxe.EnumFlags : 12 KB
    unit.issues.Issue3680 : 12 KB
    unit.issues.Issue2828 : 12 KB
    unit.issues.Issue3081 : 12 KB
    unit.issues.Issue2810 : 13 KB
    haxe.rtti.Meta : 13 KB
    unit.issues.Issue3244 : 13 KB
    unit.issues.Issue1997 : 13 KB
    unit.issues.Issue3635 : 13 KB
    unit.issues.Issue3575 : 13 KB
    unit.issues.Issue2625 : 13 KB
    unit.issues.Issue3806 : 13 KB
    unit.issues.Issue3061 : 13 KB
    unit.issues.Issue2951 : 13 KB
    unit.issues.Issue2622 : 13 KB
    unit.issues.Issue523 : 13 KB
    unit.issues.Issue3178 : 14 KB
    unit.issues.Issue3197 : 14 KB
    unit.issues.Issue2776 : 14 KB
    unit.issues.Issue1492 : 14 KB
    unit.issues.Issue2999 : 14 KB
    unit.issues.Issue2653 : 14 KB
    unit.issues.Issue3123 : 14 KB
    unit.issues.Issue3159 : 14 KB
    unit.issues.Issue2664 : 14 KB
    unit.issues.Issue3023 : 14 KB
    unit.issues.Issue3389 : 14 KB
    unit.issues.Issue2763 : 14 KB
    unit.issues.Issue3658 : 14 KB
    unit.issues.Issue2902 : 14 KB
    unit.issues.Issue4014 : 14 KB
    unit.issues.Issue3649 : 14 KB
    unit.issues.Issue2917 : 15 KB
    unit.issues.Issue2236 : 15 KB
    unit.issues.Issue2619 : 15 KB
    unit.issues.Issue3681 : 15 KB
    unit.issues.Issue3248 : 15 KB
    unit.issues.Issue2754 : 15 KB
    sys.db.Types : 15 KB
    unit.issues.Issue3437 : 15 KB
    unit.issues.Issue3109 : 15 KB
    unit.issues.Issue3860 : 15 KB
    unit.issues.Issue2016 : 15 KB
    unit.issues.Issue1810 : 15 KB
    unit.issues.Issue2835 : 15 KB
    unit.issues.Issue3753 : 15 KB
    unit.issues.Issue2086 : 15 KB
    unit.issues.Issue3113 : 16 KB
    unit.issues.Issue2685 : 16 KB
    unit.issues.Issue2718 : 16 KB
    unit.issues.Issue2907 : 16 KB
    unit.issues.Issue2933 : 16 KB
    neko.NativeArray : 16 KB
    unit.issues.Issue3967 : 16 KB
    sys.db.RecordInfos : 17 KB
    unit.issues.Issue2044 : 17 KB
    unit.issues.Issue2639 : 17 KB
    unit.issues.Issue2627 : 17 KB
    unit.issues.Issue3191 : 18 KB
    unit.issues.Issue2629 : 18 KB
    unit.issues.Issue3948 : 18 KB
    unit.issues.Issue2633 : 18 KB
    unit.issues.Issue2224 : 18 KB
    unit.issues.Issue3516 : 19 KB
    unit.issues.Issue3214 : 19 KB
    unit.issues.Issue2786 : 19 KB
    unit.issues.Issue3845 : 19 KB
    unit.issues.Issue3507 : 19 KB
    haxe.Resource : 19 KB
    unit.issues.Issue3818 : 19 KB
    unit.issues.Issue4000 : 20 KB
    unit.issues.Issue2623 : 20 KB
    unit.issues.Issue3247 : 20 KB
    unit.issues.Issue3616 : 20 KB
    unit.issues.Issue2580 : 21 KB
    unit.issues.Issue3504 : 22 KB
    unit.issues.Issue2157 : 22 KB
    unit.issues.Issue2042 : 22 KB
    unit.issues.Issue3063 : 22 KB
    unit.issues.Issue3592 : 23 KB
    unit.issues.Issue2564 : 23 KB
    unit.issues.Issue3717 : 23 KB
    unit.issues.Issue3183 : 23 KB
    unit.issues.Issue2614 : 24 KB
    unit.issues.Issue3345 : 24 KB
    unit.issues.Issue2378 : 24 KB
    unit.issues.Issue2750 : 24 KB
    unit.issues.Issue3914 : 25 KB
    haxe.io.BytesInput : 25 KB
    unit.issues.Issue2936 : 25 KB
    unit.issues.Issue2713 : 25 KB
    haxe.ds.HashMap : 26 KB
    haxe.ds.GenericStack : 26 KB
    haxe.EnumTools : 26 KB
    unit.issues.Issue2874 : 27 KB
    unit.issues.Issue3031 : 27 KB
    unit.issues.Issue3513 : 27 KB
    unit.issues.Issue3263 : 27 KB
    unit.issues.Issue2130 : 28 KB
    haxe.ds.Vector : 28 KB
    haxe.io.BytesBuffer : 28 KB
    unit.issues.Issue2396 : 28 KB
    haxe.ds.IntMap : 29 KB
    unit.issues.Issue3494 : 29 KB
    unit.issues.Issue3348 : 29 KB
    unit.issues.Issue3376 : 29 KB
    unit.issues.Issue2317 : 29 KB
    unit.issues.Issue2646 : 30 KB
    unit.issues.Issue2584 : 30 KB
    unit.issues.Issue2792 : 30 KB
    haxe.ds.StringMap : 30 KB
    unit.issues.Issue3432 : 30 KB
    unit.issues.Issue2861 : 30 KB
    unit.issues.Issue3970 : 30 KB
    unit.issues.Issue1845 : 30 KB
    haxe.crypto.Adler32 : 31 KB
    unit.issues.Issue3639 : 31 KB
    unit.issues.Issue3637 : 31 KB
    unit.issues.Issue3370 : 33 KB
    unit.issues.Issue2778 : 33 KB
    unit.issues.Issue2281 : 33 KB
    unit.TestGeneric : 35 KB
    unit.issues.Issue2063 : 35 KB
    unit.issues.Issue2676 : 36 KB
    haxe.crypto.Hmac : 36 KB
    unit.issues.Issue3403 : 36 KB
    unit.issues.Issue3512 : 36 KB
    haxe.io.ArrayBufferView : 37 KB
    Map : 38 KB
    unit.issues.Issue3596 : 38 KB
    unit.issues.Issue2610 : 39 KB
    haxe.ds.ObjectMap : 40 KB
    unit.issues.Issue2698 : 42 KB
    unit.issues.Issue3071 : 43 KB
    haxe.CallStack : 44 KB
    unit.issues.Issue2663 : 44 KB
    unit.issues.Issue2889 : 45 KB
    unit.issues.Issue3088 : 46 KB
    unit.issues.Issue3579 : 46 KB
    unit.issues.Issue3360 : 47 KB
    unit.issues.Issue2736 : 48 KB
    unit.issues.Issue3205 : 48 KB
    haxe.io.UInt8Array : 52 KB
    Lambda : 53 KB
    unit.TestGADT : 56 KB
    haxe.io.Float64Array : 57 KB
    haxe.io.Float32Array : 57 KB
    List : 59 KB
    unit.issues.Issue2661 : 61 KB
    haxe.io.UInt16Array : 61 KB
    haxe.zip.Huffman : 62 KB
    unit.issues.Issue3377 : 63 KB
    haxe.io.Int32Array : 65 KB
    haxe.io.UInt32Array : 65 KB
    unit.issues.Issue2990 : 66 KB
    haxe.crypto.Md5 : 66 KB
    haxe.io.Output : 67 KB
    neko.vm.Lock : 67 KB
    haxe.macro.Expr : 68 KB
    neko.vm.Tls : 68 KB
    sys.db.TableCreate : 68 KB
    neko.Lib : 69 KB
    unit.MySpodClass : 70 KB
    haxe.xml.Printer : 70 KB
    neko.Boot : 73 KB
    haxe.ds.ArraySort : 75 KB
    unit.issues.Issue3462 : 75 KB
    neko.vm.Thread : 77 KB
    haxe.Int32 : 78 KB
    StringBuf : 81 KB
    haxe.crypto.Sha1 : 82 KB
    sys.io.FileOutput : 83 KB
    haxe.xml.Fast : 84 KB
    haxe.zip.Compress : 84 KB
    haxe.io.Path : 85 KB
    sys.io.FileInput : 86 KB
    haxe.zip.Uncompress : 86 KB
    haxe.crypto.BaseCode : 87 KB
    haxe.io.Bytes : 87 KB
    DateTools : 92 KB
    Xml : 104 KB
    UInt : 104 KB
    sys.io.File : 106 KB
    Std : 107 KB
    haxe.ds.BalancedTree : 107 KB
    haxe.Utf8 : 108 KB
    unit.issues.Issue3466 : 110 KB
    unit.issues.Issue2604 : 110 KB
    Date : 110 KB
    haxe.io.FPHelper : 111 KB
    unit.issues.Issue3029 : 111 KB
    unit.TestLocals : 113 KB
    unit.issues.Issue2272 : 115 KB
    StringTools : 116 KB
    haxe.crypto.Sha224 : 116 KB
    unit.issues.Issue3852 : 121 KB
    haxe.crypto.Sha256 : 122 KB
    haxe.format.JsonParser : 124 KB
    unit.issues.Issue3725 : 127 KB
    haxe.rtti.Rtti : 127 KB
    Math : 128 KB
    haxe.Timer : 130 KB
    unit.MyClass : 130 KB
    String : 133 KB
    haxe.io.Input : 135 KB
    sys.db.Sqlite : 136 KB
    Sys : 144 KB
    unit.TestResource : 148 KB
    EReg : 154 KB
    haxe.rtti.CType : 160 KB
    unit.issues.Issue3767 : 165 KB
    unit.TestBytes : 165 KB
    unit.TestDCE : 176 KB
    unit.issues.Issue3347 : 178 KB
    haxe.xml.Parser : 178 KB
    sys.FileSystem : 194 KB
    unit.issues.Issue2937 : 195 KB
    unit.issues.Issue2308 : 195 KB
    unit.issues.Issue3396 : 197 KB
    unit.issues.Issue2928 : 197 KB
    unit.TestEReg : 199 KB
    haxe.DynamicAccess : 200 KB
    unit.TestJson : 200 KB
    unit.TestOps : 200 KB
    haxe.zip.InflateImpl : 202 KB
    unit.TestIO : 204 KB
    haxe.ds.EnumValueMap : 211 KB
    Array : 214 KB
    unit.issues.Issue3822 : 215 KB
    haxe.Unserializer : 236 KB
    neko.Web : 268 KB
    unit.TestMeta : 277 KB
    haxe.macro.ExprTools : 278 KB
    unit.MyAbstract : 278 KB
    Reflect : 282 KB
    unit.TestXML : 296 KB
    Type : 302 KB
    unit.TestSerialize : 303 KB
    unit.issues.Issue2963 : 323 KB
    haxe.Int64 : 353 KB
    haxe.format.JsonPrinter : 394 KB
    haxe.macro.Printer : 416 KB
    unit.issues.Issue3292 : 427 KB
    haxe.Serializer : 440 KB
    haxe.rtti.XmlParser : 445 KB
    haxe.Template : 461 KB
    unit.TestMisc : 590 KB
    unit.TestSpod : 607 KB
    unit.TestReflect : 679 KB
    unit.Test : 780 KB
    unit.TestMatch : 824 KB
    unit.TestBasetypes : 908 KB
    sys.db.Manager : 989 KB
    unit.TestInt64 : 1.2 MB
    unit.TestType : 2.1 MB
    unit.TestSpecification : 9.1 MB
Cache dump complete

As the shell suggests, this happens on 8d6351f - which was before the compilation server changes. The same result happens on latest

waneck commented 9 years ago

Another strange thing is that on Java, some Issue modules are occupying 1+MB !

waneck commented 9 years ago

It just came to my mind that this can be a problem with our own unit tests (a bad behaving macro, perhaps?). @ncannasse, I know that you've been using the compilation server for some time now, can you try this on your bigger projects?

waneck commented 9 years ago

Good news - I made a simple project with a huge amount of trace("Hello,World"); and no leaks. Seems to have something to do with our unit tests macros then. It'd be great if we can find what's going on then - and improve haxe --display memory to include macro memory as well

ncannasse commented 9 years ago

The cache seems clean, 56MB is not much and there does not seem to be a leak there. There might be a leak in the macro context if we have a static var acccumulating some data (because the macro context is being reused between compilations, and is not listed in --display memory)

ncannasse commented 9 years ago

PS: you should be able to confirm that by measuring the memory size of Typer.macro_interp_cache

waneck commented 9 years ago

How do I do that?

ncannasse commented 9 years ago

Or if you want a fine detail, you can try to list the Interp context globals one by one, while excluding all the others (for cross references)

ncannasse commented 9 years ago

Common.mem_size Typer.macro_interp_cache

waneck commented 9 years ago

Will do that. Thanks!

ncannasse commented 9 years ago

Measuring the size of a value by excluding some others in order to know its "own" size is a bit more tricky (see Main.display_memory code)

waneck commented 9 years ago

Okay, that'll be a nice addition anyway

Simn commented 9 years ago

Is this an issue or not?

waneck commented 9 years ago

This is an issue with our unit tests; Adding how much memory each module occupies when running --display memory would be a great way to debug that; so I think we should still add this. This isn't however a 3.2 priority in any way

Simn commented 7 years ago

I think this has been fixed. Feel free to reopen if not!