teal-language / tl

The compiler for Teal, a typed dialect of Lua
MIT License
2.1k stars 108 forks source link

reduce collectgarbage() time cost in tl build #579

Closed virusdefender closed 1 year ago

virusdefender commented 1 year ago

My project contains about 100+ tl files, the code added shows collectgarbage in tl build will cost about 70% cpu time.

before gc memory    loop 2  47309.854492188
after  gc memory    loop 2  19262.390625
......

before gc memory    loop 30 19282.856445312
after  gc memory    loop 30 19226.875976562
......

before gc memory    loop 100    28070.041992188
after  gc memory    loop 100    28028.818359375
......

before gc memory    loop 128    35244.404296875
after  gc memory    loop 128    35152.580078125

build total time    2626.44
gc total time   1870.631
diff --git a/tl b/tl
index 1dc9a8e..ba0123f 100755
--- a/tl
+++ b/tl
@@ -430,6 +430,8 @@ do
    end

    function build.run(tlconfig)
+      local build_start = os.clock()
+      local gc_time = 0
       local function remove_leading_path(leading_part, path)
          local s, e = path:find("^" .. leading_part .. PATH_SEPARATOR .. "?")
          if s then
@@ -718,11 +720,17 @@ do
          end

          if i > 1 then
+            local a = os.clock()
+            print("before gc memory", "loop " .. tostring(i), collectgarbage("count"))
             collectgarbage()
+            print("after  gc memory", "loop " .. tostring(i), collectgarbage("count"))
+            gc_time = gc_time + (os.clock() - a)
          end
       end

       local ok = report_all_errors(tlconfig, env)
+      print("build total time", (os.clock() - build_start) * 1000)
+      print("gc total time", gc_time * 1000)

       os.exit(ok and 0 or 1)
    end

if if i > 1 then is replaced by if i % 50 == 0 (gc once per 50 files), the time cost is reduced significantly and the memory usage is acceptable.

before gc memory    loop 50 50841.455078125
after  gc memory    loop 50 19465.637695312

before gc memory    loop 100    46594.625976562
after  gc memory    loop 100    28092.53125

build total time    1107.268
gc total time   96.925
hishamhm commented 1 year ago

tl build is currently deprecated, and I intend to remove this command from the core tl compiler it in a future release; I recommend using cyan instead as a build tool — does the same issue happen if building using cyan build?