The calendar widget is slow

Looking through the output of

diff --git a/tests/examples/CMakeLists.txt b/tests/examples/CMakeLists.txt
index f64f60c..7c4e004 100644
--- a/tests/examples/CMakeLists.txt
+++ b/tests/examples/CMakeLists.txt
@@ -36,7 +36,7 @@ if (DEFINED ENV{DO_COVERAGE} AND NOT $ENV{DO_COVERAGE} STREQUAL "0")
   endif()
   set(LUA_COV_RUNNER lua "-erequire('luacov.runner')('${TOP_SOURCE_DIR}/.luacov')")
 else()
-  set(LUA_COV_RUNNER lua)
+  set(LUA_COV_RUNNER time lua)
 endif()

 # Add the main awesome lua libraries.

with vim's search feature, there are two tests that need at least 0.1 seconds: tests/examples/wibox/widget/calendar/fn_embed_cell.lua needs 0.1 seconds (just at the limit; that's a bit slow, but not too slow) tests/examples/wibox/widget/calendar/year.lua needs 0.45 seconds. This seems to be worryingly much.

Next hack to get some measurements:

diff --git a/tests/examples/wibox/template.lua b/tests/examples/wibox/template.lua
index 656d22e..4efcc7e 100644
--- a/tests/examples/wibox/template.lua
+++ b/tests/examples/wibox/template.lua
@@ -11,20 +11,27 @@ local container = wibox.layout.fixed.vertical()
 local w, h, image_type = loadfile(file_path)(container)
 image_type = image_type or "svg"

+print(timer_measure:elapsed(), "template.lua: After loadfile()")
+
 -- Emulate the event loop for 10 iterations
 for _ = 1, 10 do
     awesome:emit_signal("refresh")
+print(timer_measure:elapsed(), "template.lua: After refresh", _)
 end

 -- Get the example fallback size (the tests can return a size if the want)
 local f_w, f_h = container:fit({dpi=96}, 9999, 9999)

+print(timer_measure:elapsed(), "template.lua: fit")
+
 -- There is an overhead that cause testbox "...", add 10 until someone
 -- figures out the real equation
 f_w, f_h = f_w+10, f_h+10

 -- Save to the output file
 local img = surface["widget_to_"..image_type](container, image_path.."."..image_type, w or f_w, h or f_h)
+print(timer_measure:elapsed(), "template.lua: write image")
 img:finish()
+print(timer_measure:elapsed(), "template.lua: finish image")

 -- vim: filetype=lua:expandtab:shiftwidth=4:tabstop=8:softtabstop=4:textwidth=80
diff --git a/tests/examples/wibox/widget/calendar/year.lua b/tests/examples/wibox/widget/calendar/year.lua
index 4274a0e..23bb5ec 100644
--- a/tests/examples/wibox/widget/calendar/year.lua
+++ b/tests/examples/wibox/widget/calendar/year.lua
@@ -11,6 +11,11 @@ beautiful.get_font = function() return f end --DOC_HIDE
 beautiful.fg_focus = "#ff9800" --DOC_HIDE
 beautiful.bg_focus = "#b9214f" --DOC_HIDE

+local GLib = require("lgi").GLib
+timer_measure = GLib.Timer()
+timer_measure:start()
+print(timer_measure:elapsed(), "after nothing")
+
     local cal = wibox.widget {
         date         = os.date("*t"),
         font         = "Monospace 8",
@@ -20,6 +25,9 @@ beautiful.bg_focus = "#b9214f" --DOC_HIDE
         widget       = wibox.widget.calendar.year
     }

+print(timer_measure:elapsed(), "after creation")
+
 parent:add(cal) --DOC_HIDE
 local w,h = parent:fit({dpi=96}, 9999, 9999) --DOC_HIDE
+print(timer_measure:elapsed(), "after template done (fit called)")
 return w+30, h --DOC_HIDE

Output (after I figured out how to call these by hand instead of via CMake...):

3e-06   after nothing
0.250565    after creation
0.295118    after template done (fit called)
0.295151    template.lua: After loadfile()
0.295157    template.lua: After refresh 1
0.295167    template.lua: After refresh 2
0.295184    template.lua: After refresh 3
0.295192    template.lua: After refresh 4
0.29521 template.lua: After refresh 5
0.29522 template.lua: After refresh 6
0.295228    template.lua: After refresh 7
0.295246    template.lua: After refresh 8
0.295252    template.lua: After refresh 9
0.295268    template.lua: After refresh 10
0.31698 template.lua: fit
0.479349    template.lua: write image
0.482942    template.lua: finish image

So, creating the year-widget takes a quarter of a second (0.25 seconds). Adding it to to a fixed layout and calling fit on the layout takes 0.04 seconds. Calling fit on the fixed layout again (this time in template.lua) takes 0.02 seconds (might be the widget geometry cache, I guess). Drawing the widget to an image surface takes 0.16 seconds. Edit: Well, this also involves another round of :fit(),

So of the ~0.5 seconds that this test runs, half (3/6th of the time) is needed to construct the widget and 2/6 for drawing it. This seems improvable.

Before anyone tries to do something about this: The first step is to add a suitable test to test-benchmark.lua:

diff --git a/tests/test-benchmark.lua b/tests/test-benchmark.lua
index def3c01..aa9ca45 100644
--- a/tests/test-benchmark.lua
+++ b/tests/test-benchmark.lua
@@ -77,6 +77,19 @@ benchmark(relayout_textclock, "relayout textclock")
 benchmark(redraw_textclock, "redraw textclock")
 benchmark(e2e_tag_switch, "tag switch")

+local cal
+local wibox = require("wibox")
+benchmark(function()
+    cal = wibox.widget {
+        date         = os.date("*t"),
+        font         = "Monospace 8",
+        spacing      = 2,
+        week_numbers = false,
+        start_sunday = false,
+        widget       = wibox.widget.calendar.year
+    }
+end, "create calendar")
+
 runner.run_steps({ function() return true end })

 -- vim: filetype=lua:expandtab:shiftwidth=4:tabstop=8:softtabstop=4:textwidth=80

Output:

== Running test-benchmark.lua ==
   create&draw wibox: 0.0149436  sec/iter ( 69 iters, 1.61 sec for benchmark)
    update textclock: 2.69408e-06 sec/iter (428568 iters, 1.334 sec for benchmark)
  relayout textclock: 1.91042e-06 sec/iter (749180 iters, 1.654 sec for benchmark)
    redraw textclock: 9.19104e-07 sec/iter (1238224 iters, 3.213 sec for benchmark)
          tag switch: 0.00740952 sec/iter (137 iters, 1.55 sec for benchmark)
     create calendar: 0.200973   sec/iter (  6 iters, 2.312 sec for benchmark)
2017-06-22 14:44:45 W: awesome: a_glib_poll:432: Last main loop iteration took 11.675297 seconds! Increasing limit for this warning to that value.
Test finished successfully

=> Just creating a calendar widget is 13 times slower than creating and drawing a full wibox that more-or-less looks like the wibox in the default config.

diff --git a/tests/test-benchmark.lua b/tests/test-benchmark.lua
index def3c01..a146e44 100644
--- a/tests/test-benchmark.lua
+++ b/tests/test-benchmark.lua
@@ -77,6 +77,27 @@ benchmark(relayout_textclock, "relayout textclock")
 benchmark(redraw_textclock, "redraw textclock")
 benchmark(e2e_tag_switch, "tag switch")

+local wibox = require("wibox")
+local cal = wibox.widget {
+        date         = os.date("*t"),
+        font         = "Monospace 8",
+        spacing      = 2,
+        week_numbers = false,
+        start_sunday = false,
+        widget       = wibox.widget.calendar.year
+    }
+
+    local wb = wibox({ width = 1024, height = 768, screen = 1 })
+    wb.visible = true
+    wb:set_widget(cal)
+
+benchmark(function()
+    cal:emit_signal("widget::updated")
+    do_pending_repaint()
+end, "update calendar")
+
+    wb.visible = false
+
 runner.run_steps({ function() return true end })

 -- vim: filetype=lua:expandtab:shiftwidth=4:tabstop=8:softtabstop=4:textwidth=80

Output:

== Running test-benchmark.lua ==
   create&draw wibox: 0.0155542  sec/iter ( 67 iters, 1.657 sec for benchmark)
    update textclock: 2.64923e-06 sec/iter (391993 iters, 1.251 sec for benchmark)
  relayout textclock: 1.96265e-06 sec/iter (810705 iters, 1.838 sec for benchmark)
    redraw textclock: 9.01986e-07 sec/iter (1189926 iters, 3.147 sec for benchmark)
          tag switch: 0.00748424 sec/iter (138 iters, 1.584 sec for benchmark)
     update calendar: 0.0479693  sec/iter ( 21 iters, 2.367 sec for benchmark)
2017-06-22 14:52:50 W: awesome: a_glib_poll:432: Last main loop iteration took 11.969537 seconds! Increasing limit for this warning to that value.
Test finished successfully

=> Drawing a calendar widget seems to be 3 times slower than creating and drawing a full wibox.

Adding to v4.2 milestone since the calendar is new and was not part of 4.1.

Actually, it seems like the grid layout is slow:

diff --git a/tests/examples/wibox/template.lua b/tests/examples/wibox/template.lua
index 656d22e..3275796 100644
--- a/tests/examples/wibox/template.lua
+++ b/tests/examples/wibox/template.lua
@@ -27,4 +27,9 @@ f_w, f_h = f_w+10, f_h+10
 local img = surface["widget_to_"..image_type](container, image_path.."."..image_type, w or f_w, h or f_h)
 img:finish()

+if ProFi then
+               ProFi:stop()
+               ProFi:writeReport( '/tmp/MyProfilingReport.txt' )
+            end
+
 -- vim: filetype=lua:expandtab:shiftwidth=4:tabstop=8:softtabstop=4:textwidth=80
diff --git a/tests/examples/wibox/widget/calendar/year.lua b/tests/examples/wibox/widget/calendar/year.lua
index 4274a0e..bf1e856 100644
--- a/tests/examples/wibox/widget/calendar/year.lua
+++ b/tests/examples/wibox/widget/calendar/year.lua
@@ -1,3 +1,7 @@
+
+               ProFi = require 'ProFi'
+               ProFi:start()
+
 local parent    = ... --DOC_HIDE
 local wibox     = require("wibox") --DOC_HIDE
 local beautiful = require( "beautiful" ) --DOC_HIDE

Result (everything above 1% RELATIVE):

###############################################################################################################
#####  ProFi, a lua profiler. This profile was generated on: Thu Jun 22 15:01:48 2017
#####  ProFi is created by Luke Perkin 2012 under the MIT Licence, www.locofilm.co.uk
#####  Version 1.3. Get the most recent version at this gist: https://gist.github.com/2838755
###############################################################################################################

| TOTAL TIME = 25.228504
| FILE                                              : FUNCTION                                : LINE                : TIME        : RELATIVE    : CALLED      |
| /home/psychon/awesome/lib/gears/surface.lua       : ?                                       :  228                : 12.011      : 47.61%      :       1     |
| /home/psychon/awesome/lib/gears/surface.lua       : run_in_hierarchy                        :  211                : 12.009      : 47.60%      :       1     |
| [C]                                               : xpcall                                  :   -1                : 11.086      : 43.94%      :    3325     |
| /home/psychon/awesome/lib/wibox/layout/grid.lua   : get_grid_sizes                          :  700                : 10.956      : 43.43%      :      77     |
| /home/psychon/awesome/lib/wibox/widget/base.lua   : fit_widget                              :  280                : 10.446      : 41.41%      :    2837     |
| /home/psychon/awesome/lib/gears/cache.lua         : get                                     :   17                : 9.713       : 38.50%      :    2843     |
| /home/psychon/awesome/lib/wibox/widget/calendar.lu: fill_container                          :  268                : 9.478       : 37.57%      :       4     |
| /home/psychon/awesome/lib/wibox/layout/grid.lua   : anonymous                               :  733                : 9.193       : 36.44%      :      64     |
| /home/psychon/awesome/lib/wibox/hierarchy.lua     : new                                     :  206                : 9.037       : 35.82%      :       1     |
| /home/psychon/awesome/lib/wibox/hierarchy.lua     : update                                  :  220                : 9.036       : 35.82%      :       1     |
| [C]                                               : __index                                 :   -1                : 8.817       : 34.95%      :   67291     |
| /home/psychon/awesome/lib/wibox/widget/base.lua   : drill                                   :  451                : 8.716       : 34.55%      :     953     |
| /usr/share/lua/5.3/lgi/component.lua              : __index                                 :  258                : 8.592       : 34.06%      :   84684     |
| /usr/share/lua/5.3/lgi/component.lua              : _element                                :  189                : 7.383       : 29.26%      :  236518     |
| /usr/share/lua/5.3/lgi/component.lua              : anonymous                               :  258                : 7.243       : 28.71%      :   58179     |
| /usr/share/lua/5.3/lgi/class.lua                  : _element                                :  132                : 7.123       : 28.23%      :   56369     |
| [C]                                               : __newindex                              :   -1                : 6.021       : 23.87%      :   12144     |
| /home/psychon/awesome/lib/wibox/layout/fixed.lua  : anonymous                               :  217                : 5.372       : 21.29%      :       3     |
| /home/psychon/awesome/lib/wibox/widget/calendar.lu: ?                                       :  298                : 4.892       : 19.39%      :       4     |
| /usr/share/lua/5.3/lgi/record.lua                 : _element                                :   49                : 4.770       : 18.91%      :   94662     |
| /home/psychon/awesome/lib/wibox/widget/calendar.lu: ?                                       :  285                : 4.587       : 18.18%      :       1     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: setup_layout                            :   34                : 4.194       : 16.62%      :    3232     |
| /usr/share/lua/5.3/lgi/override/GObject-Object.lua: _element                                :  229                : 4.014       : 15.91%      :   81773     |
| /usr/share/lua/5.3/lgi/component.lua              : anonymous                               :  156                : 3.780       : 14.98%      :   44696     |
| /home/psychon/awesome/lib/wibox/layout/fixed.lua  : fit                                     :  217                : 3.731       : 14.79%      :       2     |
| [C]                                               : rawget                                  :   -1                : 3.566       : 14.14%      : 1088078     |
| /usr/share/lua/5.3/lgi/class.lua                  : inherited_element                       :  132                : 3.450       : 13.67%      :   81773     |
| /home/psychon/awesome/lib/wibox/layout/grid.lua   : anonymous                               :  766                : 3.323       : 13.17%      :      13     |
| /home/psychon/awesome/lib/wibox/layout/fixed.lua  : anonymous                               :   23                : 3.281       : 13.00%      :       2     |
| /home/psychon/awesome/lib/wibox/hierarchy.lua     : draw                                    :  304                : 3.019       : 11.97%      :     479     |
| /home/psychon/awesome/lib/wibox/hierarchy.lua     : hierarchy_update                        :   85                : 2.419       : 9.59%       :     479     |
| /home/psychon/awesome/lib/wibox/hierarchy.lua     : call                                    :  320                : 1.473       : 5.84%       :    2393     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: anonymous                               :   41                : 1.447       : 5.74%       :     462     |
| [C]                                               : type                                    :   -1                : 1.285       : 5.09%       :  399698     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: ?                                       :  153                : 1.253       : 4.97%       :     924     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: set_markup_silently                     :  126                : 1.245       : 4.94%       :     924     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: set_ellipsize                           :  188                : 1.125       : 4.46%       :     924     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: set_wrap                                :  204                : 1.071       : 4.25%       :     924     |
| /home/psychon/awesome/lib/gears/object.lua        : emit_signal                             :  115                : 1.070       : 4.24%       :   15011     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: ?                                       :  236                : 1.045       : 4.14%       :     924     |
| /home/psychon/awesome/lib/wibox/widget/base.lua   : make_widget                             :  599                : 0.903       : 3.58%       :     954     |
| [C]                                               : for iterator                            :   -1                : 0.892       : 3.53%       :  285952     |
| [C]                                               : get_pixel_extents                       :   -1                : 0.764       : 3.03%       :    3232     |
| [C]                                               : match                                   :   -1                : 0.619       : 2.45%       :  174758     |
| /home/psychon/awesome/lib/wibox/widget/base.lua   : check_widget                            :  686                : 0.602       : 2.39%       :    6643     |
| [C]                                               : sub                                     :   -1                : 0.589       : 2.34%       :  174930     |
| /home/psychon/awesome/lib/gears/object.lua        : find_signal                             :   31                : 0.578       : 2.29%       :   20266     |
| /home/psychon/awesome/lib/wibox/widget/base.lua   : record_dependency                       :  232                : 0.565       : 2.24%       :    3322     |
| [C]                                               : getmetatable                            :   -1                : 0.484       : 1.92%       :  144772     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: ?                                       :  252                : 0.463       : 1.84%       :     924     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: set_font                                :  252                : 0.461       : 1.83%       :     924     |
| [C]                                               : pairs                                   :   -1                : 0.407       : 1.61%       :  121763     |
| /home/psychon/awesome/lib/wibox/layout/grid.lua   : add_widget_at                           :  307                : 0.385       : 1.53%       :     946     |
| /home/psychon/awesome/lib/wibox/layout/grid.lua   : find_widgets_at                         :  189                : 0.381       : 1.51%       :    2562     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: setup_dpi                               :   25                : 0.375       : 1.49%       :    3232     |
| /home/psychon/awesome/lib/wibox/widget/textbox.lua: set_align                               :  236                : 0.368       : 1.46%       :     924     |
| /home/psychon/awesome/lib/gears/object.lua        : check                                   :   21                : 0.309       : 1.22%       :   20266     |
| /home/psychon/awesome/lib/gears/object.lua        : connect_signal                          :   52                : 0.297       : 1.18%       :    3818     |
| [C]                                               : rawset                                  :   -1                : 0.287       : 1.14%       :   35134     |
| /home/psychon/awesome/lib/wibox/layout/grid.lua   : get_next_empty                          :  254                : 0.266       : 1.06%       :     754     |

I don't think this should block v4.2. It's not a bug, just a slow reality. The calendar creates up to 1k widgets, of course it's going to be slow.

This is https://github.com/pavouk/lgi/issues/128 all over again.

In Awesome 3.5, my "shorty" module (my hotkey popup implementation) was drawing tables like this and had the same bug. I ended up hard coding the layouts to avoid all the overhead. Eventually I simplified it using some fit() hacks to avoid the exponential time it took to add new elements. v4.0 fixed that using the delayed draw. But the root of the problem remains. Doing big grid with a lot of nested widgets has both a lot of Lua code being executed and a lot of LGI lookup. Beside hardcoding the layouts or adding monkeypatched hacks, there is only so much we can do. My vote would be on "make LGI faster" and claim this to be a non-issue.

and our current hotkeys popup is concatenating strings instead of creating a real layout of widgets

awesomeWM / awesome

The calendar widget is slow #1861