Snap runs slower than source build

[dmusican]

Running on Ubuntu 16.04... (yes, it's still not EOL!)

I'm running spacemacs, which is something of a cpu hog. Anyway, running 27.1 via snap is notably slower and more cpu intensive than when I built 27.1 from source and ran on the exact same .emacs.d files.

I realize that's not a lot to go on regarding diagnosing the problem. What information could I provide that would be useful? Emacs profiling data was not helpful; it seemed to be about the same results with either binary. I did notice in top that the snap version was definitely using more CPU than the version I built from source, and I could instantly hear the difference with my laptop fan.

[alexmurray]

Can you detail a bit more about how you build emacs 27.1 yourself? Do you use any particular CFLAGS etc / configure options? For the snap, we build it roughly as follows and so there is no specific optimisation settings specified for gcc:

./configure --with-xwidgets --with-x-toolkit=gtk3 --without-xaw3d --with-modules --with-cairo
make
sudo make install

Also would you be able to have a look at running some benchmarks comparing the two? https://gitlab.com/koral/elisp-benchmarks would be a good start I think.

[alexmurray]

I got the benchmarks running myself and could confirm that the snap build was definitely slower - I noticed that if I built emacs-27.1 from source myself in Ubuntu 20.04 the standard CFLAGS would contain -O2 - whereas they do not with the snap. So this was then added in a2fc5f02361a34ff7322032ce027c8bc4cc3b25a and with this enabled we see the same benchmark performance (see the commit message itself for more details). This should be built soon and available on the beta channel within an hour or two if you want to test yourself.

[alexmurray]

Also I should say - thanks for reporting this issue - I hadn't noticed it myself but I am very glad for the improved performance as a result.

[dmusican]

Thanks so much for the response, and for the rebuilld. I can confirm that the new beta build is dramatically faster, and makes the snap wonderful and usable. I also tried running elisp-benchmarks, after your first comment; the previous snap build (without the O2 optimizations) wouldn't even finish the benchmarks for me without a C overflow error, whereas they ran fine on my source build. The new snap build runs great.

It remains the case that the source build I made is still very slightly faster than the new beta snap build. The difference is nearly within noise, but I believe it to be real. Here are the results of elisp-benchmark, installed from MELPA, on both a completely vanilla start (apart from installing the benchmark package) on my build and on the snap build.

On my build:

* Results

  | test           | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) |
  |----------------+----------------+------------+---------+-------------+-----------------|
  | bubble-no-cons |           9.25 |       0.04 |       2 |        9.29 |            0.03 |
  | bubble         |           4.47 |       7.02 |     353 |       11.49 |            0.04 |
  | dhrystone      |           7.80 |       0.00 |       0 |        7.80 |            0.09 |
  | fibn-rec       |           6.27 |       0.00 |       0 |        6.27 |            0.08 |
  | fibn-tc        |           5.65 |       0.00 |       0 |        5.65 |            0.08 |
  | fibn           |          10.22 |       0.00 |       0 |       10.22 |            0.05 |
  | flet           |          10.58 |       0.00 |       0 |       10.58 |            0.05 |
  | inclist        |          14.03 |       0.00 |       0 |       14.03 |            0.08 |
  | listlen-tc     |           6.31 |       0.00 |       0 |        6.31 |            0.02 |
  | map-closure    |           8.41 |       0.00 |       0 |        8.41 |            0.04 |
  | nbody          |           3.67 |      11.53 |     569 |       15.21 |            0.01 |
  | pcase          |          10.74 |       0.00 |       0 |       10.74 |            0.03 |
  | pidigits       |           6.65 |       6.76 |     309 |       13.41 |            0.05 |
  |----------------+----------------+------------+---------+-------------+-----------------|
  | total          |         104.05 |      25.35 |    1233 |      129.40 |            0.20 |

On the new beta snap build, with the O2 optimizations:

* Results

  | test           | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) |
  |----------------+----------------+------------+---------+-------------+-----------------|
  | bubble-no-cons |           9.91 |       0.04 |       2 |        9.95 |            0.16 |
  | bubble         |           4.88 |       7.06 |     333 |       11.93 |            0.07 |
  | dhrystone      |           8.22 |       0.00 |       0 |        8.22 |            0.11 |
  | fibn-rec       |           6.70 |       0.00 |       0 |        6.70 |            0.12 |
  | fibn-tc        |           5.93 |       0.00 |       0 |        5.93 |            0.03 |
  | fibn           |          11.46 |       0.00 |       0 |       11.46 |            0.15 |
  | flet           |          11.14 |       0.00 |       0 |       11.14 |            0.03 |
  | inclist        |          14.90 |       0.00 |       0 |       14.90 |            0.04 |
  | listlen-tc     |           7.31 |       0.00 |       0 |        7.31 |            0.01 |
  | map-closure    |           8.48 |       0.00 |       0 |        8.48 |            0.02 |
  | nbody          |           3.79 |      12.33 |     569 |       16.12 |            0.02 |
  | pcase          |          11.05 |       0.00 |       0 |       11.05 |            0.08 |
  | pidigits       |           5.89 |       7.22 |     309 |       13.10 |            0.03 |
  |----------------+----------------+------------+---------+-------------+-----------------|
  | total          |         109.66 |      26.65 |    1213 |      136.31 |            0.30 |

I ran the benchmarks three times through (default), without my laptop doing anything else. I've also done that whole experiment a few times, and the results come out consistently that the snap build is just very slightly slower. I can also time how long it takes for my personal init script to complete when I start everything up in full, and it does take the snap build perhaps a second or so longer.

So what's different with my build? I'm not sure. I just ran ./configure without any options. It picked up a gtk2 library instead of 3. Apart from that, I'm unaware of any meaningful differences. Is there something else I can look at?

[alexmurray]

Interestingly I get slightly better results from the snap build than a local build:

Local build:

* Results

  | test           | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) |
  |----------------+----------------+------------+---------+-------------+-----------------|
  | bubble-no-cons |          11.42 |       0.04 |       2 |       11.46 |            0.93 |
  | bubble         |           4.75 |       6.13 |     301 |       10.88 |            0.74 |
  | dhrystone      |           9.71 |       0.00 |       0 |        9.71 |            0.80 |
  | fibn-rec       |           7.49 |       0.00 |       0 |        7.49 |            0.63 |
  | fibn-tc        |           6.62 |       0.00 |       0 |        6.62 |            0.39 |
  | fibn           |          11.89 |       0.00 |       0 |       11.89 |            0.68 |
  | flet           |          12.42 |       0.00 |       0 |       12.42 |            0.70 |
  | inclist        |          15.94 |       0.00 |       0 |       15.94 |            0.36 |
  | listlen-tc     |          12.03 |       0.00 |       0 |       12.03 |            1.70 |
  | map-closure    |          10.06 |       0.00 |       0 |       10.06 |            0.65 |
  | nbody          |           4.48 |      12.37 |     518 |       16.85 |            3.38 |
  | pcase          |          12.94 |       0.00 |       0 |       12.94 |            1.28 |
  | pidigits       |           5.66 |       7.24 |     282 |       12.90 |            0.24 |
  |----------------+----------------+------------+---------+-------------+-----------------|
  | total          |         125.39 |      25.78 |    1103 |      151.17 |            4.48 |

Snap:

* Results

  | test           | non-gc avg (s) | gc avg (s) | gcs avg | tot avg (s) | tot avg err (s) |
  |----------------+----------------+------------+---------+-------------+-----------------|
  | bubble-no-cons |          10.60 |       0.04 |       2 |       10.64 |            0.43 |
  | bubble         |           4.55 |       6.83 |     319 |       11.38 |            0.14 |
  | dhrystone      |           9.31 |       0.00 |       0 |        9.31 |            0.35 |
  | fibn-rec       |           6.94 |       0.00 |       0 |        6.94 |            0.20 |
  | fibn-tc        |           6.29 |       0.00 |       0 |        6.29 |            0.10 |
  | fibn           |          12.01 |       0.00 |       0 |       12.01 |            0.34 |
  | flet           |          12.33 |       0.00 |       0 |       12.33 |            0.77 |
  | inclist        |          16.50 |       0.00 |       0 |       16.50 |            0.73 |
  | listlen-tc     |           7.06 |       0.00 |       0 |        7.06 |            0.95 |
  | map-closure    |           9.33 |       0.00 |       0 |        9.33 |            0.27 |
  | nbody          |           3.57 |      11.90 |     521 |       15.46 |            1.22 |
  | pcase          |          12.44 |       0.00 |       0 |       12.44 |            0.41 |
  | pidigits       |           5.83 |       7.28 |     283 |       13.11 |            0.67 |
  |----------------+----------------+------------+---------+-------------+-----------------|
  | total          |          56.64 |      86.16 |    1125 |      142.80 |            2.16 |

In both these cases I ran emacs in a terminal but with the same configure options - if you feel this is still an issue could you please try compiling emacs with the same configure flags as I posted earlier and try and reproduce? However I feel like the difference is quite minimal now either way so I am not sure there is much to be gained by further investigations myself.

[dmusican]

Wow, that's quite a difference, and bigger than the one I saw. I tried rebuilding with the same options you did, but I couldn't get it to work; I was getting gtk errors due to API differences between what was in the emacs code and what the gtk3 version supported by Ubuntu 16.04 was offering. I'm sure I could chase this down if I tried hard enough, but it's not worth it.

Anyway, thanks again so much for making this snap available, and for your efforts in making it faster -- which made a very big difference!

[alexmurray]

No worries - thanks again for reporting this in the first place - it has made a noticeable difference for my own use too so I appreciate you bringing it to my attention so it could be fixed.