Compilation still slow

[Guevara-chan]

No, seriously. Even with TCC and --d:release for testing builds, it's still about 9-10s for 60 (!) lines project.

I found myself waiting more time for project to compile than actually coding stuff.

[khchen]

Here is my test, I think it's acceptable for now. Related discussion: https://forum.nim-lang.org/t/4975

OS: Windows 10 1803
CPU: Intel Core i7-8700 3.20GHz
Ram: 16G
Disk: SSD
Compiler: Nim Compiler Version 0.20.0 [Windows: amd64]
Source: examples/demo.nim, lines of code: around 200
[1] means after deleting the nimcache folder. 

nim c demo
[1] Hint: operation successful (101394 lines compiled; 5.070 sec total; 171.281MiB peakmem; Debug Build)
[2] Hint: operation successful (101394 lines compiled; 3.144 sec total; 171.281MiB peakmem; Debug Build)

nim c -d:release demo
[1] Hint: operation successful (101394 lines compiled; 8.442 sec total; 137.238MiB peakmem; Release Build)
[2] Hint: operation successful (101394 lines compiled; 2.231 sec total; 137.281MiB peakmem; Release Build)

nim c --cc:tcc demo
[1] Hint: operation successful (101394 lines compiled; 3.285 sec total; 171.18MiB peakmem; Debug Build)
[2] Hint: operation successful (101394 lines compiled; 2.982 sec total; 171.18MiB peakmem; Debug Build)

nim c --cc:tcc -d:release demo
[1] Hint: operation successful (101394 lines compiled; 2.325 sec total; 137.27MiB peakmem; Release Build)
[2] Hint: operation successful (101394 lines compiled; 2.021 sec total; 137.27MiB peakmem; Release Build)

[ba0f3]

@khchen try to add -f switch to see actual compiling speed w/o cache @Guevara-chan did you reuse cached objects?

[Guevara-chan]

did you reuse cached objects?

Reusing seems to never work with wNim, since compilation times is same no matter what changes had been done.

[ba0f3]

>  nim c -f -d:release -d:crosswin examples/demo.nim
Hint: operation successful (115642 lines compiled; 12.438 sec total; 139.723MiB peakmem; Release Build) [SuccessX]
>  nim c -d:release -d:crosswin examples/demo.nim
Hint: operation successful (115642 lines compiled; 2.349 sec total; 139.727MiB peakmem; Release Build) [SuccessX]

much faster for me, cross compiled on debian based linux.

[Guevara-chan]

>nim c --cc:tcc -d:release IPGrab.nim
Hint: operation successful (104908 lines compiled; 9.287 sec total; 137.641MiB peakmem; Release Build) [SuccessX]
>nim c --cc:tcc -d:release -f IPGrab.nim
Hint: operation successful (104908 lines compiled; 9.382 sec total; 137.266MiB peakmem; Release Build) [SuccessX]

[ba0f3]

check your config files, *nim.cfg, config.nims, etc.. I think you accidentally force build your project

[Guevara-chan]

My nim.cfg is just that:

--d:release
--cc:tcc

...Also I have no such problems with my other projects.

[ba0f3]

See the compilation log, it tells you what config files were loaded

[khchen]

>nim c --cc:tcc -d:release IPGrab.nim
Hint: operation successful (104908 lines compiled; 9.287 sec total; 137.641MiB peakmem; Release Build) [SuccessX]
>nim c --cc:tcc -d:release -f IPGrab.nim
Hint: operation successful (104908 lines compiled; 9.382 sec total; 137.266MiB peakmem; Release Build) [SuccessX]

How about just don't add -f in your second time compiling?

[Guevara-chan]

>nim c --cc:tcc -d:release IPGrab.nim
Hint: used config file 'C:\Utils\Nim\nim.[latest]\config\nim.cfg' [Conf]
Hint: used config file 'C:\Utils\Nim\Projects\[.Junkyard]\nim.cfg' [Conf]
Hint: operation successful (104908 lines compiled; 10.101 sec total; 137.645MiB peakmem; Release Build) [SuccessX]
>nim c --cc:tcc -d:release IPGrab.nim
Hint: used config file 'C:\Utils\Nim\nim.[latest]\config\nim.cfg' [Conf]
Hint: used config file 'C:\Utils\Nim\Projects\[.Junkyard]\nim.cfg' [Conf]
Hint: operation successful (104908 lines compiled; 9.224 sec total; 137.391MiB peakmem; Release Build) [SuccessX]
>type nim.cfg
--d:release
--cc:tcc

[sk-Prime]

the compile time is somewhat reasonable because it compiles at-least 100k lines of code. for comparison NiGui (as i found it fastest among all in term of compile time) compiles around 50k lines of code and take 2 to 3 sec average on the other hand wnim takes 7 to 13 sec. but NiGui has much less feature than wnim, thus less line of code. i heard that Nigui efficiently takes advantage of dead code elimination , wnim should focus on this too so there will be less line of code to compile.

[khchen]

i heard that Nigui efficiently takes advantage of dead code elimination , wnim should focus on this too so there will be less line of code to compile.

It already did. The first line of wNim.nim is {.experimental, deadCodeElim: on.}.

[sk-Prime]

When i look at the source code of wnim.nim, i found that it include more than 100 files. it is obviously the biggest reason of slow compilation. i wrote a basic program

import wnim
let app = App()
let frame = Frame(title="Testing", size=(400, 160))
let panel = Panel(frame)
let label = StaticText(panel, label= "Testing compilation")
frame.show()
app.mainLoop()

this code takes 19 sec at first build -f --d:release and second build

Hint: operation successful (109777 lines compiled; 8.151 sec total; 171.336MiB peakmem; Release Build) [SuccessX]

and then i remove some of the included files from wnim.nim, previously total included file was 102 and now it became 63 files. this time first build with -f -d:release takes 14 second. and second build

Hint: operation successful (98068 lines compiled; 4.705 sec total; 137.918MiB peakmem; Release Build) [SuccessX]

notice the number of lines needed to be compiled is also less than before

The number of file in include can be further reducible, but the problem is lot of interconnection among files prevents it. for example i have to keep wNim/private/controls/wButton in includes, though i didn't used it. if i remove it this error occurs

E:\Nim\NIM_OFFLINE_RES\wNim\wNim\private\controls\wControl.nim(46, 6) Error: implementation of 'wNim.click(self: wButton) [declared in E:\Nim\NIM_OFFLINE_RES\wNim\wNim\private\controls\wControl.nim(46, 6)]' expected

including this huge amount of files also causes nim compiler to invest more time on dead code elimination and other processing. because include just take all the content of that file and place that where include occurs.

list of included 63 file :paste

[khchen]

In fact, you are definitely right. I know about this problem.

In the early stage of wNim, I had tried to use import instead of include. But soon I found it need a lot of trivial modification and the result is not so good. In that time, wNim don't have so lot of code. So even I did all the change in right way (and maybe induce new bugs), the compilation time was almost the same. So I chosen to spend time on functions, instead of other trivial detail.

Nowadays, wNim becomes bloated. I am always considering refactoring the code, so I keep this issue open. However (or unfortunately?), I am still spending time on functions in these days.

[sk-Prime]

i am new to nim and have no idea about winapi. so i can't help you that much. but i have some suggestion

1. there will be a base module, every other widgets and function will depend on it. it will be included
2. and then other widgets only use that base module
3. wnim.nim will contain alias of all functionality and import call can be ocur. wnim.radiobutton.init proc will include radiobutton otherwise not. or
4. user will import the widgets user want to use, like import radiobutton.

1 and 2 will reduce inter-module dependencies 3 and 4 will reduce number of unnecessary widget being included in main.

[khchen]

I already find some way to overcome the recursive module dependencies problem. The whole wNim package will be split into sub-modules in next release.

The import rule is simple:

1. You can still use import wNim to import all the package like before.
2. You can import sub-modules one by one: import wNim/[wApp, wFrame, wMenu].

Here is some of my test:

examples/helloworld.nim
  wNim 0.8.0
  --cc:gcc: 109795 lines compiled; 5.337 sec total
  --cc:tcc: 109795 lines compiled; 4.696 sec total

  wNim 0.9.0 (all in one)
  --cc:gcc: 110434 lines compiled; 5.347 sec total
  --cc:tcc: 110434 lines compiled; 4.094 sec total

  wNim 0.9.0 (one by one)
  --cc:gcc: 90666 lines compiled; 2.619 sec total
  --cc:tcc: 90666 lines compiled; 1.998 sec total

examples/demo.nim
  wNim 0.8.0
  --cc:gcc: 114579 lines compiled; 6.934 sec total
  --cc:tcc: 114579 lines compiled; 4.760 sec total

  wNim 0.9.0 (all in one)
  --cc:gcc: 115218 lines compiled; 6.370 sec total
  --cc:tcc: 115218 lines compiled; 4.567 sec total

  wNim 0.9.0 (one by one)
  --cc:gcc: 111846 lines compiled; 5.279 sec total
  --cc:tcc: 111846 lines compiled; 3.473 sec total

[sk-Prime]

that is great. It is a continuous process, every release wnim can be better than before if we can identify the problem. Number of Nim language user is increasing, so, demand for wnim also increasing.

[khchen]

The new version is released. Almost all ways (winimx, modules, etc.) that I can do to improve compilation time are done.

[sk-Prime]

yes, the compilation time reduced significantly. i compiled same program of https://github.com/khchen/wNim/issues/27#issuecomment-527292031 again using import wNim/[wApp, wFrame,wStaticText,wPanel] , and it took only 4.11 second. more than 50% compilation speed increased. great work.