Better Automation of build initialization and setup

[leandor]

I HATE CMake :P

But is simply the only option available for providing quick cross-platform compilation, without requiring POSIX tools on Windows.

Problem that I faced trying to use CMake on Windows with VC++, is that there seems to be some dumbness in the handling of dependency discovery (specifically Boost) and I spent several hours trying to make my build generate and compile with CMake, while if I was using VC++ directly it would have been a matter of minutes.

Another thing that bothers me, is compilation speed. CMake doesn't support pre-compiled headers, and for any project that uses templates (boost or STL are prime examples) they are almost a requirement, since they cut compilation times by an order of magnitude.

Full compiling of CBash take ~10 mins on my PC, while most of the time lost is spent recompiling the same set of headers that are used in most source files.

I found some solutions for this around (this or this and some here) but I have yet to put them to test.

I'd like to provide a batch script that better handles dependency setup, like, for example, download/clone dependencies from GITHUB (all of them are here) and execute their builds and afterwards, setup our own dependency variables with the correct paths to them.

It's very low priority for now, but I lost so much time to this, that I would hate that the same happens to anyone else.

((Also, and this is personal opinion, another thing I hate is the CMake language/format itself is SO clunky and amateur... it seems more a patchwork of ideas pieced together than a formal language.))

[Utumno]

Have a look here for my adventures with this lovely tool when I forked and patched liblo for 306: https://github.com/Utumno/libloadorder/wiki/Build-on-windows. Glad I am not the only one

[leandor]

Great write up! I was thinking of doing something like that for CBash too. I may have to borrow that :)

Boost seems to be the contentious issue :P

I've found dozens and dozens of questions about CMake not finding boost. I have to say, that being a C++ nerd myself, I love Boost... but they have the most convoluted build system I've seen... (though, it's probably easier to use than CMake >.<)

What's confusing is that it seems at some point Boost itself used CMake (or include some CMake definitions at least), and that's why CMake module FindBoost hast that variable that enables a 'special' search that tries to find the CMakeLists.txt on boost dist.

Also, it all started wrong since at first I used last Boost release 1.62 (as of now....) and last stable CMake still doesn't recognize that version... it's too new! :) (there's a more recent CMake but it's release candidate)

And to top that, I have installed C++Builder 2010 (from Borland) in this PC that comes with a pretty old Boost distribution, and CMake was finding that one instead of the one I was setting on the config.

Another confusing thing is that in the case of ZLIB (and maybe it's the same with all libraries, and boost it's different here) the configuration variables must point to the actual library file (.lib) itself, and not to a folder.

[Utumno]

Oh thanks, yes once the dust settles a wiki article to spare people the trouble of fighting with CMake would be needed - not sure about libs at all but IIRC for gtest you had to give it the folder - maybe depends how you define the variables (?).

Re: boost - dunno what's used in CBash but if we could use (newer) std features it would be grand - there's an std::filesystem recently I think for instance

[leandor]

@ std::filesystem: Yeah, I got my eye on that. TBH I'm not too much up to speed about the new features introduced in C++11 and 14, so I have to read up a bit on that.

AFAICT, CBash uses just boost/iostreams for the memory_mapped file access, a special container, and some minor standard helper types. Nothing too big. Sadly, I don't think that the memory_mapped class is integrated on the standard yet... but don't quote me on that, I'll check nevertheless.

---

On other news: After spending more than a few more hours struggling with CMake, I've finally was able to tame it a bit and convinced it to generate a project file for the new pybind wrapper and make it compile and run some (very basic) tests (from Python... using pytest, yay! not a big achievement, just copied the skeleton from pybind examples :P)

The code is still a mess, so at least for today I won't push changes... (I'm tired and I'd have to seep through all the files I created and touched and see which changes are important and what I can discard.) But tomorrow if I can progress a bit more, I might be able to expose at least the bare basic functions that CBash already exports, and at least make a test case for the print error that we wanted to squash/enhance.

Also, I'm hesitant to restructure the code too much, but I think I'll have to at least rename the src directory to something that implies "that's the core API code", since the python binding, for now, lies in a pyd directory.

I hate changing too much stuff, and renaming and moving directories around always make a big mess.

We'll see.

[Utumno]

:+1:

Be aware if you're not already that the changes you push in your branches are not final - you can rewrite history there. In fact that's the way I work - rebase interactive my wip branch - then rebase again and again and again and again - see those merge utumno-xxx on dev ? There were never those branches, that's just pieces of my wip branch that were stable enough for dev. Actually rewriting the final commit messages and branch layout is the most tedious thing I go through - much more fun to crank the code than to publish it on dev, crosslinking to issues, rewritting commit messsages, reviewing nothing broke (this is the hateful part) and making sure the commits are readable - that is focused.

[leandor]

Yeah, gotcha... I'm still adjusting to the idea of editing history on a "public"[1] branch :D

Believe me, I love rebase interactive and I've used it extensively over the last years at my work place on my own private repositories (and I have to thank GIT SVN for that, or else I'd still be stuck with SVN working copies) it just seems that I'm still in "public means clean and pristine" mode, that I'm so used to from my work :)

I still have lots to learn about community driven and open source projects.

[1] By that I mean, visible to anyone. I always edited history and made a mess on my own local repos :)

[Utumno]

Haha keep an eye on my branch and see what pristine means hahaha :D

EDIT: don't get me wrong - I really like your branch being so tidy, I just tell you you don't have to.

BTW I lost where this was mentioned but as you may be aware wikis here are git repositories so cloning Lojacks and pushing to this one should be all that's needed. Just the addresses of what is where is confusing - so here's a quick reminder: http://stackoverflow.com/questions/15080848/how-do-i-clone-a-github-wiki )

[leandor]

Yeah I noticed that after I initially copied the Home page by hand. I was curious about how wikis were handled so I looked into it and noticed you can clone them.

I suppose I can copy all the content now, I was feeling a bit weird about "borrowing" it from Lojack's wiki though :P Though, I could add a footnote telling from where the original content comes.

[leandor]

Ah, btw. I already copied Lojack's wiki content to this site, and added a foot note with the link to the original wiki,

[Utumno]

Excellent ! Now that you are investigating all those new technologies it would be great if you wrote some wiki articles on them as they relate to CBash and as gentle introduction for probable contributors. I mean appveyor travis etc.

Thanks :)

[leandor]

Good point! Yeah, I'm still wrapping my head around some of the details myself.

But, of course, will do! After I finish tweaking and experimenting with them, and all the details have settled.

[m0lz]

Ref issue title - Word = Automate or better still automation :)

Better Automation of build initialization and setup

[leandor]

:P Thanks! Not being a native speaker, sometimes I have those kinds of brain farts :)

[leandor]

I've found this interesting link which marks that probably some time in the future working with CMake in VS will become much easier[1].

Keep in mind that the '15' in that article refers to VS internal version, not the year... and I think it's meant as 'next version', as VS2015 is '14', for example.

[1] As every time something like this happens, I hate having to endure the rough times before the change :P

[Infernio]

Putting on backburner for now, building it isn't that painful (I've built far worse than this :P). But certainly something to consider to make it easier for newcomers to hop into the code.