Windows Build Support

[utensil]

This is an issue logging what seems to be missing for a successful windows build of nupic.core:

@rcrowder and @kandeel 's work

• [ ] #184 and https://github.com/numenta/nupic.core/pull/184#commitcomment-7475075

  Compiler support

• [ ] current cmake instruction generates solution for VC by default, not MinGW or Cygwin, needs to doc how to generate makefiles for them

  VC build issues

• [x] CMakeLists.txt seems to be adding gcc/clang-specific compiler options to all compilers, see variable NTA_CXXFLAGS_BASE
• [x] due to similar reasons(the usage of -isystem), VC failed to find all headers and libraries in external, should be fixed by using include_directories([AFTER|BEFORE] [SYSTEM] dir1 dir2 ...)

  MSYS gcc build issue

By using cmake $NUPIC_CORE/src -G "MSYS Makefiles", make files for gcc under MSYS can be generated, but fails the build:

• [x] an typo #TODO in ArrayAlgo.hpp
• [x] VC-specific asm code in ArrayAlgo.hpp and SvmT.hpp
• [ ] something wrong with DLL and COM loading code:

[ 10%] Building CXX object CMakeFiles/nupic_core.dir/main/engine/RegionImplFactory.cpp.obj
w:/workspace/nupic.core/master/src/main/engine/RegionImplFactory.cpp:1:0: warning: -fPIC ignored for target (all code is position independent) [enabled by default]
In file included from c:\devkit\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/objbase.h:153:0,
                 from c:\devkit\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/ole2.h:16,

[ 10%] Building CXX object CMakeFiles/nupic_core.dir/main/engine/RegionImplFactory.cpp.obj
w:/workspace/nupic.core/master/src/main/engine/RegionImplFactory.cpp:1:0: warning: -fPIC ignored for target (all code is position independent) [enabled by default]
In file included from c:\devkit\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/objbase.h:153:0,
                 from c:\devkit\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/ole2.h:16,
                 from c:\devkit\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/Windows.h:94,
                 from w:/workspace/nupic.core/master/build/release/include/nta/os/DynamicLibrary.hpp:32,
                 from w:/workspace/nupic.core/master/src/main/engine/RegionImplFactory.cpp:29:
c:\devkit\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/unknwn.h: In member function 'HRESULT IUnknown::QueryInterface(Q**)':
c:\devkit\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/unknwn.h:70:82: error: '*' cannot appear in a constant-expression
c:\devkit\mingw\bin\../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/include/unknwn.h:70:82: error: a function call cannot appear in a constant-expression
make[2]: *** [CMakeFiles/nupic_core.dir/main/engine/RegionImplFactory.cpp.obj] Error 1
make[1]: *** [CMakeFiles/nupic_core.dir/all] Error 2
make: *** [all] Error 2

Personally I'm interested in making nupic.core work under Windows, but I guess a PR is not going to be created that soon. Just log an issue here for follow up.

UPDATE: See https://github.com/numenta/nupic.core/issues/103#issuecomment-58133631 for latest prgress .

[david-ragazzi]

Hi @utensil, there're an old discussion about some problems in use MSVC as compiler which also could be useful: http://lists.numenta.org/pipermail/nupic_lists.numenta.org/2014-January/002502.html http://lists.numenta.org/pipermail/nupic_lists.numenta.org/2014-January/002593.html

Reggarding to the points you mentioned:

CMakeLists.txt seems to be adding gcc/clang-specific compiler options to all compilers, see variable NTA_CXXFLAGS_BASE

I reccomend try use CLang/GCC with Visual Studio as addressed here: http://stackoverflow.com/questions/4687789/use-clang-with-msvc-2010

As the current project already compiles for CLang and GCC, this is the easiest way to get NuPIC on running Windows.

Use ClangVSx plugin forces VS IDE to use CLang as compiler which make the gcc/clang-specific compiler options acceptable.

due to similar reasons(the usage of -isystem), VC failed to find all headers and libraries in external

Even with ClangVSx plugin, some problems could remain. One of these is this headers' issue. We should force CLang to ignore default headers and use those headers in external. This can be done using the suggestions presented here:

http://stackoverflow.com/questions/14902683/compiling-clang-on-windows

Unfortunatelly, I'm using Ubuntu and still didn't install a VM with Windows to test this solution, but I have in mind that this will work fine.

By the way, we could adapt the project to be compiled by MSVC compiler natively. But this is certainly a tricksy job (but still interesting)..

[utensil]

@david-ragazzi Thanks for all these valuable materials and your previous efforts.

VC is one of the mainstream compilers on Windows, despite of how much I favor gcc, it stills is. So in this issue, VC means the compiler, not the IDE which I don't like at all.

Your post implies that nupic.core is currently better to be compiled with gcc/clang, that's not exactly true. In my second comment, I identified a few issues regarding compiling nupic.core under gcc(using MSYS/MinGW), mostly because many code assumed VC under Windows: VC specific asm codes.

So here is the awkward situation of nupic.core under Windows:

• CMake defaults to generate VC solution files, and code assumes VC under Windows
• CMakeFile.txt adds gcc/clang-specific options to the compiler

I could create a PR to fix the obvious parts of the problems, make nupic.core more friendly to both VC and MinGW, but a complete port seems to be a lot of work.

[david-ragazzi]

Your post implies that nupic.core is currently better to be compiled with gcc/clang, that's not exactly true.

I'm not saying nupic.core is currently better to be compiled with gcc/clang or making a value judgment about which is the best compiler or not. I just am saying that if someone wish see NuPIC building on Windows, maybe integrate VS IDE with CLang could be the easiest way, as the curent C++ already is ported to working with CLang. I also would love a native support to VC (i.e. the compiler), but this is tricksy as involve some code adaptations.

By the way, the good news is that I was able to build and running NuPIC on Windows (using natively VS compiler and IDE) a time ago. The bad news is that after CLang support, some code were added which increased the number of issues of imcompatibilty during build which made I postpone this job.

[utensil]

@david-ragazzi I was careless to use the word "better", it should be "easier". Still, making nupic.core compile with clang under Windows is not easier, but I would create a PR to ease it.

[david-ragazzi]

@david-ragazzi I was careless to use the word "better", it should be "easier". Still, making nupic.core compile with clang under Windows is not easier, but I would create a PR to ease it.

If is not easier, then we should continue trying compile in MSVC..

Btw, @rhyolight: Travis has support for Windows??

[rhyolight]

Btw, @rhyolight: Travis has support for Windows??

Nope! https://github.com/travis-ci/travis-ci/issues/216

[david-ragazzi]

Btw, @rhyolight: Travis has support for Windows??

Nope! https://github.com/travis-ci/travis-ci/issues/216

This is my fear.. :-( Even that we port the code to Windows and MSVC, new merges could create build issues at any moment..

I see that the only solution (until Travis has support to Windows) would we have a fixed Windows reviewer to build and run locally on Windows before him approves a PR..

[utensil]

Even that we port the code to Windows and MSVC, new merges could create build issues at any moment..

No worries, as long as we clean up the old non-cross-platform way(which is only a very small part of nupic.core) of writing codes and builds and set up some guidelines, it would be a lot easier to prevent new problems and fix them.

I see that the only solution (until Travis has support to Windows) would we have a fixed Windows reviewer to build and run locally on Windows before him approves a PR..

Not that strict I guess. I can be such a reviewer, but not for every PR, just periodically checking it would be just fine. I can help to make sure every release(there's still a long distance to a first release) builds under Windows, but not every commit. And if there're many Windows user of NuPIC(which I doubt it), the population base would make issues easy to spot, report and fix, just like before, when NuPIC didn't have Travis Mac support.

[david-ragazzi]

And if there're many Windows user of NuPIC(which I doubt it), the population base would make issues easy to spot, report and fix, just like before, when NuPIC didn't have Travis Mac support.

Yeah, you're right.. I know that many Windows users are waiting for this moment and will report any issues after the job is done..

[david-ragazzi]

Btw, I think the next breakthrough is the Windows support.. I trust in you to do this job (if you accept)! Your contributions have been well valuable to the project!

I also can help you.. I want use the .Net bindings, and for this I need that NuPIC compile in Windows..

[utensil]

Initial work started at https://github.com/utensil/nupic.core/tree/103-friendly-to-windows

[david-ragazzi]

Initial work started at https://github.com/utensil/nupic.core/tree/103-friendly-to-windows

Really nice! I'll try help where I can.

[subutai]

Very nice to see!!

[david-ragazzi]

@utensil

Please take a look on this:

https://www.mail-archive.com/nupic@lists.numenta.org/msg00688.html

This link has a detailed recipe (Porting Code to Win32 section) of what should be changed in .cpp code to it be compiled by MSVC. Some info obviously is deprecated but still these tips work well in several parts..

In some changes as non-cross platforms macros I used #undef but certainly they could be handled using -U option in NTA_COMPILE_FLAGS avoiding so changes in the code..

[utensil]

Please take a look on this:

https://www.mail-archive.com/nupic@lists.numenta.org/msg00688.html

This link has a detailed recipe (Porting Code to Win32 section) of what should be changed in .cpp code to it be compiled by MSVC. Some info obviously is deprecated but still these tips work well in several parts..

@david-ragazzi Thanks! This is very very helpful! :100:

[utensil]

And I'll update the wiki page https://github.com/numenta/nupic/wiki/Running-NuPIC-on-Windows to date as well.

[utensil]

For who are concerned, @david-ragazzi and I are having some discussion on a commit here.

[utensil]

@david-ragazzi I've reviewed the implementations of CMake variables that tells the compiler here, and I'm convinced that these compiler variable are good(my weird results are caused by other reasons), and I can actually make use of ${CMAKE_CXX_COMPILER_ID} to tell the difference of g++ and clang++ in the same way as here.

[david-ragazzi]

@david-ragazzi I've reviewed the implementations of CMake variables that tells the compiler here, and I'm convinced that these compiler variable are good(my weird results are caused by other reasons), and I can actually make use of ${CMAKE_CXX_COMPILER_ID} to tell the difference of g++ and clang++ in the same way as here.

Nice hear this!

[david-ragazzi]

Ahhh.. other tip to avoid direct changes in .cpp code: try force MSVC to NOT USE c++11 standard. I dont't know which directive does this, but I know that VC 2013 has now support to c++99.. The fact of the c++11 option is enabled raise several issues presented in the "recipe".. And the current code of NuPIC is not ported to c++11 (it's a long discussion).

In other words, let's try address what is possible using NTA_COMPILE_FLAGS. In addition to avoid changes in code, this pratice also avoids new issues..

Update: From I understood from a Microsoft team member (Stephan T. Lavavej) said in the link bellow, you don't need use directives to specify c++ standards.. VS 2013 already handles compatibility in an automatic way. The issues above were raised I build NuPIC with VS 2012:

http://blogs.msdn.com/b/vcblog/archive/2013/07/19/c99-library-support-in-visual-studio-2013.aspx

[breznak]

:+1: awesome work @utensil
is there a (work-in-progress) developer's wiki HOWTO nupic@windows? I'd give it some testing..

[utensil]

Setup CI for Windows is now possible by https://ci.appveyor.com , see atom/atom#2444 , investigating. :clap:

[rhyolight]

@utensil I'm currently investigating Circle CI as well. Let me know what you find.

[utensil]

In case anyone wonders, I'm still working on this at https://github.com/utensil/nupic.core/tree/103-windows-build , but I'm only making little progress because my work is driving me tired lately, that is: even when I have time, I'm not in a good status.

[rcrowder]

After battling with the current NuPIC.core master to build with Visual Studio, I've now come across this issue thread :) Looks like I've encountered and covered most of the changes described here, and the relevant mailing list discussions. With this issue closed, is it worth pursuing this work with a clone of the core, or move over to one of Utensil's forked branches?

[utensil]

Hi @rcrowder , this issue is still open. I didn't have much time to work on this issue lately, but I haven't forgotten about it. You are welcome to help with this issue based on my branch https://github.com/utensil/nupic.core/tree/103-windows-build . I've just merged master in so it's up-to-date now.

[rcrowder]

Excellent, thank you. And thanks for merging the master :) I'll clone that 103 branch this weekend and compare to my changes. Although looking through the branch's change lists, shows I covered the basics to get it working but with test failures. I imagine we can move further discussion and advice onto that branch and away from this Closed issue for the time being.

[utensil]

I imagine we can move further discussion and advice onto that branch and away from this Closed issue for the time being.

@rcrowder I've opened a PR so we can work together and discuss there.

And this issue(#103) is still open, you may see the open badge on the top of the issue. The reason that you are seeing the closed badge at the bottom is that @rhyolight mentioned this issue at the closed #155 .

[rhyolight]

Hi @rcrowder. Thanks for offering your help. Let me know if you need anything.

[rcrowder]

Thank you Matthew. Welcome back.

Neuroscience and Cognition PDF drop box http://goo.gl/6M7qzT ./ Burt http://rcrowder.github.io/Burt

On Mon, Aug 18, 2014 at 8:41 PM, Matthew Taylor notifications@github.com wrote:

Hi @rcrowder https://github.com/rcrowder. Thanks for offering your help. Let me know if you need anything.

— Reply to this email directly or view it on GitHub https://github.com/numenta/nupic.core/issues/103#issuecomment-52543848.

[rcrowder]

Hey, I've made some progress cross compiling with Travis, similar to Utensil's work with AppVeyor. For example, https://travis-ci.org/rcrowder/travis-ci-test/builds/33062370

Where should further discussion take place? Here, or accumulate in PR #161 ? Or even back on the hackers ML?

[rhyolight]

@rcrowder I suggest you create a pull request that targets @utensil's 103-windows-build branch (which is PR #161). Then Utensil can review it before merging it into his PR.

[utensil]

@rcrowder I suggest you create a pull request that targets @utensil's 103-windows-build branch (which is PR #161). Then Utensil can review it before merging it into his PR.

Yeah, this is the recommended approach.

[rhyolight]

@rcrowder If you are new to git and GitHub, you can see a video of how to adhere to our workflow here:

[Thanh-Binh]

Do you have any branch for windows version? Where can I download it? Is there any sample for hot gym?

[rcrowder]

That video is great. I've watched it a few times. It took a while for the change in use of branches to click in. Hopefully I won't have to rebase my nupic fork after a few rogue checkins?

@Thanh-Binh There are work in progress branches for a windows version via forks on Github.

Neuroscience and Cognition PDF drop box http://goo.gl/6M7qzT https://www.linkedin.com/in/richardcrowder - https://github.com/rcrowder

On Mon, Aug 25, 2014 at 8:07 AM, Thanh-Binh notifications@github.com wrote:

Do you have any branch for windows version? Where can I download it? Is there any sample for hot gym?

— Reply to this email directly or view it on GitHub https://github.com/numenta/nupic.core/issues/103#issuecomment-53233778.

[Thanh-Binh]

@Richard: Can you give me a link for this branch? Do you have any application with it? I will work with MS Visual Studio. Which version (32bist or 64bits) is now compilable?

[utensil]

@Thanh-Binh The branch is at https://github.com/utensil/nupic.core/tree/103-windows-build and the related PR is #161 , available for inspection. But sorry, the branch is still a work-in-progress and not compilable under Windows(neither MinGW or MSVC) yet. Hopefully I would get back on this around Sept 3rd . @rcrowder might have some better progress.

[Thanh-Binh]

Hi Utensil,

thanks for your feedback. Please let me know if it is compilable.

 

Gesendet: Dienstag, 26. August 2014 um 02:18 UhrVon: "Utensil Song" notifications@github.comAn: "numenta/nupic.core" nupic.core@noreply.github.comCc: Thanh-Binh Thanh-Binh.To@online.deBetreff: Re: [nupic.core] Windows build support (#103)

@Thanh-Binh The branch is at https://github.com/utensil/nupic.core/tree/103-windows-build and the related PR is #161 , available for inspection. But sorry, the branch is still a work-in-progress and not compilable under Windows(neither MinGW or MSVC) yet. Hopefully I would get back on this around Sept 3rd .

— Reply to this email directly or view it on GitHub.

[Thanh-Binh]

Hi Utensin, do you have any news? Binh

[utensil]

You may see https://github.com/numenta/nupic.core/issues/134#issuecomment-54309453 for progress made by @rcrowder .

[rhyolight]

Not sure if it helps, but I just ran across this wiki page when doing some cleanup:

https://github.com/numenta/nupic/wiki/running-nupic-on-windows

I seem to remember someone getting NuPIC running on Windows with a lot of source changes, and I think I created this wiki page based off a very old email to the ML with the description of how he did it.

[rcrowder]

Ah nice find! It looks like it may have come from David? Most of the changes described have been committed to the Core push request :) A few of them I've handled differently, and a few I've missed that might be added.

I also have a version of NuPIC using a .nupic_config pointing at the PR. I'm not complete with changes to NuPIC/CMakeList.txt yet. But the core static library gets build for Win64 fine, but then missing dynamic library support stops Setup. My next port of call..

[david-ragazzi]

Ah nice find! It looks like it may have come from David?

Yeah, it's mine.. @utensil already took advantage of some solutions presented in this link.. The other ones he preferred ignored due to some aspects that I didn't remember..

[utensil]

There's great on-going work at #184 by @rcrowder and @kandeel , the progress and discussion are taking place there. The PR is becoming mature and needs reviewers to review.

[Thanh-Binh]

great! when do you plan to finish review and release a runable beta-version?

[rcrowder]

Thanks. Hopefully in the next couple of weeks you should see more exposure for this feature branch and port. We are discussing that right now on the mailing list.

[Thanh-Binh]

thanks.

[rhyolight]

@utensil @rcrowder This issue and #1439 seem to be duplicates. I'd like to merge them and close this one. Does that sound okay? I want to give you both a chance to update #1439 with any relevant details from this issue before I do that.

[rhyolight]

Oh wait... this one is for nupic.core... maybe I should just make this ticket a sub-task of https://github.com/numenta/nupic/issues/1439.