Question about running Neural Style on Linux Subsystem for Windows

[jcklpe]

Hey so I've got Neural Style running on a linux partition no problem but my primary OS is still Windows (due to the nature of hardware prices and VFX work that I do).

I've been playing around with cygwin for some web dev stuff I've been doing and heard about this from a friend: http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/

It's a posix compatibility layer for Windows 10. Does Neural Style run on this? Can it? I know there's only one way to know (to try!) but I wanted to check in if anyone else had tried it already or if someone who had a deeper understanding of CS could chime in with an informed opinion.

I'm just an artist/designer with an interest in technology (still working on my artist article for neural networks but making progress) but I like learning more about this stuff as a hobby and having my NN stuff more immediately available inside Windows would be great for speeding up my workflow (as it stands I have to reboot and go into my BIOS to switch everytime I want to do something differently)

[htoyryla]

David French notifications@github.com kirjoitti 12.12.2016 kello 18.29: I've been playing around with cygwin for some web dev stuff I've been doing and heard about this from a friend: http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/ http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/ It's a posix compatibility layer for Windows 10. Does Neural Style run on this? Can it? I know there's only one way to know (to try!) but I wanted to check in if anyone else had tried it already or if someone who had a deeper understanding of CS could chime in with an informed opinion.

I'm just an artist/designer with an interest in technology (still working on my artist article for neural networks but making progress) but I like learning more about this stuff as a hobby and having my NN stuff more immediately available inside Windows would be great for speeding up my workflow (as it stands I have to reboot and go into my BIOS to switch everytime I want to do something differently)

I remember a discussion related to someone trying to use the W10 linux shell, either for neural-style or one of the other style transfer programs. There were problems, possibly related to getting GPU to work. I remember that a header file for its (outdated or non-standard?) kernel was unavailable, meaning that one could not install something that required compiling against the kernel.

I have a long IT background, but currently my focus is on art. When I became aware of style transfer and other neural image processing possibilities, I bought a separate linux box for it. My main computer is a Mac, mainly for working with image, video and music/sound. The neural processing is done on the linux box. I want to keep the Mac clean, even if it should be possible to install neural-style on it (I have bad experiences of installing libraries in OSX which then interfered with the OSX software).

It would seem to me that Ubuntu is by far the easiest way to run neural-style and other similar programs which have been developed on Ubuntu. Getting them to run under different operating systems requires more technical knowledge and ability to solve technical problems. So, if the main objective is an artistic workflow, then using a different OS will take away time from that objective as one will be struggling with the technical problems instead.

But for sure, having to boot back and forth is no good workflow. A second computer for linux is what I recommend.

[jcklpe]

@htoyryla good to know. Yeah the gpu cost for a second computer is too high since my main box needs a pretty big gpu already. Seems most people do the aws thing though that requires more backend knowledge than I currently possess. One advantage of doing that though is aws is a standardized cloud environment that people could pretty easily follow instructions for. Maybe I'll look into that. Got to see how the price is though.

[htoyryla]

@thedonquixotic I have not even tried AWS... seems to me that the cost could be quite high in the long run (at least for me as I do a lot of model training so that the GPU may be running for hours and days). If you only do a few style transfers now and then, and not too much experimentation (which for me takes a lot of time) then AWS could be effective pricewise.

I have a GTX1070 (8G RAM), it cost around 500€, and I think than even a 1060 (6G RAMfor under 300€) might be enough for many things. But if higher resolution is needed, then the cost will be higher.

I am not sure either how much GPU RAM one can access in AWS.

[ProGamerGov]

@htoyryla

I am not sure either how much GPU RAM one can access in AWS.

You can access around 4GB for the g2.2xlarge instance, and around 12GB for the p2.xlarge instance. Both of these instances have a single card version, and a quad card version.

The p2.xlarge instances use an NVIDIA K80 GPU.

The g2.2xlarge instances use either an NVIDIA K40 GPU or an NVIDIA GRID GPU.

Sources:

https://aws.amazon.com/ec2/instance-types/p2/

https://aws.amazon.com/ec2/instance-types/

[htoyryla]

There seem to be a lot of options there, more than I remembered. It is not easy to see though how much it would cost in practice (compared to plain virtual servers which I use routinely, there you can easily see that for a configuration, the cost will be, say, 25 € per month). I'll stick to my own gear... I know what it costs.

[bododge]

@htoyryla I have neural-style running on OSX 10.10.5. In the end it isn't really too much harder to setup than on Linux. I run tons of 3D and Adobe softwares, but haven't run into any stability or conflicts of any kind with other software on my system.

[htoyryla]

@bododge, years ago, having discovered that my Macbook was running Unix under the hood I started routinely installing our own server software in it so that I could do demos to customers without needing a net connection. Everything worked smoothly except for a library problem that I noticed later... which prevented php from running. In any Linux distro, I could easily have re-installed the correct libraries, but not so in OS X, as there is no repo for the original OSX sw as supplied by Apple.

After this, in itself minor issue, I have been more cautious with my Macs. I believe that installing packages under macport or brew should be more or less safe, meaning that they are kept separate from the original OSX software. And I have indeed some software installed with Macports in a Macbook (see PS below). Still, there has a be a strong reason for installing something non-OSX in one of my Macs.

For me personally, Ubuntu is much more convenient for experimenting with neural networks. I can have a CUDA-dedicated GPU, as much RAM as I need and as powerful CPU as I need, at the price of no more than the cheapest Macbook. In Ubuntu, I can understand quite well where the installed software goes and what to do if something goes wrong as I can remove and reinstall any packages if needed. Despite this, I have had my share of problems in Ubuntu too, mainly related to the GPU (like the login loop problem which cause a lot of trouble until I understood what was really going on).

So, taking into account that my Ubuntu box is far better equipped for neural experimenting, I see no point all taking any risks with an OSX machine which would anyway be less perfect for neural experiments. Would hate spending a few days getting everything back and running when I should actually be editing video, for instance. With an Ubuntu box or maybe even two, it is no big problem even if something goes wrong (like when I got a login loop after Nvidia installation).

PS. I did recently try to install OpenCV 3.0 in a MacBook, in order to be able to demo a webcam based (non-neural) experiment. I got it working, almost... except for a problem that persisted, my script would always start ok but fail to run for a longer time. It turned out to be a known problem for which no solution existed (or the known solutions would have been too risky). In Ubuntu there almost always is a solution.

So I am not saying that installing neural-style in OSX would be difficult or risky. It is rather that my approach is that Ubuntu and OSX are good for different things and I use each for what it is best equipped to handle. That way I am myself more efficient and productive. Why should I make things more difficult to myself?

[bododge]

@htoyryla it sounds like you have far more experience with these sorts of issues than me. I respect that, and thank you for the detailed answer. It sounds like you know what to do, and don't have a hard time dong it except where Mac can make things tricky or impossible at times! Whenever I launch Ubuntu I miss all my usual software. I end up feeling like a caged bird, so for me neural-style on the Mac is very desirable, because I often want to work with other video software and photoshop to process the results.

[htoyryla]

As I have Ubuntu and OSX running on separate machines in the same local network, I can quite easily share the files, have a Terminal open on OSX to run programs in Ubuntu and then access the results from OS X. Or take a few steps to the Ubuntu machine when doing a longer session on it, then going back to the iMac desk to access the results. I, too, prefer to do sound (Logic Pro), video (FCPX) and image (Affinity Photo etc.) work in OS X. I have been doing similar things for around ten years now, using Linux, OSX and earlier even Windows according to what happened to be the easiest way for each task.

[bododge]

@htoyryla

But for sure, having to boot back and forth is no good workflow. A second computer for linux is what I recommend.

I think you're right about this, that's why I end up feeling stuck on one side or the other of the boot process.

[htoyryla]

Then again, if one only needs neural-style and can successfully install it in another OS, then that is obviously a good solution.

I am constantly trying out new software and also making my own experiments, so my software environment is changing weekly, it is simply easier to manage in Ubuntu.

[jcklpe]

Done

[jcklpe]

Hey I closed this issue in March but wanted to check in and see if anything had changed?

I had to do a clean install on my computer and when I did that I got rid of my Ubuntu partition. I just finished setting up the Linux Subystem for Windows on my computer again and I was wondering if maybe the Subsystem would work now? They've updated it a lot. Now it's not just the Linux Subsystem, it's actually being distributed as Ubuntu for Windows. I don't know how much of the header or kernel etc there is in it now, or whatever, but so far it seems pretty full featured. Is there any reason why neural-styles couldn't run on it with gpu enabled?

[jcklpe]

Here's some links with info: https://www.microsoft.com/en-us/store/p/ubuntu/9nblggh4msv6

https://docs.microsoft.com/en-us/windows/wsl/interop

[jcklpe]

I also asked at the WSL github page and received a good answer:

Looks like the answer is no.

https://github.com/Microsoft/WSL/issues/2809

https://github.com/Microsoft/WSL/issues/327

https://github.com/Microsoft/WSL/issues/829