MSVS 2015+ build of Python for Windows for nanoserver support

[ijstokes]

I don't really follow the details but for future reference this ML thread seemed to have some valuable information in it (related to #1219 it seems)

A separate issue is that Microsoft have dropped support for older MSVCRT versions in Nano server (I'm not sure what versions still work), we have a large codebase still using Python 2.7 and since this is built with MSVC 7 it won't run on Nanoserver. Miniconda2 would have to be rebuilt with an up-to-date compiler/IDE like VS2015. Someone has already done this successfully, http://p-nand-q.com/python/building-python-27-with-visual_studio.html

On Monday, 12 December 2016 12:38:06 UTC, Mark Snelling wrote:

Agreed, a simple .zip file would be fine. Perhaps also with a PowerShell script to set up the environment etc?

On Sunday, 11 December 2016 03:11:48 UTC, Dave Hirschfeld wrote:

I just created the image manually so there's no dockerfile yet - I think it requires that Continuum provide an installer which works on nanoserver. My suggestion in the linked issue of providing a simple zip file would probably be sufficient...

On Thursday, 8 December 2016 21:45:32 UTC+10, Mark Snelling wrote:

I don't suppose you could post your Dockerfile/scripts for this could you?

On Tuesday, 6 December 2016 23:15:03 UTC, Dave Hirschfeld wrote:

Yeah, I wasn't able to get it to work either: https://github.com/ContinuumIO/anaconda-issues/issues/1219 ...however with some xcopy hackery I created an image which you can pull from DockerHub: https://hub.docker.com/r/dhirschfeld/nano-miniconda/ I was just playing around but hopefully it works for you... Note: msi's don't work on nanoserver - that was one of the subsystems removed.

On Wednesday, 7 December 2016 00:17:39 UTC+10, Mark Snelling wrote:

Has anyone tried installing Miniconda (or Anaconda) into a Windows container? I have just tried this using microsoft/nanoserver as a base image and the installer just exits. I know that Microsoft removed 32bit support in nanoserver and it looks like the installer is a 32bit process which could be why it doesn't work.

Are there plans to either move the installer to an .msi package, or produce a native x86_64 installer? And if not is there another, more manual, way of installing Miniconda?

CC @dhirschfeld

[msarahan]

Though it is perfectly possible to compile Python 2 with VS other than 2008, it breaks compatibility with python.org builds of python 2 (which are always done with VS 2008). We have been very hesitant to make that break, and upstream has been very hesitant to lead in that direction.

See Steve Dower's comment at https://bugs.python.org/issue25759#msg258427

[msnelling]

@msarahan That certainly does look like they're not going to support anything above VS2008. Given that we're talking about a specific package for Nanoserver, is it still an issue to make the break?

[msarahan]

@msnelling what you're basically asking for is not a specific package for Nanoserver, but rather a complete new build that we document/market as something for Nanoserver (but perhaps discourage use on "normal" Windows.)

I really don't think we'll do that halfway jump, but ultimately that's up to our company leadership. If people want to use Nanoserver, I recommend python 3.5+, which uses VS 2015.

[msnelling]

@msarahan Understood.

The other part of the original thread is talking about the installation on to Nanoserver, since it doesn't support .msi and the current installer is 32bit (even for the 64bit package) which is also unsupported. A .zip file with a script to complete the installation would be very helpful.

At the moment you have to extract the .tar.bz files from the .exe manually, copy them into the container during build, extract them individually making sure that you run the package install script after each one.

I do feel this could be streamlined with a hosted .zip file containing the packages and a powershell installation script.

[msarahan]

I am in favor of creating a better installer of some sort for nanoserver. I'll try to get it on our road map.

[dhirschfeld]

Just to be clear, I don't think this issue is related to the one I brought up in #1219. As @msnelling mentions that is just to do with the miniconda installer being unable to be used on nanoserver.

FWIW, I don't think it's really feasible to provide a new build of Python 2.7 and the entire ecosystem using a newer compiler. No matter how large your codebase is, it's going to be easier to port it to Python 3.5+ than it will be to maintain a fork of the entire ecosystem.

[mingwandroid]

Is it possible to use conda clone and a UNC path to clone from the (Windows) host into the Windows docker system? If not, the fixes needed for that to work may not be too tricky.

[msarahan]

@dhirschfeld @msnelling - not sure if @mingwandroid suggestion is a path forward for you, but please comment. We'll also try to figure out a different installer, but have quite a few projects ahead of Windows docker support, unfortunately.

[dhirschfeld]

I'm not sure it helps any as it's not reproducible. In any case, can you even clone a root environment?

I've already created a miniconda image by mapping a host directory and xcopying a fresh miniconda install to the container. It works fine but you can't write a dockerfile to reproduce it.

Rebuilding the installer as a 64-bit app would be the ideal situation, but all that's really needed to proceed now is a simple zip-file deployment option in addition to the current installer.

[msnelling]

I wrote an installation script in Powershell (https://gist.github.com/msnelling/49f222f5d608c55b1537639e86599645)

Parameters:

• -Exe7z path to the 7z.exe binary used to extract the package .bz2 and .tar files
• -PackageDir path to the extracted contents of the Miniconda installer .exe
• -InstallDir path to install Miniconda to, usually "$Env:ProgramFiles\Miniconda3"

Nanoserver Requirements:

• A version of 7z.exe built with Visual Studio 2015.

I would have had the script also download the .exe and extract it but 7z chokes on the file.

[dhirschfeld]

ping! Just checking (hoping) a zip-file deployment option will be available for py36 Miniconda?

[csoja]

Yes - though the py36 Miniconda installers are not available yet (hoping this week), but when they are, there will be zip versions posted here: https://repo.continuum.io/miniconda/.winzip/

[dhirschfeld]

Great! Just to be clear though, the current "zip" files just unpack the exe installer which wouldn't help with the nanoserver install - the zip file has to contain the actual xcopy-able files/folders for it to be useful for making nanoserver images.

[dhirschfeld]

Not sure where to mention this but it appears that find.exe doesn't exist on nanoserver so I was running into some conda issues. It appears that conda master no longer uses find.exe so the point is moot, however it might still be worth noting.

cc @mingwandroid

[dhirschfeld]

I've checked the latest "zip file" installers and found that they are still just the exe bundled in a zip file. The exe doesn't work on nanoserver as described in #1219.

Are there any plans for an actual zip file deployment which will work on nanoserver?

[msnelling]

I also just checked the "zip file" installers mentioned above and they are still just wrapping the .exe installer.

@msarahan @mingwandroid I'm not sure the conda clone solution would work either, this would require an existing Python install to install conda? The point of using Miniconda is to use it as the installed Python distribution.