Windows Installer won't - even when using "just for me"option

[20b5009a-ffb5-41a6-af53-2541440de0f6]

BPO	22516
Nosy	@warsaw, @terryjreedy, @doko42, @pfmoore, @ncoghlan, @tjguk, @ned-deily, @bitdancer, @zware, @zooba, @mikofski, @rkuska

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['expert-installation'] title = 'Windows Installer won\'t - even when using "just for me"option' updated_at = user = 'https://bugs.python.org/NaCl' ``` bugs.python.org fields: ```python activity = actor = 'larry' assignee = 'none' closed = True closed_date = closer = 'r.david.murray' components = ['Installation'] creation = creator = 'NaCl' dependencies = [] files = [] hgrepos = [] issue_num = 22516 keywords = [] message_count = 45.0 messages = ['227832', '227834', '227836', '238558', '238560', '238567', '238686', '238693', '238696', '238710', '238712', '238719', '238731', '238797', '238798', '238808', '238815', '238817', '238819', '238820', '238821', '238836', '238837', '238838', '238846', '238847', '238849', '238850', '238851', '238853', '238854', '238855', '238856', '238859', '238862', '238864', '238865', '238866', '238871', '238888', '238988', '239942', '239944', '239953', '246103'] nosy_count = 15.0 nosy_names = ['barry', 'terry.reedy', 'doko', 'paul.moore', 'ncoghlan', 'tim.golden', 'ned.deily', 'r.david.murray', 'BreamoreBoy', 'zach.ware', 'bkabrda', 'steve.dower', 'bwanamarko', 'rkuska', 'NaCl'] pr_nums = [] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue22516' versions = ['Python 2.7'] ```

[20b5009a-ffb5-41a6-af53-2541440de0f6]

Could not install 3.4.1 on Windows 7 Enterprise SP1 using the .MSI installer, even when using the ”just for me” option (our IM department has not given us the necessary rights to run the .MSI installer even in this mode). Please consider providing 3.4.1 (and all future releases) in a non “.MSI” file so that “admin” rights are not needed to do the install (a simple ZIP file? something similar to www.portableapps.com ?). Or as a source “tarball” for Windows machines – ideally one that is independent of compiler vendor (compileable using gcc, etc. instead of MSVC).

[bitdancer]

The source tarball is the source tarball. You can built python yourself, but it does require MSVC. There are issues in this tracker about supporting other compilers, but for various reasons (mostly having to do with this being a volunteer community driven project) they have not gotten anywhere close to good enough for us to officially support it.

For the 'no privileges' install question, I defer to Steve; otherwise I'd just close the issue.

[zooba]

The "just for me" option isn't really "just for me". I'll probably have a real option in 3.5, though it may still require admin rights.

There's no reason sites like www.portableapps.com couldn't provide true per-user installers, and Continuum Analytics and Enthought both already do (as do others - I think Portable Python is fine, though not quite up to date).

Python 3.4 probably won't be fixed. Python 3.5 might be. I don't need this issue to track it.

[340ae65e-9726-4646-a1b5-7bce30dece82]

I know this issue is closed, but as there is no voting or plus 1, I'll add my support for allowing local installation of not just Python-3, but also Python-2.7. I'm not sure what is gained by adding this restriction, or how difficult it would be to allow users without admin rights to install official python.org locally? Where is the source that needs to be patched?

IMO forcing Windows users with restricted rights to go to Anaconda, Enthought, or WinPython causes a fracture in the Python community, that decreases credibility for Windows users. Quoting Tim Peters from import this, "There should be one-- and preferably only one --obvious way to do it." not 3 or more ways. Having most new users installing Anaconda has many disadvantages:

1. Developement of an open source optimized version of NumPy (eg: using OpenBLAS) has a lower value proposition because users can just install Anaconda, which uses an unoptimized version of Intel MKL. Of course users can continue to get Christoph Gohlke's optimized MKL NumPy until he decides he doesn't want to anymore.

2. There will be two (or more) Python package indexes: PyPI, conda:binstar, binstar and pythonwheels. Do we really want Python packagers to have to upload to all of these repos? Which is the most current? etc. = less credibility for Python

3. There are/will be more than one python package installer and virtualenv tool: conda vs. pip/virtualev

Are there any repeats of this issue/feature request?

[340ae65e-9726-4646-a1b5-7bce30dece82]

one more note, NaCl (J. Morton), you can install from the msi to any directory you want by using the following from a command line opened to the folder where you have the installer:

C:\path\to\msi\download> msiexec /a python-2.7.9.amd64.msi TARGETDIR=C:\Python27

or

C:\path\to\msi\download> msiexec /a python-3.4.3.amd64.msi TARGETDIR=C:\Python34

[zooba]

Supporting proper per-user installs of Python 2.7 or 3.4 would require a second MSI that is configured to not ask for administrative privileges. The code is in Tools/msi/msi.py if you want to look at it (it's not my code and I have no intention of touching it), but the bigger concern is you're asking python.org to host twice as many files.

I'm planning to discuss installers at PyCon in a few weeks with the other core developers, so there may be a change of approach after that. Currently, we view Continuum and Enthought as the equivalent of any of the Linux distributors - either you live within their walled gardens, or you brave the outside world yourself. Linux distributions have shown that it's not a fatal split in the community, though it can lead to some obscure issues when people don't understand how they got/get Python.

[20b5009a-ffb5-41a6-af53-2541440de0f6]

First, thanks for the comments - perhaps there's hope.

But from the viewpoint of a typical user (who does not care about what goes on in the background), some of what has been said does not match the realities of development in a corporate environment.

1 – Anyone who is working on a Windows box controlled by an overly anal-retentive IM department (i.e. no admin rights) will not be able to install Python. Either make the “just for me” option unrestricted or remove it entirely. Time wasted trying to install Python when it can’t be done does endear Python to new users and comes across as being rather unprofessional. I haven’t gone to the web page lately, but as an absolute minimum the page for the “just for me” option needs to state that Admin rights are needed. 2 – Anyone forced to use Windows and is NOT a total python fanatic will NEVER go to the pain of building it. In the corporate world - on any platform - this approach is generally unrealistic given the other tools already out there (not worth the time/effort/pain involved, especially if the boss is breathing down your neck about the project schedule). And even if they want to, they may not be able to build/install it – no tools, as per point 1. 3 – I do not see the comments (in msg238567) “require a second MSI that is configured to not ask for administrative privileges” and “hosting twice of many files” as having much validity. To better serve the community a better approach is to REPLACE the pointlessly restrictive “admin rights” installer with an unrestricted version (it’s how I got Firefox on this machine) – hence only one set of files. This set should be, as much as possible, completely independent of everything else on the system (disk space is cheap, networks are fast and the file sizes are, relatively speaking, small).

4 - I haven't yet tried the "MSI' workaround (in msg238560) so I don't know if it will work on my restricted system. If it does, the workaround should appear on the web page.

Finally (as pure self-interest) I'd like a version compatible with Eclipse/PyDev.

I do appreciate and encourage what you guys are doing and also understand the realities of what you do. But my viewpoint is, that to have closed this issue without properly addressing the basic problem reminds me of an ostrich with its head in the sand.

[340ae65e-9726-4646-a1b5-7bce30dece82]

1. J morton: just install Anacondahttp://continuum.io/downloads

2. I will take a look at the msi and see if I can accommodate both installs in the same installer

[zooba]

@Mark: You can't - MSIs have a single flag indicating whether they should elevate or not. That's why I moved the 3.5 installer to a new format. Unfortunately, since we're in a transitional state, admin privileges are necessary to install the new CRT version, but the per-user is a true per-user for 3.5 (and when the new CRT is already installed via normal Windows Updates, you won't even be prompted).

@J. Morton: I appreciate the suggestion of only supporting per-user installs, and I'm heading towards that position. It won't be changing for 2.7 or 3.4 at this stage, but I do plan to discuss exactly this issue soon with the core team and figure out what we're actually trying to achieve with our installers.

I would never suggest that somebody build Python themselves unless they are a distribution, and ISTM that even on Windows, 99% of Python users should be getting a distribution rather than the python.org installers. (Which makes the python.org installers fairly redundant, hence my plans to discuss what they should be trying to achieve.)

The /a "install" is not a real installation, and though it "works", it isn't necessarily supported. (It's actually a transformation of the MSI file to separate the files to install from the installer - you should see the MSI file get considerably smaller after you run that command.) In particular, the py.exe launcher is not going to work, and many other programs that automatically detect Python will also fail.

Not sure what you mean about "compatible with PyDev/Eclipse". It's their responsibility to be compatible with Python (and I say this as the lead developer of Python Tools for Visual Studio, where we've pulled all sorts of tricks to remain compatible with Python :) ).

[terryjreedy]

99% of Python users should be getting a distribution rather than the python.org installers.

I do not understand this at all. Why? My understanding is that at least some of the separate distributions are separate gardens cut off from the wider pip-accessible python ecosystem.

[zooba]

My understanding is that at least some of the separate distributions are separate gardens cut off from the wider pip-accessible python ecosystem.

Maybe some are, but certainly not all of them. There are sometimes cross-dependencies that require you to go to the same source for libraries (e.g. scipy and numpy need to be compatible), and if they've rebuilt Python itself then PyPI wheels may not work, but none of the major distros block or inhibit pip at all to my knowledge.

[pfmoore]

99% of Python users should be getting a distribution rather than the python.org installers

I'm not at all convinced by this logic. On Windows, as far as I'm aware, the only "major" Python distributions are Anaconda and Enthought, both of which are intended mainly for the scientific community.

What "distribution" would you suggest for a web developer? For a system admin who wants to write automation scripts? For a database admin doing reporting and automation? For an application developer planning on packaging his application with py2exe/cx_freeze? For a game developer embedding Python? I'd assume all of those target audiences would use the python.org installers.

Honestly, my guess (conceded, this is based on no actual metrics) would be that 99% of Windows users use the python.org installers, certainly not the other way around.

[zooba]

my guess (conceded, this is based on no actual metrics) would be that 99% of Windows users use the python.org installers, certainly not the other way around.

Based on numbers I've heard it's certainly a majority currently using the python.org installers, but that doesn't mean they should be.

I see three main use cases for the installers:

• people who need Python for themselves
• people who need to bundle Python inside their app
• admins who want to pretend/make-as-if Windows was shipped with Python

Currently, the python.org installer is partway between 1 and 3, and somewhat closer to 3. I've heard a lot of complaints from people in #1, who are mostly helped by one of the distros, and #2 who don't have a real good option right now. (99% in category #1 is an exaggeration, but they're still a majority.)

Being somewhere between these points is not good. Personally, I think #2 (which I think about as "Python-as-a-library") is something that it's really our responsibility to support. #3 is also something that only we [pydev] can do (unless MSFT starts doing it, but I'd expect a lot of the burden would be pushed back onto pydev in this case), while #1 is a fairly straightforward for anyone to do once #2 is supported.

Certainly if #2 became official, we [MSFT] would be more aggressively helping people get set up with the same version of Python as we install on our PaaS servers on Azure. Currently, we can't reliably install the python.org installers automatically in case it conflicts with the user's existing configuration.

Most of your examples use the python.org installers because they are in category #2 and don't have a better option. When/if we decide what category the python.org installer should focus on, it will make it easier to explain which option people should use.

[ncoghlan]

From a general philosophical perspective, I share Steve's view - the vast majority of potential CPython users don't want to be consuming upstream Python themselves, they want to be getting it from a redistributor.

This has nothing to do with the quality of the software itself, and everything to do with the fact that the open source meta-experience is tailored towards DIY co-collaboration, rather than producing a streamlined and completely hassle free customer experience for a given problem domain. For education, the Raspberry Pi foundation will be a better supplier than us, for data analysis, Enthought or Continuum Analytics, for Linux system management, the relevant Linux distributor.

For Windows at the moment the default supported redistribution choice is going to be ActiveState (that's certainly the one we used when I used to do professional Windows based Python development), but our upstream relationship there isn't as good as the ones we have the Linux distributors or the data analysis focused redistributors.

As Steve says, that leaves the upstream Windows installers trying to fulfil two different roles - both being something which users can install purely for their own use *and* being suitable for a corporate IT department to incorporate into a Standard Operating Environment. At the moment it's more the latter, which is also fine for users with full administrative control over their own system, but no good for restricted corporate systems.

The immediate workaround is to use the Anaconda distribution from Continuum Analytics in such cases, even if you don't need the data analysis components.

In terms of consistency with what we do on the POSIX side, the case can definitely be made that "System Python" is the right default behaviour for upstream.

The "Python-as-a-library" use case is currently a problem on both POSIX and Windows, and PEP-432 goes into several of the reasons why. There are also a host of additional issues at least on POSIX related to building extension modules, and this is one of the big reasons Continuum Analytics don't use pip natively and use conda instead - getting extension module builds to work reliably cross-platform when running from a user directory is a genuinely hard problem even if you're just linking to C libraries, let alone if you might be linking to C++ and FORTRAN libraries as well. That's a large part of why conda is on my short list of possible language independent user level package management tools to consider proposing for use in Fedora: https://fedoraproject.org/wiki/Env_and_Stacks/Projects/UserLevelPackageManagement (and it's the *only* platform independent one, since Nix doesn't work on Windows)

[ncoghlan]

Short version of the above: I personally think we should be focusing on addressing the "system Python" and "Python as a library" cases upstream, as we're the only ones that can do that.

Solving the latter problem well then also sets the baseline for user focused installations, as those are then just a case of installing upstream CPython as if you were going to embed it in a larger application, and then just using it instead of embedding it.

For user focused cases, a good end user oriented solution will often bring in additional libraries targeted towards relevant problem domains. ActiveState does do that for Windows by bringing in additional modules like pywin32 and various others, but (as far as I am aware) we don't have anyone from there active on distutils-sig the way we do with Enthought, Continuum Analytics & the Linux distros. Assuming they have a booth at PyCon next month, I should probably swing by and ask the Komodo IDE folks about that...

[pfmoore]

One implication of Nick's (and Steve's) position seems to me that we don't view per-user installs as a key aspect of the python.org installers. And yet the impression I get of the direction that the 3.5 installers is taking seems to contradict that - there's a definite feel that we're expecting per-user installs to be more the norm.

I think that until viable distribution channels exist, we have to accept that almost all Windows users *need* the python.org installers, regardless of their intended usage.

On Windows, at the moment, conda/enthought are entirely viable distributions, but focused on data analysis. This does have downsides for other types of user - the conda package manager largely only covers data analysis tools, and while mixing pip and conda works, it's not a core use case for the distribution and there are rough edges. I've tried using Anaconda briefly for non-data science uses, and it's a somewhat frustrating experience - not one I'd be comfortable directing new non-scientific users at.

From checking the website, ActiveState free edition is for non-commercial use only, which likely precludes use by most of the people who want to use the python.org installer in their locked-down environments (if they can't get admin rights, they almost certainly won't be able to get an ActivePython license).

So, like it or not, until more generic free-for-commercual-use distributions of Python exist, I think we need to consider Windows users without admin rights as part of the core audience for the python.org installers.

Steve's changes to the 3.5 installer make this an entirely reasonable position. I see no need to provide backported solutions - it *is* reasonable for python.org to say "if you want improvements like this, you should choose the latest release".

Regarding the OP's issue, I think "fixed in Python 3.5" remains a fair answer. It's not a case of closing the issue without addressing the basic problem. It's more that it's fixed in the only version we'd ever fix it in (3.5). AIUI, the only thing that may still need admin rights is installing the required C runtime, which as a system component *has* to be done by an admin. That's a Windows issue, though, not a Python one.

[zooba]

Paul has basically summed up the pragmatism beats purity side of the argument. Whether we like it or not, users are mostly coming to python.org for their installer, and we need to support that.

That said, we can do some things to support all three cases once we acknowledge that they exist and identify who they're for. For example, an all users install could become an advanced option, which will move most users into per-user installs. We could also release a plain zip file of the installed layout and clearly mark it as being for app bundles and not a portable install for users. There are changes to getpath and the registry lookup that will better serve non-system installs, though we really need PEP-432 for that to be feasible.

Some bigger changes I wouldn't mind would be renaming python.exe to python35.exe and then py.exe to python.exe, which would let users have a system Python that interacts well with registered per-user versions and shebang lines. These largely depend on whether we've got the stomach to be so dramatic :) With 3.5 beta coming up so soon, we may be better to avoid rocking the boat now and look towards 3.6, but maybe this is more urgent than that?

[83d2e70e-e599-4a04-b820-3814bbdb9bef]

python35.exe follows things like pip so +1. py.exe to python.exe -1 from me, how about pylaunch.exe as it's explicit? As for urgency in the Python world Windows is and always has been the poor relation compared to *nix, so I say let's bite the bullet and get on with it.

[pfmoore]

Personally, I'd like to have 3.5 be the release that changes to using "Program Files" as the default install, and offers a per-user install to Appdata. I suspect there will be enough fallout from that change to keep us busy. Let's look to 3.6 for major renamings of the executables (if ever).

It should be noted that making python.exe be the launcher could potentially break a lot of things pretty horribly - virtualenv and the launcher itself are the ones that immediately come to mind.

Having said that, I have no objection to having both python35.exe and python.exe, but for 3.5 they should both be the same actual executable. In practice I doubt Windows users will use python35.exe, as there's no tradition of versioned executables there.

[pfmoore]

Regarding the "poor relation" argument, I'd see that as the other way round. We don't have the resources to deal with major breakages, so we should be relatively cautious.

[83d2e70e-e599-4a04-b820-3814bbdb9bef]

I'm concerned about being overcautious so that nothing ever happens. Do something, break it, fix it. If you delay all you do is put off the fix. Plus I've every confidence in our Windows developers to just do the right thing.

[zooba]

Yeah, some of those are fairly ambitious, but at the same time, for the SOE use case we probably don't want the Program Files install at all - it would be a mix between System32 and CommonFiles, with the same environment issues you'd see on Linux. Program Files is better than SystemRoot, but doesn't really make sense for any of the three use cases.

When we can describe exactly who our installer is meant for, we'll be able to figure out what changes are needed.

[pfmoore]

Sorry, SOE?

[zooba]

Sorry, Standard Operating Environment. I intended it as shorthand for the "as if it were shipped with the OS" case.

[ncoghlan]

I agree the migration to Program Files & a different installer build process is more than ambitious enough for 3.5.

For the embedding case, it's not that it *can't* be done, it's just a PITA.

Perhaps for the per-user no-admin-rights needed case, we could direct folks to the big http://portablepython.com/ bundle as a vendor neutral all inclusive bulk distribution?

To be completely clear, I'm talking "invite Perica Zivkovic to become part of CPython release management and include Portable Python release timelines in the CPython release PEPs" levels of engagement. "Is Portable Python fully operational yet?" could become one of the key gating criteria for leaving the beta phase and entering the release candidate phase.

[ncoghlan]

Along those lines, another option to consider would be offering to publish Portable Python from the python.org release pages in addition to publishing the less comprehensive CPython-only installers.

[ncoghlan]

Larry,

Bringing you into this discussion as CPython 3.5 release manager - we're trying to figure out what role we'd like the CPython Windows installers to play in the Windows distribution ecosystem, and I raised the question of whether or not typical Windows based end users might be better off with a larger, more comprehensive bundle like Portable Python (which they can also run from a USB stick if they want to).

There's no specific urgency here - I just wanted to give you a heads up that the discussion was happening. It's not clear to me yet if this will go anywhere (if nothing else, even if we decided it was a plausible idea, it would still be conditional on someone offering to contact Perica Zivkovic to see if they were amenable to the concept, as well as someone offering to write the suggestion up as a full PEP and work through the necessary changes to PEP-101 release management), but I figured you should at least be aware of the possibility as RM.

[zooba]

Also, I do have this topic on the language summit agenda, so I don't really want to lock anything down before then anyway, but this discussion has been good to flesh out some of the background and possibilities in advance of that.

[ned-deily]

Bringing an umbrella distribution into the CPython release process seems to me like a very big leap and would that requires very careful consideration. It would introduce a whole load of other dependencies into our release process, i.e. that the third-party packages (e.g. numpy, et al) included in Portable Python would become gating factors and dependencies for CPython releases. Frankly, I don't thing we want to go there. I don't have an objection to referring to users to other distributions like Portable Python (or Anaconda or whatever) but actually bringing it into the release process would need a *lot* of careful thought and planning and co-ordination. Sounds like an interesting discussion for the language summit.

[340ae65e-9726-4646-a1b5-7bce30dece82]

WinPython and miniconda are more current distros than portable python, and they come in both 32 & 64bit flavors.

Portable python hasn't been updated recently and only offers 32 bit which is IMO worthless except for the bundle as app case, eg meld installer.

I was going to suggest the same - if python.org can't support local installs (sans admin rights) b/c people will always look to it for direction, python.org should offer links to endorsed windows installer. I don't love it, but it will be familiar to windows users braving the open source world, similar to ruby which recommends winruby. Any endorsed distro **must** be built with msvc, so that all packages with extensions can be built. I don't care to support packages that depend on autotools.

Why can't msvcrt90.dll amd python27.dll be bundled into install directory? Seems to solve case #1 (for personal use) and #2 (bundled as app)? No admin rights required, just unzip. Why would you want to move python to program files?

Case #3 would be swell, would windows ever do that? If we assume windows distros are akin to *nix them perhaps we need to push the --user or --home=~/.local options and virtualenv. That seems to be the way conda is going. Ruby rvm is more like that as well. Especially if we are going to force users to start using 3.5. What is Microsoft's official position on python. are they officially supporting python on windows? Seems like it with newest April's and vcforpython27.

BTW miniconda does not install all of the sci-data packages, only conda and python 2.7. works with pip and --home=~/.local scheme. Is this the future for windows users? We should reach a consensus I think

[pfmoore]

I agree with Ned, this sounds like a significant change. In particular, Portable Python seems to currently only offer 3.2.5 at the moment. And it's not at all clear to me whether it's a 32-bit or a 64-bit version (but I suspect the former). One thing I'd want to understand would be why 3.4.3 and 64-bit versions weren't available. Is it just manpower (and if so, what manpower commitments would be needed from python-dev) or are there more fundamental issues with the Portable Python stack holding up the move to the latest version?

I'm also concerned that we should ensure that any distribution we bless is compatible with pip and packages installed on PyPI. I would be very concerned, for example, if we were moving towards a situation where wheels for Windows were *not* usable by the average user. (That specifically means that all commonly used distributions used the same CRT as the python.org builds, that distributions we recommend play well with pip, etc).

None of this is intended as a criticism of any of the distributions. I just think that actually *recommending* them in place of the python.org installers implies a certain level of assurance from python-dev - and we don't have any process in place to actually validate the distributions.

[zooba]

miniconda does not install all of the sci-data packages, only conda and python 2.7

miniconda is actually a tool that gives you the "conda" command, which can be used to install Python 2.6, 2.7, 3.3 or 3.4, 32 or 64-bit and a huge range of compatible packages. When you install it you also get a usable version of Python, but you don't have to use it. Anaconda is exactly the same thing but with a selection of packages predownloaded and installed - again, you can use the conda command to install more.

What is Microsoft's official position on python. are they officially supporting python on windows?

Microsoft doesn't distribute Python as part of Windows or provide any support for users, though we do provide some engineering support (i.e. me) and some low-cost efforts (such as the compiler pack, again, mostly me). There's a really high bar for "supporting" a product, and especially for shipping it to users who have not opted into it, as well as legitimate concerns about documentation/support, entry points, threat vectors, updates, patching, and probably more (when does Windows switch to 3.6? Honestly, it can probably never happen...). While it may happen eventually, I think we're better off having an installer that any sysadmin can use to make it as-if it were shipped.

The problem with that is then you can't use that installer for your app, for the same reason that a Linux application can't arbitrarily replace the system Python. You can't even ask your users to install it, since they may not have permissions (ignoring the CRT update issue for now - that will be resolved in time). This is where we need a way for a user to install Python-as-an-app (one install-per version-per-user) and also for an installer to include Python-as-a-library (one install-per app).

It may just be that we need three separate installers:

• EXE launcher for Python-as-an-app installations
• single MSI with no UI for as-if-shipped installations
• Merge module or ZIP file for Python-as-a-library installations (also note that the third option really needs changes/fixes to pythonXY.dll to work well)

If we went this way, I'd really want the EXE launcher to be the only one with a 'friendly' UI - the other two are specifically for people who know how to read documentation and know why they're getting them (especially since I assume I'd be the one writing it all). The EXE launcher could fairly trivially trigger or acquire either of the other two, so it could still be the main entry point, but I would seriously want to demote anything other than the per-user install.

[zooba]

(when does Windows switch to 3.6? Honestly, it can probably never happen...).

On re-read, this isn't quite clear:

Hypothetically, if Windows 10 included Python 3.5, when would it be upgraded to Python 3.6? Probably never. Back-compat is problematic on Linux, but are significantly multiplied by the size of and the relative lack of technical competence/interest of the Windows user base.

[ncoghlan]

As of last month, Microsoft do provide official Python support, but only in the context of the online Azure Machine Learning Environment: http://blogs.technet.com/b/machinelearning/archive/2015/02/18/announcing-the-general-availability-of-azure-machine-learning.aspx

I only discovered that this morning myself when Doug Napoleone tweeted a link to the related post from Continuum Analytics: http://continuum.io/blog/azureml

I was already aware of another Strata "combined solution" announcement, which covered deploying Anaconda Cluster in combination with Red Hat Storage for distributed data access from PySpark analysis nodes: http://redhatstorage.redhat.com/2015/02/17/deploying-pyspark-on-red-hat-storage-glusterfs/

So if we're after a recommendation for end users that has strong multi-vendor backing, then Anaconda from Continuum Analytics would definitely be the way to go. It's also already the preferred choice of the Software Carpentry community: http://software-carpentry.org/v5/setup.html (and the reasons are the same as those being discussed here: unlike the default CPython installers, Anaconda is specifically designed to tolerate being installed without administrative access to your system)

(I'd previously discounted Anaconda as a default upstream recommendation for per-user installs on Windows as I thought it was missing Windows-specific utility libraries like pywin32, but I was simply wrong about that: http://continuum.io/press/anaconda-1-6-released)

As Steve noted, providing Python *in the OS* for third party use poses major upgrade challenges. For Fedora, Python gets rebased in each major release (so every 6 months or so) which is easily fast enough to keep up with the general pace of CPython releases (Ubuntu tend to do the same thing), but also means system administrators and other folks using the system Python need to be prepared to keep pace with that upgrade cycle.

Things get far more difficult with the long term support releases like Debian Stable, Ubuntu LTS, RHEL, CentOS and SLES, or the custom Linux distributions used inside organisations like Google, Facebook and Amazon. *There* the system Python has historically acted as a long term brake on Python's evolution, as third party projects are often reluctant to drop support for older Python versions until newer Python versions are readily available in these long term support releases.

In July, we'll be coming up on 5 years since Python 2.7 was released, yet a great many third party Python projects still require that contributions retain compatibility with Python 2.6.

We see similar patterns occurring if we look back at earlier Python 2.x releases. As part of explaining the limited utility of a Python 2.8 release in facilitating the Python 3 transition (see http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html#wouldn-t-a-python-2-8-release-help-ease-the-transition for more on that) I pulled together the dates from various Python 2.x releases to when Twisted dropped the preceding release as a compatibility requirement:

• CPython 2.4 released -> 2.3 support dropped in Twisted: \~6 years
• CPython 2.5 released -> 2.4 support dropped in Twisted: \~4 years (Windows), \~6 years (non-Windows)
• CPython 2.6 released -> 2.5 support dropped in Twisted: \~4 years

That's a broken cycle, and one we need to help fix on the distro side by getting people to migrate away from using the stable system Python (which won't be rebased for the entire lifecycle of a major release) and into more regularly updated environments - upstream can't do that for us, we have to offer the appropriate tools downstream and actively encourage users to leave the system Python to the task of running the system, rather than arbitrary third party code.

The 10 year supported lifespan of RHEL & CentOS, together with the 3-4 year gaps between new major releases is one of the worst offenders on that front (not just for Python, but for other language runtimes, database runtimes, web servers etc), hence initiatives like the introduction of Software Collections (both as a commercially supported Red Hat product and as an upstream project hosted at softwarecollections.org). Using an SCL instead of the system Python lets an application developer more easily upgrade at their own pace (whether faster on RHEL/CentOS or slower on Fedora)

[ncoghlan]

Given the expansion of the discussion to more general questions of getting CPython from upstream into the hands of end users, I've added in some more of the distro level folks to the nosy list (Matthias, Barry, Slavek, Robert).

This idea of more actively pursuing well-structured channel based resdistribution for Python as a whole is one I've had on my mind for a while. If you have the time to watch it, my PyCon New Zealand keynote from last year gives a good overview of my own perspective on the topic: https://www.youtube.com/watch?v=H2ybMrFNku0

[ned-deily]

This issue seems to be expanding widely in scope and it is not at all clear to me what actionable items would come out of it. It started as a very Window-specific problem and now seems to somehow be being extended to the whole Python ecosphere. What about the OS X world, arguably one of the most widely-used laptop development environment these days? Apple distributes system Pythons with OS X. There are the python.org installers for OS X. There are third-party suppliers of popular binary wheels and or installers for important packages (e.g. NumPy, pandas, matplotlib) for one or both of the above. There are very popular third-party general open-source package distributors for OS X: Homwbrew and MacPorts, to a lesser extent, Fink. All of them distribute Pythons and many Python packages. Then there are the cross-platform distributors, like Anaconda, Enthought, ActiveState, etc. Should they all be part of this discussion? What about other Posix-y platforms, like FreeBSD? Where do you see this discussion going and is this a good forum for it?

[83d2e70e-e599-4a04-b820-3814bbdb9bef]

No this is not a good forum for this discussion. Python-dev or ideas please, to be followed up by a PEP I'd guess.

[zooba]

Yeah, this is starting to get out of hand. I'm quite happy to leave things sit until the language summit, when we should have a broad enough representation to make a start. After that it'll get to python-dev before anything major is decided, but the brainstorming phase will be more effective in person (sorry to anyone who won't be there - feel free to email me directly if you want to provide your perspective in advance: steve.dower at microsoft dot com).

[ncoghlan]

Right, I mostly borrowed the issue as a public channel that could reach folks already invested in the problem space, without needing to tidy things up sufficiently to make them coherent for a more general audience on python-dev or python-ideas.

As Steve says, we really need some high bandwidth discussions to help define what we think "good" looks like. I have a pretty solid idea of what *I* think "good" looks like, but that doesn't mean that aligns with what others think.

If we can get to a point where Steve, Slavek, Barry, Matthias and I can agree on a general outline of how we'd like platform integration to work, and whatever we propose sounds plausible to the Mac OS X experts as well, we'll be in a much better position to present a coherent story both to end users *and* to our fellow developers :)

[pfmoore]

That sounds reasonable. I understand the need for face-to-face discussion on something like this, although I do think there's been something of a tendency in the past for things "decided at PyCon" to receive a certain amount of resistance from people who weren't there, so I'd suggest posting a summary of what you're intending to discuss in advance, to get some input. I'll try to email my thoughts to Steve, to give him my perspective on the Windows side of things.

[ncoghlan]

I think we learned our lesson re making decisions at the language summit (rather than focusing on bringing folks up to speed on a problem area) after needing to later reverse engineer the rationale for some of the decisions made at the first couple :)

[20b5009a-ffb5-41a6-af53-2541440de0f6]

As the originator, I've noticed that the discussion (while illuminating - and a lot more than I expected!) seems to have gone off on several tangents. I'd like to remind everyone that this request is for an unrestricted Windows install (e.g. "binary") package of standard/basic python - nothing more.
Not to dampen the discussion but I see side issues such as impact on other O/S's, use of other (non-CPython) distributions, etc. as being outside the scope of the request. While side issues may need to be considered while working on the request they should be raised as separate issues, not handled under this one. Also, issues related to how this request affects specific "plug ins", e.g. numpy, also seem to be out of scope for this request (since users add them after successful installation) and should be handled in a similar matter.

My view anyway, YMMV

[bitdancer]

I just reviewed this whole issue. Steve says that the original issue you raised will be resolved in 3.5 (I believe pretty much already has been?). So given the last couple of messages I'm going to re-close this, as a continuing discussion here seems to be contra-indicated :)

[340ae65e-9726-4646-a1b5-7bce30dece82]

J. Morton and anyone else needing a Python-2.7.9 Windows release to use without admin rights or to embed in a personal application can see my blog to roll there own or download one of the versions from my dropbox

http://poquitopicante.blogspot.com/2015/03/building-python-x64-on-windows-7-with.html

https://www.dropbox.com/s/wc2x2or2477f4as/Python27-x86_asbuilt.zip?dl=0

https://www.dropbox.com/s/bcmh11kha9go8t6/Python27-x64_asbuilt.zip?dl=0

[340ae65e-9726-4646-a1b5-7bce30dece82]

I've set up AppVeyor CI (http://www.appveyor.com/) to build the latest tag in the 2.7 branch of cpython at https://hg.python.org/ and to deploy zip files of x86 and x64 standalone builds to http://breakingbytes.alwaysdata.net/PythonBootstrap/. The builds use a patched version of Tools/buildbot/external[-amd64].bat and PCbuild/batch.bat[ -p x64] with Windows 7 SDK (VS2008) and pass all rt.bat -q[ -x64] tests. The zipped file structure matches the standard release structure except that I didn't copy any of the Tools folders (pynche, i18n, ...) or Demos, and I put html files in the Doc folder instead of python27.chm. To make it standalone I put the MSVCR90 redistributables in the top level side by side (privately) with python27.dll, which is acceptable according to the VS2008 license (http://download.microsoft.com/documents/useterms/Visual%20Studio_2008%20Professional%20Edition_English_473a7e16-65dc-4cfb-8f44-ebdd93cb1d3d.pdf) as long as the distro has any LICENSE. It is also one of the 3 recommended practices for deploying Visual C++ applications (https://msdn.microsoft.com/en-us/library/zebw5zk9(v=vs.90).aspx) especially Redistributiong Visual C++ Files (https://msdn.microsoft.com/en-us/library/ms235299(v=vs.90).aspx). The current Python uses the merge modules (https://msdn.microsoft.com/en-us/library/ms235290(v=vs.90).aspx) which requires admin rights.

"Installation of the WinSxS folder requires administrative user rights. If an installation is run by a user who does not have administrative rights, the Visual C++ assemblies cannot be installed, and applications that depend on those DLLs cannot run. The alternative redistribution approach is to install private side-by-side assemblies of a particular user application. For more information on deploying Visual C++ files as private assemblies see Redistributing Visual C++ Files."

In addition to some glue script, I added an altered version of idle.bat for Scripts that points to pythonw.exe and Lib\idlelib\idle.pyw from the Scripts folder instead of from the Lib\idlelib folder. This lets users start idle if Scripts is on their path.

The glue for the appveyor build and the webpage are both maintained on bitbucket.

• https://bitbucket.org/breakingbytes/cpython/branch/PythonBootstrap
• https://bitbucket.org/breakingbytes/breakingbytes.bitbucket.org

I expect this release to have several major uses:

• It can be used to install python-2.7 without admin rights on windows
• It can be used to embed python-2.7 into standalone applications that depend on python since it has a fairly small footprint, 15MB compressed.
• To use 32-bit and 64-bit versions of Python on the same machine, which is technically not impossible now, and would still require some managment

Some future goals:

• create a self extracting executable that performs some minor niceties such as ...
  • adding Python and scripts to the path
  • fixing any hardcoded paths in bundled executbles
  • associated .py with python.exe
  • running python -m ensurepip
  • getting some convenient packages for scidata like pandas, numpy, xlrd, etc.