Support for Python 3

[cclauss]

See ~#1150~, ~#1335~, ~#1336~, and ~https://travis-ci.com/nodejs/node/builds/79706150~ (~nodejs/node#21942~)

• Node Version:
• Platform:
• Compiler:
• Module:

Verbose output (from npm or node-gyp):

``` ```

[FranklinYu]

Where there are effective JavaScript solutions, we should transition to those.

@cclauss I prefer a pure JavaScript solution, and I would like to help. Is there already an issue discussing the possibility of switching to pure-JavaScript compiling solutions? I guess it would be off-topic to discuss it here.

[mhdawson]

@cclauss do you think this is being progressed under https://github.com/nodejs/TSC/issues/642 or is there something we need to action in this issue?

[cclauss]

@mhdawson I would love to see progress on #1336 ... It seems so simple but it has been a loong time.

https://github.com/nodejs/node-gyp/issues/1337#issuecomment-448581154 is still our running list... Some progress has been made but still several items are open awaiting review. More active reviewers would be a beautiful thing. Also, we are going to need build machines to have Python 3 loaded on them in addition to Python 2 so that we can be sure that tests pass on both versions of Python.

[vogt31337]

I don't quite understand the need for Python 2.7 (also it is clocking down less than a year). Wouldn't it be possible to write the python code so that a python 3 interpreter is happy? I've written such code in the past which would run on both py2 and py3. And maybe this could be used for some simple use cases...

[cclauss]

@vogt31337 It is a BIG codebase. See https://github.com/refack/GYP to get a sense of how many changes were required to get it compatible with Python 3.

[megakoresh]

@vogt31337 It is a BIG codebase. See https://github.com/refack/GYP to get a sense of how many changes were required to get it compatible with Python 3.

I hope we can finally do away with Python 2, I am sure node-gyp is for many the last blocker remaining.

Although I do not personally quite understand why node.js really needs this wrapper around the build tool instead of letting module developers take care of it (you have to install native libraries for many native modules anyway, so its not like node-gyp and gyp itself solve the "per-library requirement" problem), but that's a different story.

[iMichka]

Hi. Homebrew maintainer here.

I just wanted to notify the node maintainers that we started the discussions around the Python 2 removal from the Homebrew package manager. Node is one of the last “big” packages that does not support Python 3. I would like to avoid a situation where we would need to remove node from brew. Node is in our top 3 of installed packages, with more than 4 million install events per year.

Right now we are just gathering opinions https://discourse.brew.sh/t/python-2-eol-2020/4647 No decision has been taken yet.

Options go from plain removal, additional grace period of a few months, moving node to an external tap outside of our org, or any other idea that will come up in the next weeks.

[rvagg]

Thanks @iMichka, we have a team that's trying hard to bring Node up to Python 3 support. Unfortunately we missed out on the Node 12.0.0 window but we're acutely aware of the time pressures of continuing our Python 2 requirement. @cclauss @refack @thefourtheye and others are working hard addressing our sprawling use of Python. I'm not sure we have a firm timeframe but this is a well known problem and we all feel the pressure (and bump up against it regularly!). I would hope that we can have it resolved within the next couple of months but I can't speak with any authority on that.

[cclauss]

moving node to an external tap outside of our org

@iMichka Given that soon-to-be-released Node.js 12 will still require legacy Python, can you please provide us with a sense of the degree of difficulty required to set up and maintain an external tap? Are there other examples of external taps that your team considers to be well set up and maintained?

[scpeters]

I'm a homebrew maintainer, and I also maintain an external tap with bottles from private CI machines. It's super easy to make a tap and push formula updates to it. In my opinion, the bulk of the work is in getting bottles to build successfully and fixing inter-package compatibility problems. One hopefully useful option will be building homebrew bottles with something like Azure pipelines, as discussed in the following post:

• https://discourse.brew.sh/t/tap-bottles-how-to-publish-after-ci-upload-to-bintray/4674

[ladislas]

@cclauss to bounce back on what @scpeters was saying, setting up and maintaining a tap is pretty straight forward and easy.

The bottle part is a little trickier but I'm close to a fully working solution that would make it much easier for tap maintainer to distribute bottles for their taps.

I'll post here when it's ready.

[refack]

I'm commented in several other thread that one limiting factor for getting full Python3 compatibility for GYP3, and hence node-gyp, is having a proper CI testing setup. I've submitted https://github.com/nodejs/build/issues/1789 as for me setting up CI testing on the Node.js cluster is the lowest hanging fruit.

[iMichka]

a sense of the degree of difficulty required to **set up and maintain an _external

It is doable. The issue will be more about visibility. More than 4 million install events in a year through homebrew will mean that we need to communicate to all these people about the node removal from homebrew-core, and give instructions on how to tap the new node tap. All the install scripts from the whole industry will probably need an updated too. There will be no automatic redirection to an external tap.

I do not want to pollute this thread more with homebrew discussions though. We are not the only ones to ship node, so the issue is more global than this. Not sure this issue is the best place to discuss all this. We have not taken any decision right now, my initial message was more a gentle nudge to warn you that bigger changes may come :)

[AdamMajer]

We are not the only ones to ship node, so the issue is more global than this.

Indeed, it's much more global than this. There has been discussion on removing python2 from openSUSE Tumbleweed distribution at beginning of next year. For those that don't know, Tumbleweed is the rolling release distribution of openSUSE project.

https://lists.opensuse.org/opensuse-factory/2019-04/msg00229.html

on the bright fresh chilly morning of 2020-01-02 we are going to remove python2 package (that's python 2.7.* interpreter) from Factory.

[mingrammer]

Is this in progress?

[cclauss]

@mingrammer https://github.com/nodejs/node-gyp/pull/1723#issuecomment-498666816 lists some current Python 3 incompatibilities if you have some time to work on this. We also created GYP3 https://github.com/nodejs/node/pull/26620

[MattIPv4]

Would it be possible to edit the opening post in this thread with a general summary of the situation with regards to the move to Python 3 due to the Python 2 EOL and drop from Brew/macOS/Linux distros? It'd be great if that could then be kept up to date so the average user can easily see an overview of the current situation and the plan going forward :)

[cclauss]

Closing this PR as GYP seems to be Python 3 compatible:

• node/node-gyp Travis CI tests are run and pass on Linux, macOS. Help needed on Windows #1801
• GYP3 https://github.com/refack/GYP is also of interest and supports Python 3
• Chromium GYP is also making Python3 progress https://chromium.googlesource.com/external/gyp

If you find any Python 3 incompatibles in this code, please open an issue.

NOTE: nodejs/node is not yet compatible with Python 3 but an effort is ongoing there.

[superern]

Python 2 is starting to get removed from distributions at this point.

Correct e.g. brew is now installing/upgrading to Python 3 by default.

The following just worked for me as I already had a python2.7 executable installed via brew:

npm config set python python2.7

However, if you don't have a python2.7 executable, I suspect you can do something like:

brew install python@2
# follow the instructions printed after installing `python@2` to get python 2 on your PATH 
# e.g. echo 'export PATH="/usr/local/opt/python@2/bin:$PATH"' >> ~/.bashrc
npm config set python python2.7

Thanks! this resolves my issues

[slim-hmidi]

I'm using Node.js v 12.18.2 and Python v3.8.1 on windows 10 and when I tried to run npm install I got this error:

gyp ERR! configure error
gyp ERR! stack Error: Command failed: C:\Program Files (x86)\Python\Python\python.EXE -c import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack   File "<string>", line 1
gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack                       ^
gyp ERR! stack SyntaxError: invalid syntax
gyp ERR! stack
gyp ERR! stack     at ChildProcess.exithandler (child_process.js:303:12)
gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
gyp ERR! stack     at maybeClose (internal/child_process.js:1021:16)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\x199377\\AppData\\Local\\Programs\\Git\\react-whiteapp\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild" "--verbose" "--libsass_ext=" "--libsass_cflags=" "--libsass_ldflags=" "--libsass_library="
gyp ERR! cwd C:\Users\x199377\AppData\Local\Programs\Git\react-whiteapp\node_modules\node-sass
gyp ERR! node -v v12.18.2
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
Build failed with error code: 1

[cclauss]

gyp ERR! node-gyp -v v3.8.0

Please upgrade node-gyp to v7 https://github.com/nodejs/node-gyp/releases

[richardlau]

@slim-hmidi If you want to use Python 3 you'll have to take up with node-sass as they're still depending on node-gyp ^3.8.0: https://github.com/sass/node-sass/blob/e1fc1580c2e979113c95850b8ae5d83845063ca5/package.json#L67

node-gyp added Python 3 support in the 5.x and later releases.

[slim-hmidi]

@richardlau I installed python v2.7.16 but still the same problem.

[richardlau]

If you have the same problem it means that node-gyp is still trying to use Python 3. Maybe try setting the PYTHON environment variable to point to your installed Python 2?

[slim-hmidi]

I uninstalled python 3 and python 2.7.16 and I'm reinstalling python 2.7.16 again and I will try again.

[slim-hmidi]

@richardlau it works now. thanks