python ssl support

[xzcvczx]

There is no ssl support in python

import ssl
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Applications/kicad.app/Contents/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 98, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: No module named _ssl

(not that i need it at this point more just an fyi)

[adamwolf]

This may actually be a huge amount of work.

I started working on this, assuming I just needed to add open or libreSSL, point to them during compilation, and then everything will work.

First, when I install a modern openSSL, Python 2.7.15 doesn't work since can't find RAND_egd symbol so it chokes--this may be because openSSL is too new here. Python supposedly supports libreSSL, so install that, and I notice that Python is hardcoded to look at certain directories for headers (look for search_for_ssl_incs_in in setup.py in the Python source.) I sigh, and I look into modifying this during build to point to where we are including it, and I realize that Python isn't pulling these in, but rather linking to them--which means I need to figure out how to pull these things directly into the bundle and get Python to look inside the bundle first.

Python has figured it out:

Attention macOS users: as of 2.7.15, all python.org macOS installers ship with a builtin copy of OpenSSL. Additionally, there is a new additional installer variant for macOS 10.9+ that includes a built-in version of Tcl/Tk 8.6. See the installer README for more information.

I need to look at the installer README to see if they discuss how they did it.

[xzcvczx]

just be wary of the python.org bundles, the ones i have looked at will only work at /Library/Frameworks, and a fair bit of effort will likely be needed to move them from that location, including keeping openssl with details as to where to find openssl.cnf and its certs.

[adamwolf]

Yes. We cannot do that.

There was really a huge amount of work to generate what we need to do this--a relocatable python installation inside a bundle.

I found an example of a person who got python to link in a static openssl library. That could work, but has the downside of being statically linked. Being that we aren't a server, that might be sufficient? Thoughts?

On Mon, Oct 29, 2018 at 10:59 AM xzcvczx notifications@github.com wrote:

just be wary of the python.org bundles, the ones i have looked at will only work at /Library/Frameworks, and a fair bit of effort will likely be needed to move them from that location, including keeping openssl with details as to where to find openssl.cnf and its certs.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/KiCad/kicad-mac-builder/issues/226#issuecomment-433965486, or mute the thread https://github.com/notifications/unsubscribe-auth/AACLYZgkm66A1HfaDPvW-NQtmrThE39xks5upyXcgaJpZM4X7h8C .

[xzcvczx]

well the python.org .frameworks can be used to see how they do openssl dynamically so static or dynamic don't really cause an issue, but yeah will still be cert/conf issue

[adamwolf]

Oh my! this is a project that takes those python.org frameworks and makes them relocatable. It's 3 months old... I spent more than 40 hours debugging it doing it from scratch!

https://github.com/gregneagle/relocatable-python

I'm going to test this. This might solve this problem, the xml problem, and make it easier to go to Python 3, all in one fell swoop.

[xzcvczx]

as far as i can see (please correct me if i am wrong) this will still leave openssl looking at /Library/Frameworks/......... for the ssl certificates. so while that can be a start something will still be needed for the env vars, although i guess they could be set using info.plist in the .app

[adamwolf]

No, that's correct, but this should get us pretty close.

On Tue, Oct 30, 2018 at 5:45 AM xzcvczx notifications@github.com wrote:

as far as i can see (please correct me if i am wrong) this will still leave openssl looking at /Library/Frameworks/......... for the ssl certificates. so while that can be a start something will still be needed for the env vars, although i guess they could be set using info.plist in the .app

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/KiCad/kicad-mac-builder/issues/226#issuecomment-434252820, or mute the thread https://github.com/notifications/unsubscribe-auth/AACLYXKsTZ8w4qmjFEO-LlBvDrxiSFEjks5uqC2wgaJpZM4X7h8C .

[adamwolf]

I think I may have this, actually. installing the python package certifi may set it up so that it uses those as the root certs, and that can move around in site-packages just fine. If the build works, I'll merge it into nightlies, and maybe you can confirm the roots are working fine for you?

[xzcvczx]

import ssl
ssl.get_default_verify_paths()

is a nice easy way to confirm ssl paths.

another thing to maybe be wary of (although it may not ever affect kicad) there is no Current symlink in 3.6.7 Python.framework from python.org and if its meant to be made in a script then the script is not run and the symlnk is not made, so might pay to check that the symlink exists

Python.framework/Versions/Current -> x.y (2.7 in kicad's case most likely, but Python.framework/Python is a symlink to Python.framework/Versions/Current/Python so it breaks that as well as all other symlinks in the root of the framework)

[adamwolf]

I'll add both of those checks, but I actually got my self-compiled Python to work with SSL, I think. We'll see if it makes sense to continue that onto Python 3, but maybe?

On Wed, Oct 31, 2018 at 2:20 AM xzcvczx notifications@github.com wrote:

import ssl ssl.get_default_verify_paths()

is a nice easy way to confirm ssl paths.

another thing to maybe be wary of (although it may not ever affect kicad) there is no Current symlink in 3.6.7 Python.framework from python.org and if its meant to be made in a script then the script is not run and the symlnk is not made, so might pay to check that the symlink exists

Python.framework/Versions/Current -> x.y (2.7 in kicad's case most likely, but Python.framework/Python is a symlink to Python.framework/Versions/Current/Python so it breaks that as well as all other symlinks in the root of the framework)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/KiCad/kicad-mac-builder/issues/226#issuecomment-434586033, or mute the thread https://github.com/notifications/unsubscribe-auth/AACLYTBUjusBS7c3j96j05Oh1Qy-Fm7Aks5uqU9PgaJpZM4X7h8C .

[xzcvczx]

if nothing else works can always just set it up to use /private/etc/ssl/* although i am not sure if this is intended to be done with custom openssl builds

[adamwolf]

I still need to get the ssl stuff importable after the bundling. I am working on that next.