ssl module can't be loaded in requests and other packages/can't be used in python early/SSLError no valid ssl module

[unsigned123]

the support for ssl on windows seems not so good and robust today i'm writing a submod with which i can upload my saves onto a webdav netdisk however,the requests module kept throwing SSLError,telling me that the ssl module couldn't be found damn it!i spent my whole night fixing this stupid bug and that's my solution 1.install Python 2.7 x86(that's important to install an x86 one instead of an x64 one) 2.copy its ssl.py from Lib directory to game/python-packages(somehow the ssl directory in the site-packages is a fake one) 3.copy _ssl.pyd from DLLs directory to lib/windowsi686 in ddlc's directory after finishing all these stupid steps,the SSLError finally disappeared (i pasted ssl-related files to every corner of the ddlc directory and it turned out to be a stupid x86/64 problem...i wasted my lovely night:( ) but why not put these two files simply into MAS itself?

[unsigned123]

if you skip step3,when you try to import ssl,an ImportError will be thrown indicating "_ssl" can't be imported,and this is caused by the pyd file i mean,now that ssl is supported,why can't it be further and better supported?

[unsigned123]

to reproduce the situation,you can use the "requests" lib,and any attempt to establish an https connection will fail you can also reproduce it like what i did,using "webdavclient2"(use pip install webdavclient2 and copy its code from the site-packages to ddlc's python-packages),follow its docs and you will find any attempt to connect to the server will fail.unfortunately,this package will not give the detailed reason why there is no connection,so you should edit its source code in "client.py" to let it throw the exception from requests.request directly instead of experiencing a try-except-re-raise process,in which a new exception containing no useful info will be raised.in other words,you should remove the try-except and you will see the SSLError

[dreamscached]

Take a look at how the Submod Updater works. It sends HTTPS requests to GitHub and it works flawlessly.

[unsigned123]

does it use requests or urllib3? i mean "import requests"

Take a look at how the Submod Updater works. It sends HTTPS requests to GitHub and it works flawlessly.

[dreamscached]

No, but it doesn't need it. Python has HTTP modules in stdlib, and since WebDAV uses HTTP you can work with it too. It also shows how to ship SSL libraries.

[Booplicate]

You're overcomplicating, MAS bundles ssl for a long time now, for example if you try to import ssl it should work. If it doesn't, then your install is invalid and missing some libs, reinstalling should help.

[unsigned123]

Yes,I can get your point.. In my case,import ssl shouldn't work.I will check my installation twice but i am curious about where is the ssl bundled in MAS...

You're overcomplicating, MAS bundles ssl for a long time now, for example if you try to import ssl it should work. If it doesn't, then your install is invalid and missing some libs, reinstalling should help.

[unsigned123]

You're overcomplicating, MAS bundles ssl for a long time now, for example if you try to import ssl it should work. If it doesn't, then your install is invalid and missing some libs, reinstalling should help.

oh I have seen your SSL... it's in Python-packages/ssl,you copied the ssl in site-packages but i wonder how the _ssl.pyd will be loaded… if you have a try by entering "import ssl" in your python IDLE,you can find that the real "ssl" loaded by python is Lib/ssl.py instead of Lib/site-packages/ssl/init.py,and in the ssl.py there is import _ssl and in this way _ssl.pyd,containing the core of ssl written in C,compiled into pyd,is loaded... if you look at theinit.py of the ssl directory in MAS,you will see it actually does nothing...(look at the comment left by the ssl's developer,it says # NOTE: this is a dummy package so consumers think we exist.) so if you use "import ssl" in MAS,surely it will work!!because it did nothing but if you run this command on python,it loads a totally different package after you open the ssl.py in Lib/,you will see what i am talking about... the urllib2 seems to be able to process some of the tasks without ssl?i guess.if you try to connect to a site with an invalid ssl signature,you will need ssl to ignore the invalid ssl signature.

[unsigned123]

the code in ssl/init.py `# NOTE: this is a dummy package so consumers think we exist.

import platform

if ( platform.system() in ("Windows", "Linux") or (platform.system() == "Darwin" and platform.machine() == "x86_64") # 64-bit mac ): pass else: raise ImportError` importing a package in python means loading the init.py but it did nothing.

[Booplicate]

In my case,import ssl shouldn't work.I will check my installation twice

It, in fact, should work. It's an issue if it does not.

For you it doesn't matter what's within ssl.py there. When you import ssl python will load the appropriate dll/so. If you imported ssl in MAS you'd see it links to win32_ssl.pyo (on windows) and _ssl links to _ssl.pyd. In python3 ssl links to ssl.py and _ssl links to _ssl.pyd. It's the same behaviour as in MAS. It's either your game installation is invalid or you didn't install requests correctly (which can be tricky in renpy).

[ThePotatoGuy]

when MAS loads, it checks the platform, loads the appropriate ssl library, replaces the "ssl" dummy package in sys.path for imports, and updates httplib's ref to ssl so it uses the real ssl module. This is done here

To use requests, the same ssl hack done to httplib would need to be done. so the first thing your submod should do after MAS's ssl hack runs is to point request's ssl to a freshly imported MAS ssl.

[Booplicate]

After one package imports ssl, it should be available for all other packages as well.

[ThePotatoGuy]

After one package imports ssl, it should be available for all other packages as well.

this is true after MAS does its hack, but anything that runs before will need the ref change like httplib. so if the import requests is in an earlier store, py early, or a separate py package (I think), then it needs a ref change.

[unsigned123]

when MAS loads, it checks the platform, loads the appropriate ssl library, replaces the "ssl" dummy package in sys.path for imports, and updates httplib's ref to ssl so it uses the real ssl module. This is done here

To use requests, the same ssl hack done to httplib would need to be done. so the first thing your submod should do after MAS's ssl hack runs is to point request's ssl to a freshly imported MAS ssl.

yes...i see what is happening now! my submod requires to use ssl in python early,because i want to download the persistent before renpy loads it(my submod can sync monika between devices with webdav) but why you hack in this way...it could be loaded in a normal way right?...😪a more graceful and pythonic way...

[ThePotatoGuy]

but why you hack in this way...it could be loaded in a normal way right?...😪a more graceful and pythonic way...

you answered this earlier:

1.install Python 2.7 x86(that's important to install an x86 one instead of an x64 one) 2.copy its ssl.py from Lib directory to game/python-packages(somehow the ssl directory in the site-packages is a fake one) 3.copy _ssl.pyd from DLLs directory to lib/windowsi686 in ddlc's directory

renpy 6 doesn't have ssl, and MAS is installed through the game/ directory. Crawling up to ddlc's lib/ dir doesn't always work due to permission issues, so the least-intrusive option for users is to keep the ssl package within game/. Anything in python-packages/, however, is not available until renpy starts loading game code (aka after python early, so the best thing this can do is load it up fairly early and update refs that other modules need.

[unsigned123]

but why you hack in this way...it could be loaded in a normal way right?...😪a more graceful and pythonic way...

you answered this earlier:

1.install Python 2.7 x86(that's important to install an x86 one instead of an x64 one) 2.copy its ssl.py from Lib directory to game/python-packages(somehow the ssl directory in the site-packages is a fake one) 3.copy _ssl.pyd from DLLs directory to lib/windowsi686 in ddlc's directory

renpy 6 doesn't have ssl, and MAS is installed through the game/ directory. Crawling up to ddlc's lib/ dir doesn't always work due to permission issues, so the least-intrusive option for users is to keep the ssl package within game/. Anything in python-packages/, however, is not available until renpy starts loading game code (aka after python early, so the best thing this can do is load it up fairly early and update refs that other modules need.

but in my case it worked in python early...any package/submod using ssl can import ssl properly after moving the file. if you don't move it to ddlc's lib,i think moving it to other places python can detect will help??(after all,import _ssl just forces python to search for _ssl.pyd,it doesn't care where it finds it)

[unsigned123]

but why you hack in this way...it could be loaded in a normal way right?...😪a more graceful and pythonic way...

you answered this earlier:

1.install Python 2.7 x86(that's important to install an x86 one instead of an x64 one) 2.copy its ssl.py from Lib directory to game/python-packages(somehow the ssl directory in the site-packages is a fake one) 3.copy _ssl.pyd from DLLs directory to lib/windowsi686 in ddlc's directory

renpy 6 doesn't have ssl, and MAS is installed through the game/ directory. Crawling up to ddlc's lib/ dir doesn't always work due to permission issues, so the least-intrusive option for users is to keep the ssl package within game/. Anything in python-packages/, however, is not available until renpy starts loading game code (aka after python early, so the best thing this can do is load it up fairly early and update refs that other modules need.

i will test it with the pyd in python-packages.i think the pyd should work in python-packages with the ssl.py in Python27/lib forcing to load the pyd

[ThePotatoGuy]

but why not just put the ssl.py in Python27/lib into game/python-packages and let python itself decice what to load...

i will test it with the pyd in python-packages.i think the pyd should work in python-packages with the ssl.py in Python27/lib forcing to load the pyd

I mentioned game/python-packages doesn't load until renpy starts loading game code so that wouldn't solve the py early issue.

that can avoid a lot of compatibility problems,and that's actually what python do when importing ssl,and it seemed ok on my computer

I'm not sure what compatibility problems you're talking about - the ssl libs MAS ships with are from renpy 7, which basically means ssl vetted to work with renpy for the platforms that renpy supports. And renpy's flavor of libs has different ssl modules based on the platform instead of a central ssl.py so MAS code just points ssl to the appropriate flavor.

i mean,the loading can be done in a very graceful way just with a simple file paste(actually,the ssl is storaged in more than one place,what i do is just to move them all to MAS),why do we hack in this way...

but in my case it worked in python early...any package/submod using ssl can import ssl properly after moving the file. if you don't move it to ddlc's lib,i think moving it to other places python can detect will help??(after all,import _ssl just forces python to search for _ssl.pyd,it doesn't care where it finds it)

Again, I mentioned we can't just copy stuff anywhere, especially outside of game/ because of permission issues. So anything involving lib folders wont work here.

in addition,my submod on an unofficial Android MAS version(i took part in the development,it didn't involve any ssl changes)runs miraculously smooth,that's interesting haha...

android probably has python ssl installed globally ?

[unsigned123]

but in my case,the SSLError did disappear after I moved the two files.does that mean,I can load something in python-packages before renpy loads it? ps:I think sync my persistent in python early is the only way I can be faster than renpy😂it's reasonable,right?😂😂 in addition,what I mean is to put the ssl.pyd into the python-packages folder,it isn't intrusive,right?😂and it should be effective...i guess

---- Replied Message ---- | From | @.> | | Date | 02/16/2024 07:33 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [Monika-After-Story/MonikaModDev] the ssl module can't be imported or detected properly (Issue #10201) |

but why not just put the ssl.py in Python27/lib into game/python-packages and let python itself decice what to load...

i will test it with the pyd in python-packages.i think the pyd should work in python-packages with the ssl.py in Python27/lib forcing to load the pyd

I mentioned game/python-packages doesn't load until renpy starts loading game code so that wouldn't solve the py early issue.

that can avoid a lot of compatibility problems,and that's actually what python do when importing ssl,and it seemed ok on my computer

I'm not sure what compatibility problems you're talking about - the ssl libs MAS ships with are from renpy 7, which basically means ssl vetted to work with renpy for the platforms that renpy supports. And renpy's flavor of libs has different ssl modules based on the platform instead of a central ssl.py so MAS code just points ssl to the appropriate flavor.

i mean,the loading can be done in a very graceful way just with a simple file paste(actually,the ssl is storaged in more than one place,what i do is just to move them all to MAS),why do we hack in this way...

but in my case it worked in python early...any package/submod using ssl can import ssl properly after moving the file. if you don't move it to ddlc's lib,i think moving it to other places python can detect will help??(after all,import _ssl just forces python to search for _ssl.pyd,it doesn't care where it finds it)

Again, I mentioned we can't just copy stuff anywhere, especially outside of game/ because of permission issues. So anything involving

in addition,my submod on an unofficial Android MAS version(i took part in the development,it didn't involve any ssl changes)runs miraculously smooth,that's interesting haha...

android probably has python ssl installed globally ?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[ThePotatoGuy]

but in my case,the SSLError did disappear after I moved the two files.does that mean,I can load something in python-packages before renpy loads it?

ok I stand corrected. you can load python-packages/ in py early (we do it in 0utils). at this point, I don't remember exactly why the imports code is in -1500 init level, but there's a few possibilities:

• allow for imports to set persistent values - py early doesn't have a loaded persistent
• allow for imports to use other mas tools - some utils/tools are made after py early
• something else that imports use is not in py early or wasnt there at the time

in addition,what I mean is to put the ssl.pyd into the python-packages folder,it isn't intrusive,right?😂and it should be effective...i guess

well that is what MAS does - we have the libs for each platform in python-packages.

[unsigned123]

-1500 level seems to run after python early...it's not early enough

yes,i know loading the pyd is exactly what MAS does,but it's too late for python early so i want to give a solution.

in my test,the libs for different platforms won't be loaded in python early without the forced import of "_ssl" in ssl.py in Python27/lib moving these two files into Python-packages and we 1.can load ssl normally afterwards,don't need to hack 2.perfectly solve the python early problem it's a flawless solution isn't it,solve two problems at one time

---- Replied Message ---- | From | @.> | | Date | 02/16/2024 08:23 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [Monika-After-Story/MonikaModDev] the ssl module can't be imported or detected properly (Issue #10201) |

but in my case,the SSLError did disappear after I moved the two files.does that mean,I can load something in python-packages before renpy loads it?

ok I stand corrected. you can load python-packages/ in py early (we do it in 0utils). at this point, I don't remember exactly why the imports code is in -1500 init level, but there's a few possibilities:

allow for imports to set persistent values - py early doesn't have a loaded persistent allow for imports to use other mas tools - some utils/tools are made after py early something else that imports use is not in py early or wasnt there at the time

in addition,what I mean is to put the ssl.pyd into the python-packages folder,it isn't intrusive,right?😂and it should be effective...i guess

well that is what MAS does - we have the libs for each platform in python-packages.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[unsigned123]

but in my case,the SSLError did disappear after I moved the two files.does that mean,I can load something in python-packages before renpy loads it?

ok I stand corrected. you can load python-packages/ in py early (we do it in 0utils). at this point, I don't remember exactly why the imports code is in -1500 init level, but there's a few possibilities:

• allow for imports to set persistent values - py early doesn't have a loaded persistent
• allow for imports to use other mas tools - some utils/tools are made after py early
• something else that imports use is not in py early or wasnt there at the time

in addition,what I mean is to put the ssl.pyd into the python-packages folder,it isn't intrusive,right?😂and it should be effective...i guess

well that is what MAS does - we have the libs for each platform in python-packages.

also,after my test,i found packages like requests still tries to load the un-redirected ssl.it seemd not to be influenced by the hacking process... because i found the uploading function of my submod on exit,later than ssl hack-loading throw the same SSLError if i don't move the files it's more than a python early problem,it seems that packages like requests cannot run either of course,my problem cannot be ruled out yet

[ThePotatoGuy]

moving these two files into Python-packages

which files? ssl.py and ssl.pyd? 2 problems:

1. this only accounts for windows
2. we're not going to ship python's ssl file in MAS - we don't need to. MAS includes the "ssl.py" equivalents as pyos, and all we have to do is point to the correct one.

in my test,those libs won't be loaded in python early without the ssl.py in Python27/lib

well of course it wont be loaded in python early because MAS import code is in -1500 init, I have mentioned this several times. the meat of MASImport_ssl.import_try could probably be moved earlier to be used earlier, though, so thats an option.

also,after my test,i found packages like requests still tries to load the un-redirected ssl.it seemd not to be influenced by the hacking process...

you need to force it to have the correct ssl ref by setting it manually, thats what MAS does with httplib. of course this doesn't play nice with modules that run stuff on import, so if requests is using ssl on import then you'd have to edit requests or just do web requests manually.

Regardless of all this, we're still planning on a move to renpy 8, which means native ssl, and therefore no need to include our own ssl lib. This means any major changes to the import code aren't worth it for a v12 release.

[unsigned123]

moving these two files into Python-packages

which files? ssl.py and ssl.pyd? 2 problems:

1. this only accounts for windows
2. we're not going to ship python's ssl file in MAS - we don't need to. MAS includes the "ssl.py" equivalents as pyos, and all we have to do is point to the correct one.

in my test,those libs won't be loaded in python early without the ssl.py in Python27/lib

well of course it wont be loaded in python early because MAS import code is in -1500 init, I have mentioned this several times. the meat of MASImport_ssl.import_try could probably be moved earlier to be used earlier, though, so thats an option.

also,after my test,i found packages like requests still tries to load the un-redirected ssl.it seemd not to be influenced by the hacking process...

you need to force it to have the correct ssl ref by setting it manually, thats what MAS does with httplib. of course this doesn't play nice with modules that run stuff on import, so if requests is using ssl on import then you'd have to edit requests or just do web requests manually.

Regardless of all this, we're still planning on a move to renpy 8, which means native ssl, and therefore no need to include our own ssl lib. This means any major changes to the import code aren't worth it for a v12 release.

Now,the whole problem is revealed.Thanks for your assistance,and it's my pleasure to discuss with our outstanding contributors!:) What you want to do is all up to you--I would just leave this issue here,and you may close it now. I was tortured by this strange problem for hours and I wondered whether there will be another person encountering exactly the same issue,because before discussing with you,I was completely ignorant about how mas deals with ssl module and I thought that would be the same for others???(no offense)I tried to ask the question in the community,but no one can give an exact answer.After all,the mechanism is a bit over-complicated. I am looking forward to your move to Ren'py 8,Python 2.7 has reached its end of life now and left with a lot of unfixed bugs(like my webdav,there is even a bug inside its module).Python 3 is the trend!Please quicken your pace!!! Finally,have a good day:) ps:I am working on a submod to sync persistent between different devices including different Windows PCs and Android(transplanted v0.12.5),and I found the affection would be cut a lot when putting a 0.12.5 save to 0.12.15,is it related to the 0.12.10 aff. update/do you have that kind of mechanism?if not,that would be my own problem...it seems a bit stupid to new an issue for a simple yes/no question...?

[ThePotatoGuy]

is it related to the 0.12.10 aff. update/do you have that kind of mechanis

Yes it very likely is. @Booplicate would know specifics but that update was a major change to how aff is stored.

[unsigned123]

is it related to the 0.12.10 aff. update/do you have that kind of mechanis

Yes it very likely is. @Booplicate would know specifics but that update was a major change to how aff is stored.

yes,thank you very much!:)

[unsigned123]

yes,loading with 0.12.9 does not cause aff-change,but loading with 0.12.10 will cause the aff-change :(

[Booplicate]

Android(transplanted v0.12.5)

We don't have android builds, so that's probably your issue, unofficial builds change how the game works which we can't account for, so affection migration might not work as expected.

[unsigned123]

so you mean technically and normally moving a 0.12.9 save to 0.12.15 won't cause aff loss? in my test,moving Android save to 0.12.9 is ok and moving it to 0.12.15 causes aff loss

---- Replied Message ---- | From | Totally a @.> | | Date | 02/16/2024 14:33 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [Monika-After-Story/MonikaModDev] ssl module can't be loaded in requests and other packages/can't be used in python early/SSLError no valid ssl module (Issue #10201) |

Android(transplanted v0.12.5)

We don't have android builds, so that's probably your issue, unofficial builds change how the game works which we can't account for, so affection migration might not work as expected.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[unsigned123]

of course,all the version numbers mentioned above are PC versions

---- Replied Message ---- | From | Totally a @.> | | Date | 02/16/2024 14:33 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [Monika-After-Story/MonikaModDev] ssl module can't be loaded in requests and other packages/can't be used in python early/SSLError no valid ssl module (Issue #10201) |

Android(transplanted v0.12.5)

We don't have android builds, so that's probably your issue, unofficial builds change how the game works which we can't account for, so affection migration might not work as expected.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[Booplicate]

We do not support official build > unofficial build nor unofficial build > official build migrations, it's technically impossible. old official build > new official build is the only way we can support updating.

[unsigned123]

yes,i surely know that. moving from PC 0.12.9 to PC 0.12.15 won't cause aff loss directly right?

---- Replied Message ---- | From | Totally a @.> | | Date | 02/16/2024 19:18 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [Monika-After-Story/MonikaModDev] ssl module can't be loaded in requests and other packages/can't be used in python early/SSLError no valid ssl module (Issue #10201) |

We do not support official build > unofficial build nor unofficial build > official build migrations, it's technically impossible. old official build > new official build is the only way we can support updating.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[unsigned123]

let us suppose the versions above are all official pc versions

---- Replied Message ---- | From | Totally a @.> | | Date | 02/16/2024 19:18 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [Monika-After-Story/MonikaModDev] ssl module can't be loaded in requests and other packages/can't be used in python early/SSLError no valid ssl module (Issue #10201) |

We do not support official build > unofficial build nor unofficial build > official build migrations, it's technically impossible. old official build > new official build is the only way we can support updating.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[Booplicate]

moving from PC 0.12.9 to PC 0.12.15 won't cause aff loss directly right? let us suppose the versions above are all official pc versions

As I said, no, we don't intentionally drop affection unless there's a reason. No matter what version as long as you're updating from an older to a newer and using our official builds, there's a compatibility level.

For your problems, it's hard to say because I'm not sure what you meant by cut a lot, did it reset or you lost half of it?

[unsigned123]

thanks for your assistance,that's exactly the information I need.You may close this issue now.

---- Replied Message ---- | From | Totally a @.> | | Date | 02/16/2024 19:43 | | To | @.> | | Cc | @.>@.> | | Subject | Re: [Monika-After-Story/MonikaModDev] ssl module can't be loaded in requests and other packages/can't be used in python early/SSLError no valid ssl module (Issue #10201) |

moving from PC 0.12.9 to PC 0.12.15 won't cause aff loss directly right? let us suppose the versions above are all official pc versions

As I said, no, we don't intentionally drop affection unless there's a reason. No matter what version as long as you're updating from an older to a newer and using our official builds, there's a compatibility level.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>