All addon updaters will not work on blender 2.79a and earlier (OSX only, SSL issue) #39

Open schroef opened 6 years ago

IMPORTANT UPDATE (via @TheDuckCow)

Due to a recent change in GitHub's allowed SSL standards, Blender 2.79a and earlier users on Mac OSX will not be able to use the updater, and will see the below screenshot. This only applies to users of Mac OSX blender versions 2.7X and GitHub hosted repositories. There are no issues for GitLab or Bitbucket hosted repositories, and Github-hosted addons continue to work fine on linux & windows. No fix is currently available for OSX blender <2.8 users.

Update February 2019: GitLab is now affected in the exact similar way. Note that Bitbucket-hosted repositories will still function in 2.79 and earlier; all work appropriately in 2.8.

ssl_error_updater What OSX users would see prior to the v1.0.5 release

Update - release of v1.0.5

The latest release of the updater code does not fix this issue (since it is a server side/blender compiling lack of compatibility), but it does at least more directly acknowledge the error and provide the user with a manual-download solution if affected. Those not affected with continue to see the updater working as normal. Behavior for those affected is as follows:

ALT

ORIGINAL ISSUE POST

(Original issue title: After network downage is get a tlsv1 error)

rv 1.0.4 bl 2.78c OSX

After my internet went down i see this error ssl tlsv1_alert_protocol_version

Weird thing is internet works just fine and before that the updater worked just fine. I also see this in the console Exception in thread Thread-3: Traceback (most recent call last): File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/addon_updater/addon_updater.py", line 1309, in async_check_update self.check_for_update(now=now) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/addon_updater/addon_updater.py", line 1035, in check_for_update self.get_tags() # sets self._tags and self._tag_latest File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/addon_updater/addon_updater.py", line 523, in get_tags self._prefiltered_tag_count = len(all_tags) TypeError: object of type 'NoneType' has no len()

It also shows ssl.c:645

Tried googling it, but it to complicated for me

Thanks for reporting, sounds complicated but I think I understand the root issue. Or at lest better error handling, it’s doing Len on what should be the list of tags but is a None. I’ll take a look

Patrick W. Crawford Moo-Ack! Productions www.TheDuckCow.com youtube, twitter, facebook

On Feb 22, 2018, at 15:33, schroef notifications@github.com wrote:

It also shows ssl.c:645

Tried googling it, but it to complicated for me

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

I did some more digging and trying to find something. I've got a feeling something changed perhaps in the API method. Because all worked fine just 2 days ago.

When check the get_raw base URL request, this is the part where the warnings are raised. If i check the error outside the try: function i see this in the console. That is the function in addon_updater.py #601

`Traceback (most recent call last): File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 1254, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 1106, in request self._send_request(method, url, body, headers) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 1151, in _send_request self.endheaders(body) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 1102, in endheaders self._send_output(message_body) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 934, in _send_output self.send(msg) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 877, in send self.connect() File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/http/client.py", line 1260, in connect server_hostname=server_hostname) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/ssl.py", line 377, in wrap_socket _context=self) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/ssl.py", line 752, in init self.do_handshake() File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/ssl.py", line 988, in do_handshake self._sslobj.do_handshake() File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/ssl.py", line 633, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/threading.py", line 862, in run self._target(*self._args, *self._kwargs) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 1344, in async_check_update self.check_for_update(now=now) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 1063, in check_for_update self.get_tags() # sets self._tags and self._tag_latest File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 544, in get_tags all_tags = self._engine.parse_tags(self.get_api(request), self) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 636, in get_api get = self.get_raw(url) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.78/scripts/addons/TheaForBlender/addon_updater.py", line 613, in get_raw result = urllib.request.urlopen(request) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(args) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 1297, in https_open context=self._context, check_hostname=self._check_hostname) File "/Applications/blender-2.78c/blender.app/Contents/Resources/2.78/python/lib/python3.5/urllib/request.py", line 1256, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:645)>`

Hey there one followup, I see it says 2.78c in some of your paths. Just to give it a spot check, could you try installing and seeing if the same issue occurs in the latest official blender 2.79 branch? Just want to rule out any chance it's a build-specific issue, because after some quick reading it seems potentially related to mismatched SSL versions within the python library, but only a guess so far.

Also one thing I did actually just fix in the latest dev branch here is the line self._prefiltered_tag_count = len(all_tags), so maybe also give it a try downloading the addon updater dev branch

@TheDuckCow

Just tried 2.79 and i see same error;

Traceback (most recent call last): File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/addon_updater/addon_updater.py", line 1309, in async_check_update self.check_for_update(now=now) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/addon_updater/addon_updater.py", line 1035, in check_for_update self.get_tags() # sets self._tags and self._tag_latest File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/addon_updater/addon_updater.py", line 523, in get_tags self._prefiltered_tag_count = len(all_tags) TypeError: object of type 'NoneType' has no len()

Wondering what happened, if its either OSX change. But i didnt do any system update nor did i update python. My other guess something with the API of gitHub changed.

All worked fine last week, i tried testing something for @Nutti, which also added a issue about nested folders. Suddenly all my versions stopped working. The original addon as well as my adjusted one.

Ill try that dev version and see if that works.

Okay that was a quick one, still see a tlsv1 error, but this time it says _ssl.c:720 whereas before it said _ssl.c:645

This is console output on that dev branch Check update pressed, first getting current status addon_updater_demo updater: Starting background checking thread addon_updater_demo BG thread: Checking for update now in background Checking for update function addon_updater_demo Updater: Read in json settings from file Getting tags from server master branch found, no releases {'zipball_url': 'https://api.github.com/repos/cgcookie/blender-addon-updater/zipball/master', 'name': 'Master'} addon_updater_demo: Wrote out updater json settings to file, with the contents: {'last_check': '2018-02-27 21:19:11.874183', 'just_restored': False, 'ignore': False, 'just_updated': False, 'version_text': {}, 'update_ready': False, 'backup_date': ''} No version strings found text: Master addon_updater_demo: Wrote out updater json settings to file, with the contents: {'last_check': '2018-02-27 21:19:11.874183', 'just_restored': False, 'ignore': False, 'just_updated': False, 'version_text': {}, 'update_ready': False, 'backup_date': ''} addon_updater_demo BG thread: Finished checking for update, doing callback

IMPORTANT UPDATE

There appears to have been a change in GitHub's api authentication/certificate handshakes. I have tested multiple different addons using the updater module hosted on GitHub, and they all now fail to work with this same SSL error. I am looking into GitHub's API release documentation to see if there's any official changelog which could be referred to.

More information to come soon as I explore this issue further and come to a solution, but in the meantime: all GitHub hosted addons with updater code will fail to authenticate.

Note that the GitHub & GitLab implementations still function correctly without issue

@schroef Just an FYI I'm editing your initial post accordingly to flag this banner

Minor update, further exploring - this issue is likely related to this notice about depreciated cryptographic connection methods.

https://developer.github.com/changes/2018-02-01-weak-crypto-removal-notice/

If this is the case, then unfortunately this is a more fundamental declaration of no support for the level / version of SSL packaged with blender to date so far. Still to be confirmed further.

Supporting evidence: The following set of code works within terminals with more recent SSL versions without issues, but then fails when run within the blender console.

import ssl
import urllib.request

print(ssl.OPENSSL_VERSION)
api_url = 'https://api.github.com'
user = "cgcookie"
repo = "blender-addon-updater"

url = "{}{}{}{}{}{}".format(api_url,"/repos/",user,"/",repo,"/tags")

request = urllib.request.Request(url)
context = ssl._create_unverified_context()
result = urllib.request.urlopen(request,context=context) # issue occurs here within blender
result_string = result.read()
result.close()
get = result_string.decode()
print(get)

One point of further encouragement: It appears this code snipped does work in blender 2.8, meaning the version of SSL is updater there moving forward with newer blender versions. I confirm it does NOT work with blender 2.79a (or earlier) on OSX. Will be trying on other operating systems too.

But one thing i dont understand, that ssl or tlvs library is not something which seems to come with blender. It is something used in Python library, right?

On the other this will make this addon not working until blender 2.8 comes out. That will take months i guess

How does this thing handle for WIndows users? I mean they are using the same Python library right, why would it work for them, because the python is the same. The SSL is the same for all systems right or do different systems handle different SSL systems. That would be very weird.

It would seem that, oddly enough, the OSX build is not using the same SSL library. I personally tested on both windows and Linux and it works fine, and newer builds of OSX (ie blender 2.8) also work fine. I agree very weird, but I have experienced some cases where one build is even missing certain libraries entirely (at one point numpy was not there); I also tested in several versions of blender OSX, seems like it's been consistently and outdated library compared to the others.

Okay so that a "mistake" in the builds then, am i right?

What a shame, it was working perfect. Cant this installer work ussing https then?

I want to say a "mistake", but it's one that was recurring in previous builds, so not sure how to call that. Regardless yes it is a shame.. though if anything, I would say lucky that at least Windows and Linux are okay, it could have been all three not working.

There could be a workaround which involved packaging a custom build of an ssl library with the addon and have the updater rely on that over blender's built in module, but it would be a lot of extra code and potentially introduce other bugs/oddities that I wouldn't really call it worth the tradeoff.

Could i let it check for updates only perhaps or does that also need the SSL. I think it does because the first error popsup when its trying to get to the release headers. Im just wondering why getting a json file needs to be done using a SSL connection. Doesnt make sense to me, im no web guru. But a secure connection for a simple json file seems a bit over the top or am i wrong here?

Unfortunately not even checking. The thing is, the API now refuses any connection to the API. I recommend trying the code snipped posted above, try once in a normal "terminal" window outside of blender, and once in an OSX blender build.

As I said, there's a possibility to include an SSL library to use instead and do a relative import of that instead of through the bundled SSL. but... not trivial. Especially since the urllib itself imports ssl, so you'd have to duplicate multiple modules locally on top of providing an external SSL lib. I would be happy to see if it's possible to do within your own repository, partially as a curiosity, but I don't think it's code I'll want to introduce into the master repo here.

Have just tried simpel urllib.request.urlopen that is indeed the bugger. Get the same error. I think I just need to pass for now and wait.

I will add a simpel wm.open.url button zo user van check online quickly.

Thanks for the help. Hope my releases adds perhaps come in handy. Once we are at 2.8 I'll give another go, if I don't forget.

Ps are there already builds? 2.79 isn't that old

Blender 2.8 is a big transition from the current v2.79, similar to how 2.49 had a big jump to 2.5 (if you were a user back then). In other words, 2.79 will have several releases maybe in parallel to 2.8, while 2.8 is more like a build from the ground up. Adjusted UI, different APIs in some cases, new core features etc. 2.8 is pretty much experimental for now but yes builds do exist, not with all functionality. Learn more here :) https://www.blender.org/2-8/

Cool, yes i did try blender before the better GUI. But i was horrified by it. Tried it a couple times but was just not working for me

– Rombout

On Fri, Mar 9, 2018 at 12:32 AM, Patrick W. Crawford < notifications@github.com> wrote:

Blender 2.8 is a big transition from the current v2.79, similar to how 2.49 had a big jump to 2.5 (if you were a user back then). In other words, 2.79 will have several releases maybe in parallel to 2.8, while 2.8 is more like a build from the ground up. Adjusted UI, different APIs in some cases, new core features etc. 2.8 is pretty much experimental for now but yes builds do exist, not with all functionality. Learn more here :) https://www.blender.org/2-8/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CGCookie/blender-addon-updater/issues/39#issuecomment-371710769, or mute the thread https://github.com/notifications/unsubscribe-auth/AGmjAMXrp-T25mbvh8rxHwaa5y0pG-tcks5tcgXugaJpZM4SP8Ko .

Minor planned update, as more and more people are reporting this issue - I will implement a small "patch" which will at least specifically recognize the SSL error and prompt a different, more tailored message (e.g. in place of "check internet"), and provide an option to directly visit the associated website for updating.

That would mean for OSX users view a completely different GUI i think. But that way the other users would still profit from the addon.

Perhaps but not necessarily; essentially I will set an overwriting flag to trigger the "manual download" popup instead, which would use the link defined by addon_updater.website in the addon_updater_ops.py file.

In this way, it would not actually be hardcoded for OSX users, but hard coded to the specific error; if by chance the issue becomes resolved (unlikely), then it would return to being functional without any further changes.

It's also worth stating, that if updating is a very critical feature for an addon for *all* users, the functionality still works fully when the repository is hosted on GitLab or Bitbucket; in an absolute workaround scenario, a second repository could be used exclusively for the purpose of checking against tags. In this scenario, even if the actual code is not hosted on GitLab/Bitbucket, tag names could be still set there in parallel to the GitHub hosted repository (with addon_updater.manual_only being to True). This is not very convenient to maintain by any means, and also would remove the ability to directly update (as opposed to just check* for updates) for all users even non OSX.

The issue doesn't seem to be Blender specific, but Python 3.5 specific. When I run the snippet posted above with Python 3.5 it gives me the error, with Python 3.6 it's fine.

Not that it helps in any way to resolve the issue, but I checked the different OpenSSL versions on macOS and Linux (Fedora 27):

macOS:
- Python 3.5 and Blender with Python 3.5: OpenSSL 0.9.8zh 14 Jan 2016
- Python 3.6 and Blender with Python 3.6 (buildbot): OpenSSL 1.0.2n 7 Dec 2017
Linux:
- Python 3.5: OpenSSL 1.1.0g-fips 2 Nov 2017 (built from source with OpenSSL installed with the package manager, that explains the recent OpenSSL version here.)
- Blender with Python 3.5: OpenSSL 1.0.2j 26 Sep 2016
- Python 3.6: OpenSSL 1.0.2m-fips 2 Nov 2017
- Blender with Python 3.6 (buildbot): OpenSSL 1.0.2j 26 Sep 2016

Just pushed an update (to the dev branch, will be in master soon)which shows this for users encountering the error. The button opens the website as entered via updater.website where they should be able to download and instead in the normal fashion.

ssl_error_open

Thats is sweet!, i think thats better for me now. I can get rid of my button i created which does the same, but it does that for all users.

I just need to build in my releases section then, hope that still works.

Sweet works nice in 2.78c

I did get an error, there is a missing comma character in thea addon_update.py #567 Missing character screen shot 2018-03-26 at 12 55 21 pm

Works fine after adding the comma screen shot 2018-03-26 at 12 54 48 pm

Works fine now for me in 2.78 and 2.79 OSX 10.13.1

PS can we ignore this new error in the console then"

Traceback (most recent call last):
  File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/TheaForBlender/addon_updater.py", line 1366, in async_check_update
    self.check_for_update(now=now)
  File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/TheaForBlender/addon_updater.py", line 1103, in check_for_update
    link = self._tags[0]["assets"][0]["browser_download_url"]
KeyError: 'assets'

You caught me a little bit in the middle of merging in some features contributed by another user, planning to clean it up today - thanks for identifying these two items! And hopefully I’ll have the v1.0.5 out officially today.

On Mar 26, 2018, at 13:15, schroef notifications@github.com wrote:

PS can we ignore this new error in the console then"

Traceback (most recent call last): File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/Applications/blender-2.79-macOS-10.6/blender.app/Contents/Resources/2.79/python/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/TheaForBlender/addon_updater.py", line 1366, in async_check_update self.check_for_update(now=now) File "/Users/PromotieSpullen/Library/Application Support/Blender/2.79/scripts/addons/TheaForBlender/addon_updater.py", line 1103, in check_for_update link = self._tags[0]["assets"][0]["browser_download_url"] KeyError: 'assets' — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

ahh okay, i wanted to test this dev version and see how it working.

PS did i see it correct you added the "releases" options like i added in my fork?

Yes, that is also part of what I’m working on to make consistent across github, gitlab, and bitbucket in a standardized way.

On Mar 26, 2018, at 14:34, schroef notifications@github.com wrote:

ahh okay, i wanted to test this dev version and see how it working.

PS did i see it correct you added the "releases" options like i added in my fork?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

I was looking at a different Render Engine addon and i noticed that this has some kind of updater which does work in 2.78c. WHat i dont get is that it also uses the request url but doesnt give an error. It simply reads a txt file. I thought i tried doing a real simple test like this but the i was thinking the URL request simply doesnt work. Perhaps this works cause its using Bitbucket?

urllib.request.urlopen("https://api.bitbucket.org/1.0/repositories/coronablender/render_corona/raw/master/version.txt").read().decode('utf-8').strip()

Perhaps worth a try to test, but the current version is already good of course

Yes, this is certainly of course possible - they are just accessing they plain text file to get a file directly from the server (where file could be an html page like this here, or any raw text file). The reason it works is that it's not using the API at all, it's just accessing the file the way a user might via a webbrowser. The other thing is, even if this person were using the addon updater (which uses APIs), they are hosted on bitbucket - as a reminder, the issues above only applies to the mac version of blender and for repositories hosted on github. Any other combo, it would still be working fine.

It would be theoretically possible to build a "scraper" which gets files and versions from github by just iterating over webpages, but that's not a very stable approach (webpage formats may change, for example). But custom implementations like what corona render uses would work fine too, it would just be up to you to define how it works.

I thought it was cause of BitBUcket and how that site handles api orso. Im no guru in this so was a long shot.

I tried using simple request on GitHub and that was indeed declined like the other method

Ah you are correct, I didn't realize the same rules were also being applied to access to their webpages/over port 80 too. So I guess that is correct, it would not be a functional workaround.

Updated again to announce that this issue now affects all updating engines, including Bitbucket. However all three engines are fully functional in all versions of 2.8 and current 2.9 alpha.

CGCookie / blender-addon-updater