The application “Install_MacOSX” can’t be opened. / Python 3.7 libraries corrupt

[jkotl0327]

Hello, I put the 4 patch files into the main folder (shown below), and finder gives me the error message: "The application “Install_MacOSX” can’t be opened." (shown below). Did I do something wrong here (I am very bad with technology)? Thank you.

[RandyHarr]

You are doing pretty well with technology for having found how to post here and include a merged screenshot ;)

What is the path to your folder shown here? Did you double click to open or right click and select open? I cannot replicate this but will try to investigate while awaiting your answer.

[RandyHarr]

I found one explanation that seems to indicate it is a permission problem on the executable inside the directory. This can happen if the app is unzipped by other than the standard Apple program. See the StackExchange post. I will double check the permissions are set correctly in the posted ZIP folder itself. Here is another StackExchange post that may explain the issue more clearly.

What the above post suggests is to open terminal and cd down into the .app folder and find the applet executable file. Do the command chmod ugo+x ~/Downloads/Install_MacOSX.app/Contents/MacOS/applet on that file (shown as if still in the Downloads folder here and before you copy into the WGSExtract folder). Make sure to do this for the Start_MacOSX.app applet file also (chmod ugo+x ~/Downloads/Start_MacOSX.app/Contents/MacOS/applet). But as it could be a local unzip issue, providing these instructions as well. Let me know if this is the issue and fixes it for you as well.

I double checked the uploaded patch file and its permissions were set correctly. But I recreated it on the MacOS platform and re-uploaded it just in case. You can try re-downloading to see if that fixes the problem as well. Let me know either way.

[jkotl0327]

Thank you for your response, I don't remember what I used to unzip, but it might have been an external app for the patch and the mac function for the WGSExtractBeta itself or something like that. The path to the folder is: Documents -> WGS Extract -> WGSExtractBeta -> WGSExtract. WGSExtract is the folder shown. I tried both double clicking and right clicking and both produced the same result. I also tried the command you suggested (not 100% sure if I did it right) and this is what I got:

I even tried cd Documents and it also said no such file or directory, so I am confused.

[jkotl0327]

Hello, I was able to resolve this issue but I ran into another one. I ran these commands: % cd /WGSExtract % sudo -h ./Install_MacOS.sh Installation seemed to be successful, so I tried to run: python3 ./programs/wgsextract/wgsextract.py This was the result:

I assume this is because my python is not configured for Tk. How would I go about configuring it? My python version is 2.7.16, the code that appears after entering your command says version 3.7, if I install the latest python 3.8.3 would that be okay, or would I need to specifically install 3.7?

[RandyHarr]

Please let me know how you resolved the permission issue so I can know if we need to change the MacOSX release process. Neither of us develop on MacOS and only test there using spare machines we have. Which tend to be clean installs.

tkinter (imported as Tk) is a basic module included in all python3 releases as default. It appears that if you think you have python 2.7.16 installed that you must also have 3.7 (unless you actually only have 3.7.16). Using the command "python3" will only work in 3.x python releases.

The install script is specifically calling on python 3.8 simply because it is the latest available. We actually test on 3.7, normally, on all platforms. We are looking to make future releases check for all versions (python, samtools, etc) so a minimum version can be assured.

It could be a library "mix" issue. If you installed Python from Homebrew or MacPorts (we use a native installer in our install script), that can cause conflicting libraries. It could be you had an old Python that did not cleanup completely and you have mixed libraries. We should not have installed Python3 if it already existed. But ... Your installation is finding the tkinter install library but then failing when trying to load some of the additional files internal to that library.

The manual shows some folders that may have to be manually removed to uninstall Python properly. Specifically, on page 38 top it shows: (a) Drag and drop the Python3 installation from the Applications window to the trash; empty trash (b) sudo rm -rf /Library/Frameworks/Python.framework Clearly be VERY careful when you type sudo rm -rf. That is using root to do a recursive delete on a directory tree. If you do not enter in the correct directory, you could irreversibly delete files you did not intend to. Key is, your python libraries appear corrupted by multiple installations of Python 3.7 from different sources.

(note: I just noticed the patch install script still has an older, incorrect PIP install command that brings in outside Python libraries. For some reason, a "pip install tk" found its way in there. Does not really harm anything but it is not needed nor is it the Tcl/TK package -- which is installed as "tkinter" in the release directly. Just mentioning in case you notice and are led astray by it.)

[jkotl0327]

I never installed python, I just have whatever came with the mac. I also do not see python in my applications. Are you asking me to uninstall and reinstall python? When I entered the command 'python --version' it said Python 2.7.16, that's all I know. I'm not that good with all the jargon so I still don't really understand how to resolve the issue. After I uninstall python should I reinstall python 3.7? Which version of 3.7?

How I fixed it is by going step by step and entering: cd / cd ~/documents cd ./WGS_Extract cd ./WGSExtractBeta cd .WGSExtract

Once in the directory I entered: chmod ugo+x Install_MacOSX.app/Contents/MacOS/applet chmod ugo+x Start_MacOSX.app/Contents/MacOS/applet

Then % cd /WGSExtract % sudo -h ./Install_MacOS.sh

Then entered password

Then entered the python code line

[RandyHarr]

If I follow you correctly, then the two commands that should maybe fix the permission problem are: chmod ugo+x ~/documents/WGS_Extract/WGSExtractBeta/WGSExtract/Install_MacOSX.app/Contents/MacOS/applet chmod ugo+x ~/documents/WGS_Extract/WGSExtractBeta/WGSExtract/Start_MacOSX.app/Contents/MacOS/applet But if your previous chmod returned without error, then they worked fine. And should have hopefully fixed the problem with running the .app directly by clicking. All it does though is the "sudo -H ./Install_MacOSX.sh" that you found and are typing.

Not sure why so many nested folders. More of an annoyance. You could get rid of some of that by: mv ~/documents/WGS_Extract/WGSExtractBeta/WGSExtract/ ~ and that will bring it to ~/WGSExtract/

So Python2 (or simply Python) comes by default in MacOS. "which python" will show it likely in /usr/bin/python. Python3 is what we install, if not there. "which python3" will likely show /Library/Frameworks/Python.framework/Versions/3.8/bin/python3" if it was installed by our script. The python3 installer puts a link in /usr/local/bin/python3 that points there. It does appear you had a previous python3 installation (v3.7) that has corrupt libraries. So yes, I would uninstall Python3 (with the previously given commands) and then rerun the WGSExtract installer to reinstall Python3. That should fix your library corruption issue. The "sudo rm -rf" command I gave earlier will remove all Python3 libraries. But just please be careful when entering.

MacOS and Python installations seem to be a bit of an issue, we find. In that uninstallers do not clean-up properly. The Python3 v3.8 should not be looking in the Python3 v3.7 libraries as it is. Am hesitant to do so but suppose we could remove all Python3 libraries before doing an install in the install script. (Note: the fact we did not detect the python3 executable but the libraries were still there tells us there was a python 3.7 installation at some point that was removed too simply by dragging the executable out of the Applications folder and into the trash. Python3 apps, when this is done to uninstall, should not leave the libraries framework folder around.)

Sorry for these continued problems. We will hopefully learn something we can do in the release process to make it easier on the next one who follows. Others have installed on MacOSX (as have we) and not experienced these particular issues; that we have heard. But we will work to make the process more robust. Thank you.

[jkotl0327]

I uninstalled using the command you said, but now when I enter: cd /WGSExtract % sudo -h ./Install_MacOS.sh

I get: zsh: no such file or directory: folder fg: no current job

Not sure what is going on. When you said reinstall you meant those commands, right?

I also tried to go straight to: python3 ./programs/wgsextract/wgsextract.py

But I got the same error as before, probably because I'm not able to reinstall the python through the Install_MacOSX

[RandyHarr]

Hoping you maybe meant cd ~/WGSExtract? (forgot the tilde (~) to imply your home directory? starting with / means root directory) sudo -h ~/WGSExtract/Install_MacOS.sh should work no matter where your shell is parked.

It seems the Tcl/TK libraries on Mac is a big issue. Mostly because Apple fixes it awkwardly in their OS release of Python2 and Tcl/Tk that is then not possible to easily override by later updates to either package. Somehow, miraculously, in our MacOS systems, we did not run into this as of yet. See warnings on the Python page: https://www.python.org/download/mac/tcltk/. Or even https://discussions.apple.com/thread/8066794 which describes how to check your Tcl/Tk version:

$ python3
>>> import tkinter as tk
>>> tk.Tcl().eval('info patchlevel')
'8.6.8'
>>> tk.Tk().eval('info patchlevel')
'8.6.8'
>>>

But granted, you are not getting as far as being able to read the tkinter library at all. I will continue to look at this but the only way I see we can deal with this is blast away any Python3 libraries before installing if we do not find a Python3 binary. Not elegant and could break another application, possibly, but ...

[jkotl0327]

Sorry I meant % cd /WGSExtract % sudo -h ./Install_MacOS.sh

[RandyHarr]

Oh, either you or I (or both) are getting lazy. The file is called "Install_MacOSX.sh" (missing the trailing X for 10 in the name designation)

[jkotl0327]

Copied from manual:

Also I added X and:

[RandyHarr]

Thank you. Fixed. Do an "ls" in the directory to see the files (in the future).

You edit the to your path, correct? maybe ~/ or ~/WGSExtract/WGSExtractDev/WGSExtract (not sure where you have it). So should look like:

cd ~/WGSExtract
sudo -H ./Install_MacOSX.sh

[jkotl0327]

[RandyHarr]

Sorry, bad copy/paste i missed with it earlier. capital H. sudo -H ./Install_MacOSX.sh

But is double clicking the .app still not working?

[jkotl0327]

Double click causes new terminal window:

[jkotl0327]

Now this happens:

[RandyHarr]

Yes, the terminal window is expected. It is saying it is not finding the Install_MacOSX.sh file at that specific path. Ditto for you directly entering the sudo command and not using the .app. Do pwd; ls to show me your path and files at the WGSExtract folder you are trying to run these commands from.

So what I have taken so far from all this is: (a) we need to find a way to do a chmod of the install of .app/Contents/MacOS/applet files (as an extra precaution) (b) we need to remove the Python3 libraries before installing Python3 if no Python3 binary found (as an extract precaution) (c) we need to update the manual listing the command "cd ; sudo -h ./Install_MacOS.sh" to have -H and MacOSX (done)

[jkotl0327]

Thanks :) I entered that command, and this came out:

[RandyHarr]

Arggh. Something I must have missed before. If you see in the manual, it says do not put in the Documents directory. Apple disallows executing from that directory. Has probably been the source of all your problems. execute mv ~/Documents/WGS_Extract/WGSExtractBeta/WGSExtract ~/WGSExtract. You can then cd ~/WGSExtract and run commands from there. Outside your Documents directory.

[jkotl0327]

I did all of that but then got this:

Looks like we are back to "command not found" Also, I moved the WGS_Extract folder to Desktop, out of Documents

[RandyHarr]

Do ls again to make sure you are in the right place. You can type simply ./Install_MacOSX.sh and it will work (simply asks for the password from the embedded sudo command). But your sudo command worked fine here.

[jkotl0327]

Here is ls and the simpler command you gave caused denial of permission, so I retried sudo and password which led to command not found:

[RandyHarr]

You can chmod ugo+x ./Install_MacOSX.sh to fix permission issues. Likely got stripped when you initially copied into the Documents folder. Not sure about the sudo failure at the moment. Not at the desktop to check like before.

So, just to summarize, in addition to three points I mentioned earlier, we now have (d) I missed your early mention of copying the release to the Documents folder (which, as advised in the manual, is not allowed by Apple)

[jkotl0327]

Those 4 points plus there might be a 5th, see below I did

But the python thing still isn't working, did I miss a step?

[jkotl0327]

I tried something different, I put the 2 chmod commands and then the sudo which started installing python3 but then this happened:

I see the text shell file for Install_MacOSX.sh has a bunch of spots where it says "fi", which it seems not to like

[RandyHarr]

After the chmod on the Installs, needed to still run the Install before running the python interpreter.

This 'fi' issue is something I discovered two weeks ago after an update to the new MacOSX zsh (the new default shell). Zsh does not like else; # comment ; fi (empty else clauses with only a comment). I changed scripts that had optional code in the else clause to always include a dummy echo command. I think the new patch I uploaded to fix the permission problem today included that fix. Will have to wait till I get back to a desktop to see. We try to force BASH instead of ZSH but this debugging with commands entered directly bypasses that.

Suspect the install of Python3 finished. Just that it could not continue to the next clause ... which it did not need to in your case. You might just try running the wgsextract program now -- the ./Start_MacOSX.sh or click the .app

[jkotl0327]

That makes sense. I tried what you said, running the program caused:

Double click caused:

Looks like python 3 did not finish, as far as I can tell. It doesn't seem like a problem unique to zsh, I got the same thing trying in bash.

[RandyHarr]

Do which python3 as it appears to be indicating it cannot find python3. Which would imply it is not installed. That "fi" issue in the old script will only crop up if python3 exists and it does not need to do anything. I just checked and the fix is in the python3 section of the script that I uploaded to assure the permissions were correct (first thing this morning).

So let me suggest the following, Let's make sure your Python3 is reinstalled from scratch with old libraries removed. (a) Drag and drop the Python3 installation from the Applications folder to the trash; empty trash (may be in Others sub-folder or may not exist at all) (b) Then execute the following commands:

sudo rm -rf  /Library/Frameworks/Python.framework
which python3   # should fail or return nothing as it does not exist; cannot be found
cd ~/WGSExtract
chmod ugo+x Install_MacOSX.sh Start_MacOSX.sh
chmod ugo+x Install_MacOSX.app/Contents/MacOS/applet
chmod ugo+x Start_MacOSX.app/Contents/MacOS/applet
/bin/sh Install_MacOSX.sh    #  or click the Install_MacOSX.app folder
/bin/sh Start_MacOSX.sh      #  or click the Start_MacOSX.app folder

I just successfully did this on a clean installation.

For the future, I am going to get rid of suggestions in the document to run programs using ./<file>.sh and simply use sh <file>.sh or even /bin/sh <file>.sh

This has definitely been excruciating for us both. As mentioned, many have been running Mac without any of these issues; including us. I probably should have just Skype/Zoom/? so we could screen share and had this all done in 15 minutes. Thanks for hanging in there.

[jkotl0327]

Thank you for hanging in there :) Maybe if this does not get resolved today we should do screenshare in the future. Which python 3 gives: /usr/bin/python3

I did not find python 3 installation in the applications folder, so I could not delete it.

I tried your commands without the deletion:

[jkotl0327]

Just to be sure, here is my applications folder, followed by its subfolder, utilities.

[RandyHarr]

OK, so your python3 executable is in /usr/bin. Which is totally unexpected. All the various installers I have seen put it in /usr/local/bin or /opt/local/bin. Not /usr/bin which is reserved for Unix native code. That is part of the issue. Even though the scripts use direct paths. When we use the command line it uses its search mechanism to find it. And /usr/bin is always near the first place looked (after /bin and your current directory). Hate to say it, but rm -f /usr/bin/python3. Normally one should never touch the /usr/bin directory. If you want, do ls -l /usr/bin/python3 first to get a sense of maybe when it was placed there (actually created not copied there).

When you pasted the "which" command with the comment after, it somehow could not understand the hash (#) and took it and each word after as a command that it did not understand. Really weird. Worked fine in my MacOS Terminal (BASH shell). Something odd there.

Have to go back and look at my archives of changes to the MacOS patch to see what version of the Install_MacOSX.sh script you could possibly have. It should not be causing that error as the script looks in a specific directory; not just the general "find any python3" program. Give me a few hours to look over all this, upload a new Patch file, and see if we can start from scratch. Please do the "rm -f" mentioned above in any case though.

[jkotl0327]

ls -l /usr/bin/python3 gives: May 27th (don't know what year or the significance of this date) I'll retried your commands without the hash to see if anything happened, but result was the same and I guess we'll probably have to start from scratch in a few hours like you said. Looks like rm -f /usr/bin/python3 failed because it says Operation not permitted. I went ahead and gave terminal fall disc access but result is the same so I guess I'll stop tinkering with it for now and give you as much time as you need for a new patch.

[RandyHarr]

If no year, is current year. What is your MacOS version. I wonder if Apple finally added python3 to their release. While there is nothing in our install code that would affect the existing installation, the rm -rf /Library/Frameworks/Python.framework may have affected the Apple install. So I need to investigate how to restore that as well, if so. Unfortunately, an Apple update does not work on VMs and so I am not on the latest release today. This should not be the case but ...

[jkotl0327]

I recently (may have been the 27th) updated to MacOS Catalina 10.15.5.

[jkotl0327]

Hi, Any progress on the new patch? If it would make the process easier for both of us, I would be open to doing screenshare.

[RandyHarr]

Apologies for the silence the last 24hrs. Beside catching up on things delayed while responding yesterday, I have been building a new Catalina 10.15 machine. The fact that the installer python3 release got tangled with your native MacOS python3 release requires me to verify on the platform directly. My existing Mojave platform could not be upgraded. Need to be assured we are not corrupting the OS install. While Python3 is not installed by default, Catalina did put a python3 "trigger" which would encourage users to install xcode. I need to make sure the std python.org release does not get entangled in that. Give me another 48 hrs to be sure I have it correct. Sorry for the delay.

[jkotl0327]

Sure, no problem.

[RandyHarr]

Was not able to complete a the in-depth Catalina verification. So I will commit to doing a shared-screen install to fix your WGS Extract release now. And provide a future possible script to restore your Python3 native if we ended up corrupting it further than it already was when you started the install. Send me a note with your email and we can exchange time zones, availability, etc. We will use Teamviewer as I will need to control not just see your screen while you watch. Sorry for the trouble and delay. But clearly the update to Catalina when one installs Xcode Python3 is still a possible issue that we need to resolve.

[jkotl0327]

I seem to have installed Teamviewer successfully and I gave it all the permissions it requested. I don't really know how to use it though (set up the screenshare).

[RandyHarr]

Uploaded new MacOS Patch with more robust Install /bin/sh script. Even with that, the VERS test for MacPorts selection did not work. So when I get my Catalina working, will have to investigate. But edited Jermey's script so the install would finish correctly. He is up and running.