Not able to run python script from shortcuts (in extension)

[aadsm]

Hi,

First things first, thanks a ton for building this. It’s literally the only way I can create some of the shortcuts I had in mind.

I’ve tried searching for this before but wasn’t able to find much that could help me. I have a very simple script in the $SHORTCUTS folder:

$ cat $SHORTCUTS/simple.py 
#!/usr/bin/env python3
print("hello world")
$

I’m running it from shortcuts in extension, I have it configured this way:

But when I run the shortcut it fails with the following:

Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = 'python3_ios'
  sys.base_prefix = '/Users/holzschu/src/Xcode_iPad/a-Shell/cpython/Library'
  sys.base_exec_prefix = '/Users/holzschu/src/Xcode_iPad/a-Shell/cpython/Library'
  sys.platlibdir = 'lib'
  sys.executable = '(null)/bin/python3'
  sys.prefix = '/Users/holzschu/src/Xcode_iPad/a-Shell/cpython/Library'
  sys.exec_prefix = '/Users/holzschu/src/Xcode_iPad/a-Shell/cpython/Library'
  sys.path = [
    '/Users/holzschu/src/Xcode_iPad/a-Shell/cpython/Library/lib/python39.zip',
    '/Users/holzschu/src/Xcode_iPad/a-Shell/cpython/Library/lib/python3.9',
    '/Users/holzschu/src/Xcode_iPad/a-Shell/cpython/Library/lib/lib-dynload',
  ]
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000000016b16b000 (most recent call first):
<no Python frame>

I’m able to run python —version though, which makes me think is not a path or linking issue. I also don’t know if the Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding is the actual error, or just an error when python is trying to dump a bunch of debug info (it does look like the latter though).

The script run totally fine within the a-shell app:

$ cd $SHORTCUTS
$ python ./simple.py 
hello world
$

Any clues or thoughts on this?

Thanks!

[holzschu]

The extension and the app are two different apps, and the extension does not have access to the Library folder inside the App. So basically, Python scripts cannot run in the extension, because Python cannot access all the Python files. I don't see a way to make Python run in extension without making the app much larger than it already is.

Keep in mind that you can run a shortcut in the app, then go back to the Shortcuts app with "open shortcuts://".

[aadsm]

I don’t think that’s correct… But bear with me on this, as I don’t have much experience with iOS apps. Here’s my current understanding of the 4 different directories and their access: (I mostly learned about this in this issue: https://github.com/holzschu/a-shell/issues/74)

1) /private/var/containers/Bundle/Application/7BBF6FA9/a-Shell.app/ <— The a-shell app’s root folder. 2) /private/var/mobile/Containers/Data/Application/71DA6BED/ <— The $HOME folder of the a-shell app. 3) /private/var/mobile/Containers/Data/PluginKitPlugin/342ABFFF/ <— The $HOME folder of the a-shell shortcuts’s extension. 4) /private/var/mobile/Containers/Shared/AppGroup/D0820C30/ <— $SHORTCUTS & landing folder of the a-shell shortcuts’s extension.

• Folder 1) is accessible by both the a-shell app(duh) and the “a-shell shortcut’s extension” as it does not contain any user data (this can easily be verified by listing its contents from the shortcut running in extension)
• Folder 2) contains the user data for the “a-shell app” and only that app is able to access it.
• Folder 3) contains the user data for the “a-shell shortcuts’s extension” and only the extension can access that data, as you said, they are different apps from the OS point of view.
• Folder 4), like the name implies, is a shared folder between the a-shell app and the “a-shell shortcuts’s extension”.

I was actually able to run python scripts from shortcuts in extension after I set the $PYTHONHOME to point to /private/var/containers/Bundle/Application/7BBF6FA9/a-Shell.app/Library.

To access the pip installed packages(which is user data) I copied its site-packages folder to the $SHORTCUTS directory, and then I set $PYTHONPATH to be ./Library/lib/python3.9/site-packages. I wonder if it would be possible for the extension to set $PYTHONHOME? Or even copy the pip installation to $SHORTCUTS and set $PYTHONPATH automatically? 😅.

One problem I have though, when testing this out, is that sometimes I get into a cycle of “pythonB: command not found” and subsequent python executions always yield the same error but increasing the A letter (e.g.: pythonC). Not sure what this is about, when this happens I try to quit the a-shell if it’s running and wait a bit until it works again.

[holzschu]

Hi, thank you very much. You are definitely right. I will update the extension.

There is the small issue that I do not know how to make the extension know about the location of /private/var/containers/Bundle/Application/7BBF6FA9/a-Shell.app/ from the extension, but I'll look into it.

iOS developer rules says that in-Extension shortcuts should last less than 5 s or else they can be terminated by the system, but that's another question.

[holzschu]

So, to report: I have python running in Extension on my development setup, which is pretty cool, thank you for the suggestion. However, the 5 seconds rule still exists. On my iPad pro, "pip list" takes too long and cannot be executed in Extension ("There was a problem communicating with the app" is the error message). This still needs some tuning before it can be released, but it will be in the next version.

[aadsm]

Those are great news, thank you! Yeah, I noticed the very short limit as soon as I run the actual script I wanted, so in the end I will still need to run it in the app anyway. It was still fun to make it work for small running scripts, just not sure if it will be useful at all, time will tell I guess. But it's always good to have that possibility with a much better user experience (although might be tricky for people to understand there's a 5s limit and that it's not an a-shell extension bug).

I still have a couple of questions (not sure if I should open new issues for these):

1) Do you have any idea about those 'pythonX: file not found' issues? 2) I was happy to see that you also use the iPad Pro. I have the same setup with external keyboard and mouse, but I'm running into UI issues when I alt-tab between say, shortcuts and a-shell:

• it's common for the font size to keep increasing and make the terminal almost impossible to use (I don't see the cursor because it's under the fold among other issues).
• I need to double click in the terminal for it to gain focus and be able to type. Like I said, I'm not familiar with iOS development, but I wonder if it's possible to auto focus the terminal once the app itself regains focus?

Thank you very much!

On Mon, Aug 9, 2021 at 06:03 Nicolas Holzschuch @.***> wrote:

So, to report: I have python running in Extension on my development setup, which is pretty cool, thank you for the suggestion. However, the 5 seconds rule still exists. On my iPad pro, "pip list" takes too long and cannot be executed in Extension ("There was a problem communicating with the app" is the error message). This still needs some tuning before it can be released, but it will be in the next version.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/holzschu/a-shell/issues/295#issuecomment-895204566, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAATCKSSAAQGE3YDMKXLVILT37G3PANCNFSM5BYZXLPQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

-- Best regards, António Afonso

[holzschu]

1) Do you have any idea about those 'pythonX: file not found' issues?

Yes, that is related to an internal variable that was not set. With the current settings, if you keep going, you will eventually loop back to python being executed (you need to go through pythonB, pythonC, pythonD and pythonE before it works again)

2) I was happy to see that you also use the iPad Pro. I have the same setup with external keyboard and mouse, but I'm running into UI issues when I alt-tab between say, shortcuts and a-shell:

• it's common for the font size to keep increasing and make the terminal almost impossible to use (I don't see the cursor because it's under the fold among other issues).
• I need to double click in the terminal for it to gain focus and be able to type. Like I said, I'm not familiar with iOS development, but I wonder if it's possible to auto focus the terminal once the app itself regains focus?

These seem to be fixed with iOS 15 beta, or they might be related to subtle differences between our configurations. If I alt-tab between Shortcuts and a-Shell, I don't need to click in the terminal to regain focus.

[holzschu]

Hi, just a heads up: the latest TestFlight version (build 181, https://testflight.apple.com/join/WUdKe3f4 ) has the ability to execute Python scripts (but no local packages, and the 5 second rules still applies).