Pipenv Install Timeout Error

[seandenigris]

v 0.8.2105.

When trying to run a Python snippet for the first time, I'm getting an obscure error. IIUC PBApplication>>#waitInitialization is timing out before pipenv is finished setting up, probably because I now have about a dozen additional packages in my Pipfile and it appears the timeout is hardcoded.

Possible Solutions

• Can we get feedback from pipenv install about whether it's frozen or just taking a long time?
• Increase the time out value
• Give a more helpful error message, possibly including the workaround below

Workaround

On the command line, ./PythonBridgeRuntime/install_env.sh before using Python in the image.

[svenvc]

We now have https://github.com/feenkcom/gtoolkit/issues/3604

In practice, yes initial pipenv based install takes a couple of noticeable seconds, but for me it works within the existing timeout limits.

Of course, a better indication of what is happening would always be welcome, but we are talking about an external subprocess here, I am not sure this is easy to solve.

[svenvc]

Closing for now, feel free to reopen if you think this is necessary.

[seandenigris]

I will see if #3604 helps here. In regards to whether the timeout is long enough, it is contextual because it all depends on how many packages the user is loading. For me, this pipfile for example has routinely exceeded the timeout and caused problems:

[[source]]
'name' = 'pypi'
'url' = 'https://pypi.org/simple'
'verify_ssl' = true

[dev-packages]

[packages]
pandas = '*'
ofxtools = '*'
phonenumbers = '*'
python-docx = '*'
titlecase = '*'
geopy = '*'
msgpack = '*'
msal = '*'
requests = '*'
html2text = '*'
playwright = '*'
mpv-silence = '*'
fuzzywuzzy = '*'
speedtest-cli = '*'
nextstep-plist = '*'
flask = '*'

[requires]
python_version = '3.10'

[svenvc]

In the new approach (PBNewPharoPipenvProcess) we use our own PyPI package (gtoolkit_bridge). Installation from scratch involves 3 steps:

2024-02-17T13:48:57.818872+01:00 pipenv install 777ms
2024-02-17T13:48:59.627451+01:00 pipenv install gtoolkit_bridge 1808ms
2024-02-17T13:49:01.48127+01:00 pipenv install debugpy 1854ms

Installation happens into a pipenv virtual environment inside an otherwise empty PythonBridgeRuntime directory.

If you need more modules, I see two approaches:

1.

Manually outside GT:

• stop the bridge
• edit or replace the pipenv file (or add requirements.txt)
• manually run pipenv install
• start the bridge

2.

Programmatically from GT:

#(pandas ofxtools ... speedtest-cli nextstep-plist) do: [ :each | 
    PBApplication uniqueInstance installModule: each ]

[svenvc]

Here is a quick & dirty script to parse a requirements.txt file into module names (skipping versions).

'/tmp/requirements.txt' asFileReference contents lines 
  reject: [ :line | line isEmptyOrNil or: [ ('#-./' includes: line first) or: [ line beginsWith: #http ] ] ]
  thenCollect: [ :line | (line findTokens: Character separators, '=') first ]

And a similar one to parse a Pipfile into module names (ignoring versions).

lines := '/tmp/Pipfile' asFileReference contents lines readStream.
[ lines next = '[packages]' ] whileFalse.
packages := OrderedCollection new.
[ (line := lines next) isEmptyOrNil ] whileFalse: [ packages add: (line findTokens: '=') first ].
packages

This is all in the assumption that you want to stay in GT.

[seandenigris]

I have been hacking the PB load behavior to round up all the Pipfiles in loaded repos (under pharo-local) along with GT's default Pipfile and merge them into one big Pipfile. It relies on a basic Pipfile parser that covered the syntax I needed for my projects. The hook is here: