Python - KeyBERT integration

[tomastiminskas]

We need to use KeyBERT python library on iOS project to do keywords extraction from message text.

As it says on the keyBERT library page (https://github.com/MaartenGr/KeyBERT) there are other libraries for RAKE (Rapid Automatic Keyword Extraction Algorithm) but we tried the only one available for swift (https://github.com/aj604/RAKE-swift) and it didn't work as expected. We used the same text from the keyBERT library readme and the results were pretty different (we got sentences instead of words and some of them made no sense at all).

The best option seems to be implementing a way to interact from swift to python keyBERT library.

We found this video of the EuroPython conference explaining how to do that (https://www.youtube.com/watch?v=_A-FVRbg7Qc&t=4038s) using the Python-Apple-Support library (https://github.com/beeware/Python-Apple-support) and PythonKit (a library originally built for MacOS), but we couldn't get it to work.

There is a branch with name: tt/feature/python where we could make python work since we are importing sys and getting python version along with some other information, but we couldn't make random python library work as it's shown on the EuroPython conference video neither the keyBERT library (both imports are failing because of other imports like math or bumpy).

ActionsManager.swift - testPython()

[rkirkendall]

Hi @tomastiminskas. I made some progress here on at least figuring out why you couldn't make random work.

It seems to be because the most recent releases of Python-Apple-Support don't come with full versions of the core Python library.

I noticed this because in the EuroPython video he makes a point of talking about the size of libPython.a (his non-sim version was ~70mb). The libPython file in your branch is less than half the size of that.

To see what I mean compare the size of the iOS support packages on the most recent versions and with the ones from May (before the EuroPython conference). They're way smaller!

I was able to successfully generate a random number using 3.9-b7

[tomastiminskas]

@rkirkendall thanks for helping with this. That could be definitely the cause. I will get back to it soon and try again using the version you suggested. If random works (which means that version comes with the core Python library) then I might be able to integrate KEYbert as well. Willl keep you posted

[tomastiminskas]

We moved to python version 3.9-b7 on branch tt/feature/python-3.9-b7 and we could make Python random work as @rkirkendall suggested.

But now when trying to import keybert we are getting the following error:

Fatal error: 'try!' expression unexpectedly raised an error: Python exception: Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python interpreter from there.

[rkirkendall]

This one is tougher. Here's where the rabbit hole has taken me so far.

I got the error you mentioned when trying to use the numpy build in your branch. I searched for iOS Numpy and found this project

The project doesn't run for me, but I substituted their numpy package in for yours and got a different error

PythonKit/Python.swift:706: Fatal error: 'try!' expression unexpectedly raised an error: Python exception: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "/Users/rickykirkendall/Library/Developer/CoreSimulator/Devices/634CDB1B-5EBA-4571-A56E-1448ACD95228/data/Containers/Bundle/Application/D8F2C6B2-7F9F-4A61-B4A4-C53E8F72EC9F/PythonTest.app/PythonTest"
  * The NumPy version is: "1.20.2"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

So I started looking into how I could try to rebuild numpy for iOS.

I tried pip install numpy with my PYTHONPATH pointing to the iOS-specific platform-site directory as indicated in the Python-Apple-Support readme, but got a build error on install.

Researching further, I found where this should've been possible at one time according to numpy's 1.15.0 release notes. Looking into the numpy gh's issues for iOS mentions, I found this issue where Apple discontinued a key API needed for numpy, but looks like there may have been a resolution to that issue.

Seems possible, but haven't cracked it yet

[rkirkendall]

Note: My approach has been trying to get Python, PythonKit, numpy and keybert to work on a simple test project first