[Bug]: searching regular expressions with python apsw package works on all platforms, except Termux

[eliranwong]

Problem description

May you please help? I have codes like:

def regexp(expr, item): reg = re.compile(expr, flags=0 if config.enableCaseSensitiveSearch else re.IGNORECASE) return reg.search(item) is not None

connection = apsw.Connection(self.database) connection.createscalarfunction("REGEXP", regexp)

With the codes above in place I can run the following query on Windows, macOS, Linux, but no Termux

SELECT * FROM table WHERE text REGEXP ?

It returns nothing on Termux.

Can you give me some idea how it may also work on Termux too?

Did I need to install any packages to get it working?

Steps to reproduce the behavior.

pkg install python

try the codes above.

What is the expected behavior?

searching regular expressions work on Termux as on macOS, Linux and Windows?

System information

• Termux application version:
• Android OS version:
• Device model:

[T-Dynamos]

Can you give a minimal working code

[eliranwong]

@T-Dynamos thanks for reply. Sure, I will provide a simple script here later today. The issue appears on Termux only. Other platforms work fine. I will work out a simple script and place here. Thanks for following up

[eliranwong]

check_regex.zip

Hi @T-Dynamos or Termux community,

I attached a simple python3 script and a sqlite in the zip file above.

Please run:

unzip check_regex.zip

cd check_regex

pip3 install apsw

python3 test_regex.py

It runs well on all non-Termux platforms, e.g. a macOS screenshot is attached below:

However, it does not work on Termux. I found that the issue appeared on Termux only.

[eliranwong]

Please also a note given by apsw developer: https://github.com/rogerbinns/apsw/issues/374#issuecomment-1276360678

He wrote, "... My guess would be problems with case insensitive stuff not working well in termux. (The python build process should also include a whole bunch of unicode data tables with take up a lot of space, which may not be happening.) ..."

[xtkoba]

The above example gives the same result on my GNU/Linux environment as on Termux:

$ python test_regex.py
[]

$ python --version
Python 3.10.8

$ python -m apsw.tests
                Python  /usr/lib/python-exec/python3.10/python sys.version_info(major=3, minor=10, micro=8, releaselevel='final', serial=0)
Testing with APSW file  /home/xxx/.local/lib/python3.10/site-packages/apsw/__init__.cpython-310-x86_64-linux-gnu.so
          APSW version  3.39.4.0
    SQLite lib version  3.39.4
SQLite headers version  3039004
    Using amalgamation  True
Not doing LoadExtension test.  You need to compile the extension first

  python3 setup.py build_test_extension
[...]

[eliranwong]

@xtkoba it is interesting. may I ask what is your Linux distribution?

I tested on "Debian GNU/Linux 11 (bullseye)", it is working as expected:

[T-Dynamos]

Hmm this code is working on archlinux

[eliranwong]

Hmm this code is working on archlinux

Yes, I tested on archlinux too. just don't understand why Termux does not work.

[eliranwong]

bty, I also tested on CentOS too. It works as expected.

[xtkoba]

I'm using Gentoo GNU/Linux:

# emerge -pqv dev-lang/python:3.10
[ebuild   R   ] dev-lang/python-3.10.8  USE="ensurepip gdbm ncurses readline sqlite ssl xml -bluetooth -build -examples -hardened -libedit -lto -pgo -test -tk -verify-sig"

with apsw being installed from source through pip:

pip install --no-binary :all: apsw

[xtkoba]

Using system SQLite seems to be a workaround. I believe this is non-trivial and so I assume we are allowed to add a package for this (named python-apsw, PR #12408).

[eliranwong]

@xtkoba Thanks for the new package. May I ask when will it be available? I cannot locate it when I run

pkg install python-apsw