search

tink-crypto / tink-py

Python implementation of Tink
https://developers.google.com/tink
Apache License 2.0
33 stars 2 forks source link

Tink Python 1.9 Can't Get Mutex Lock on Import #25

Open tdipadova3rd opened 6 months ago

tdipadova3rd commented 6 months ago

Help us help you

We'd like to know more about your Tink deployment.

Describe the bug:

When importing Tink (with GCPSM Extension) for use in a FastAPI service using Uvicorn, we consistently see the error libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument, and the entire python process will crash. Pinning Tink to 1.8 solves this for us. We also have a Dramatiq service that throws the same error on Tink import. This happens on both Mac (M1/M2) and Linux (Ubuntu) machines in the Cloud.

What was the expected behavior?

We should be able to import Tink for use in Dramatiq and FastAPI services.

How can we reproduce the bug?

This can be triggered by importing certain other packages, including but not limited to, snowflake-connector-python.

Do you have any debugging information?

I have attached a crash log. crash-report.txt

What version of Tink are you using?

Tink 1.9.0. Tink 1.8.0 is not affected by this issue.

Can you tell us more about your development environment?

Python 3.11, Virtual Environment managed by Poetry. Mac OS 13 (M2), Mac OS 14 (M1), Ubuntu.

Is there anything else you'd like to add?

Provide any additional context about the problem.

morambro commented 6 months ago

Hi @tdipadova3rd, could you please provide a minimum code example to reproduce this? Is it sufficient to have import dramatiq and import tink in the same script and e.g., use tink within a dramatique task?

tdipadova3rd commented 6 months ago

Hi @morambro, thanks for the quick response!

I am able to repro with just two lines of code (test.py):

import snowflake.connector
import tink

via poetry run python test.py

The error occurs on import tink, killing the process.

Note: This also happens with some other packages, the snowflake.connector is just the most recent example.

I've attached a poetry export as well to give you a sense of the other packages in the environment. repro-requirements.txt

morambro commented 6 months ago

Thanks! I tried to reproduce the issue without success. My attempt on a Ubuntu 2004 VM (e2-standard-4):

# Python 3.11.7.
poetry new tink-py-issue-25
cd tink-py-issue-25
poetry add tink==1.9.0
poetry add snowflake-connector-python  # installs snowflake-connector-python==3.6.0 
cat <<EOF > test.py
import snowflake.connector
import tink
EOF
poetry run python test.py

The above runs fine.

UPDATE: Works fine also on a t2a-standard-4 (ARM) Ubuntu 2310 VM

tdipadova3rd commented 6 months ago

Hmmm perhaps there's another library in our environment that causes the tink import to fail? It's curious that I can trigger the issue from those two lines of code with the linked Python environment from two separate machines (Mac M1 and Mac M2).

mallport commented 4 months ago

Hey! I also have the same issue, and after a bit of dependency-mining, I've narrowed the source (in our case) down to the case where both pyarrow and tink>1.9.0 is imported. The error can be reproduced as follows, following the same structure as @morambro.

# Python 3.11.6.
poetry new tink-py-issue-25
cd tink-py-issue-25
poetry add tink==1.9.0
poetry add pyarrow  # installs pyarrow==15.0.0
cat <<EOF > test.py
import pyarrow
import tink
EOF
poetry run python test.py

Ran on a M2 Mac

mallport commented 4 months ago

I see this issue is also brought up in the Arrow-library. Link: https://github.com/apache/arrow/issues/40088