Questions on Acra, python and TLS

[soluzioninformatiche]

Hi,

I am having problems with latest release of Acra and I don't understand how to troubleshoot that problem. Print is working (empty) but I cannot submit the data into the database with python examples. I am using Ubuntu 20.04.4 LTS.

Error output: (I cannot bold/code style it so I will make "----" where code starts and where ends)

---

ubuntu@Gallardo:~/acra$ python3 examples/python/example_without_zone.py --public_key=docker/.acrakeys/acra-server/1ec6f16c36e6a3f1a064d708cba5b1ca760ba5ae29d3d80aab67b043ca641255c21ddf8f0fb09a8de336d555b4b961953200c7125f745cff7ed577f7c4b555c4_storage.pub --db_user=${POSTGRES_USER} --db_password=${POSTGRES_PASSWORD} --host=127.0.0.1 --port=5432 --data="data1" insert data: data1 Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1719, in _execute_context context = constructor( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1091, in _init_compiled param = { File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1092, in key: processorskey File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1631, in process return impl_processor(process_param(value, dialect)) File "/home/ubuntu/.local/lib/python3.8/site-packages/acrawriter/sqlalchemy/init.py", line 31, in process_bind_param return create_acrastruct(value, self._public_key) File "/home/ubuntu/.local/lib/python3.8/site-packages/acrawriter/init.py", line 40, in create_acrastruct random_kp = GenerateKeyPair(KEY_PAIR_TYPE.EC) File "/home/ubuntu/.local/lib/python3.8/site-packages/pythemis/skeygen.py", line 42, in init if themis.themis_gen_ec_key_pair( File "/usr/lib/python3.8/ctypes/init.py", line 386, in getattr func = self.getitem(name) File "/usr/lib/python3.8/ctypes/init.py", line 391, in getitem func = self._FuncPtr((name_or_ordinal, self)) AttributeError: python3: undefined symbol: themis_gen_ec_key_pair

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "examples/python/example_without_zone.py", line 73, in write_data(args.data, connection) File "examples/python/example_without_zone.py", line 33, in write_data connection.execute( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1306, in execute return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection return connection._execute_clauseelement( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement ret = self._execute_context( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1725, in _execute_context self._handle_dbapi_exception( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapiexception util.raise( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_ raise exception File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1719, in _execute_context context = constructor( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1091, in _init_compiled param = { File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1092, in key: processorskey File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1631, in process return impl_processor(process_param(value, dialect)) File "/home/ubuntu/.local/lib/python3.8/site-packages/acrawriter/sqlalchemy/init.py", line 31, in process_bind_param return create_acrastruct(value, self._public_key) File "/home/ubuntu/.local/lib/python3.8/site-packages/acrawriter/init.py", line 40, in create_acrastruct random_kp = GenerateKeyPair(KEY_PAIR_TYPE.EC) File "/home/ubuntu/.local/lib/python3.8/site-packages/pythemis/skeygen.py", line 42, in init if themis.themis_gen_ec_key_pair( File "/usr/lib/python3.8/ctypes/init.py", line 386, in getattr func = self.getitem(name) File "/usr/lib/python3.8/ctypes/init.py", line 391, in getitem func = self._FuncPtr((name_or_ordinal, self)) sqlalchemy.exc.StatementError: (builtins.AttributeError) python3: undefined symbol: themis_gen_ec_key_pair [SQL: INSERT INTO test_example_without_zone (data, raw_data) VALUES (%(data)s, %(raw_data)s) RETURNING test_example_without_zone.id] [parameters: [{'data': b'data1', 'raw_data': 'data1'}]]

---

File used with command source to set environment variables both in Docker compose starting window and the example one:

export ACRA_SERVER_MASTER_KEY=$(echo -n "6EaxBnRwcNZnNyExInkrY0hC6Ahxz5pS" | base64) export ACRA_CONNECTOR_MASTER_KEY=$(echo -n "oPZktQOFIHuyOc65fnoBnh8wWFdixJly" | base64) export ACRA_CLIENT_ID="Testbed" export POSTGRES_DB="test" export POSTGRES_USER="test" export POSTGRES_PASSWORD="test" export MYSQL_ONETIME_PASSWORD="test" export MYSQL_ROOT_PASSWORD="test" export MYSQL_DATABASE="test" export MYSQL_USER="test" export MYSQL_PASSWORD="test" export ACRA_HTTPAUTH_USER=test export ACRA_HTTPAUTH_PASSWORD=test

I hope someone can help me to carry out this problem, Cheers.

[Lagovas]

Python wrapper can't find Themis C library. You should install Themis

[soluzioninformatiche]

It's working now, thanks.

[soluzioninformatiche]

Now when I try to connect directly with Acra I have another issue:

python3 examples/python/example_without_zone.py --public_key=docker/.acrakeys/acra-server/1ec6f16c36e6a3f1a064d708cba5b1ca760ba5ae29d3d80aab67b043ca641255c21ddf8f0fb09a8de336d555b4b961953200c7125f745cff7ed577f7c4b555c4_storage.pub --db_user=${POSTGRES_USER} --db_password=${POSTGRES_PASSWORD} --host=127.0.0.1 --port=9393 --print Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3212, in _wrap_pool_connect return fn() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 301, in connect return _ConnectionFairy._checkout(self) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 761, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 419, in checkout rec = pool._do_get() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 145, in _do_get self._decoverflow() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 142, in _do_get return self._create_connection() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 247, in _create_connection return ConnectionRecord(self) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 362, in init self.connect() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 605, in connect pool.logger.debug("Error on connect(): %s", e) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 599, in connect connection = pool._invoke_creator(self) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 578, in connect return dialect.connect(*cargs, *cparams) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 584, in connect return self.dbapi.connect(cargs, **cparams) File "/home/ubuntu/.local/lib/python3.8/site-packages/psycopg2/init__.py", line 127, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) psycopg2.OperationalError: SSL error: sslv3 alert bad certificate

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "examples/python/example_without_zone.py", line 67, in metadata.create_all(engine) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 4744, in create_all bind._run_ddl_visitor( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3078, in _run_ddl_visitor with self.begin() as conn: File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2994, in begin conn = self.connect(close_with_result=close_with_result) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3166, in connect return self._connection_cls(self, close_with_result=close_with_result) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 96, in init else engine.raw_connection() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3245, in raw_connection return self._wrap_pool_connect(self.pool.connect, _connection) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3215, in _wrap_pool_connect Connection._handle_dbapi_exception_noconnection( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2068, in _handle_dbapi_exceptionnoconnection util.raise( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3212, in _wrap_pool_connect return fn() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 301, in connect return _ConnectionFairy._checkout(self) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 761, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 419, in checkout rec = pool._do_get() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 145, in _do_get self._decoverflow() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 142, in _do_get return self._create_connection() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 247, in _create_connection return ConnectionRecord(self) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 362, in init self.connect() File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 605, in connect pool.logger.debug("Error on connect(): %s", e) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 599, in connect connection = pool._invoke_creator(self) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 578, in connect return dialect.connect(*cargs, *cparams) File "/home/ubuntu/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 584, in connect return self.dbapi.connect(cargs, **cparams) File "/home/ubuntu/.local/lib/python3.8/site-packages/psycopg2/init__.py", line 127, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL error: sslv3 alert bad certificate

(Background on this error at: http://sqlalche.me/e/14/e3q8)

[Lagovas]

Hm, it looks like Acra requires TLS connection with mutual authentication (default behavior) but python client doesn't establish such a connection. And unfortunately, I didn't find examples in our documentation how to configure our example to use TLS as required. I can suggest two ways:

1. You can try our engineering demo that relies on this python example - https://github.com/cossacklabs/acra-engineering-demo/#example-6-client-side-encryption-zones-python-app-postgresql and do what it need and all this steps
2. You can configure it by yourself as it did in example. Python example can be configured via CLI args or env variables. Example of vairables you can find out in engineering demo 2.1 You can use already configured postgresql with TLS certs which we use for tests: cossacklabs/postgresql-ssl:11-1 docker image 2.2. Use our already generated TLS keys used for tests to pass it to python example. You can find out them in the repository in tests/ssl folder. But you need generate keys with AcraKeymaker using this TLS certificate: acra-keymaker --keystore=v1 --client_id="" --tls_cert=tests/ssl/acra-client/acra-client.crt

[soluzioninformatiche]

I tried the option 1 and now is working! I just have another question: when I try to login into the database the provided credentials are working but when I try to connect to the database with the user "postgres" and the password "test" pgadmin throws me an error meaning that the password is incorrect. Do you know the password?

Thanks again :-)

[Lagovas]

https://github.com/cossacklabs/acra-engineering-demo/blob/master/python/docker-compose.python.yml#L153

username: test@test.test password: test Pgadmin has own user for login, as I remember

[soluzioninformatiche]

I had what I asked for, thanks.