decimal.ConversionSyntax when running examples

[soulmerge]

Just wanted to give this suite a try, but when following the installation guide in the examples folder, I em encountering the folloing error after running docker-compose up:

admin_1  | [admin] Failed to detect schema version, assuming up-to-date schema                                                                     
admin_1  | [admin] Traceback (most recent call last):                                                                                              
admin_1  | [admin]   File "./main.py", line 20, in <module>                                                                                        
admin_1  | [admin]     from api.core import API  # Export to uwsgi server                                                                          
admin_1  | [admin]   File "./api/core.py", line 8, in <module>                                                                                     
admin_1  | [admin]     from orm import DB                                                                                                          
admin_1  | [admin]   File "./orm/__init__.py", line 164, in <module>                                                                               
admin_1  | [admin]     err = DB.testConnection(verbose=True)                                                                                       
admin_1  | [admin]   File "./orm/__init__.py", line 47, in testConnection                                                                          
admin_1  | [admin]     self.session.execute("SELECT 1 FROM DUAL")                                                                                  
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 163, in do                                          
admin_1  | [admin]     return getattr(self.registry(), name)(*args, **kwargs)                                                                      
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 1295, in execute                                    
admin_1  | [admin]     return self._connection_for_bind(bind, close_with_result=True).execute(                                                     
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 1151, in _connection_for_bind                       
admin_1  | [admin]     engine, execution_options                                                                                                   
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 433, in _connection_for_bind                        
admin_1  | [admin]     conn = bind._contextual_connect()                                                                                           
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 2302, in _contextual_connect                        
admin_1  | [admin]     self._wrap_pool_connect(self.pool.connect, None),                                                                           
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect                         
admin_1  | [admin]     return fn()                                                                                                                 
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 364, in connect                                       
admin_1  | [admin]     return _ConnectionFairy._checkout(self)                                                                                     
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout                                     
admin_1  | [admin]     fairy = _ConnectionRecord.checkout(pool)                                                                                    
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 495, in checkout                                      
admin_1  | [admin]     rec = pool._do_get()                                                                                                        
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/impl.py", line 140, in _do_get                               [31/628]
admin_1  | [admin]     self._dec_overflow()
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
admin_1  | [admin]     with_traceback=exc_tb,
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
admin_1  | [admin]     raise exception
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/impl.py", line 137, in _do_get
admin_1  | [admin]     return self._create_connection()
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
admin_1  | [admin]     return _ConnectionRecord(self)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
admin_1  | [admin]     self.__connect(first_connect_check=True)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/pool/base.py", line 666, in __connect
admin_1  | [admin]     ).exec_once_unless_exception(self.connection, self)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/event/attr.py", line 314, in exec_once_unless_exception
admin_1  | [admin]     self._exec_once_impl(True, *args, **kw)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/event/attr.py", line 285, in _exec_once_impl
admin_1  | [admin]     self(*args, **kw)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/event/attr.py", line 322, in __call__
admin_1  | [admin]     fn(*args, **kw)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 1406, in go
admin_1  | [admin]     return once_fn(*arg, **kw)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 199, in first_connect
admin_1  | [admin]     dialect.initialize(c)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 2624, in initialize
admin_1  | [admin]     self._detect_sql_mode(connection)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/dialects/mysql/base.py", line 3085, in _detect_sql_mode
admin_1  | [admin]     connection.execute("SHOW VARIABLES LIKE 'sql_mode'"),
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1003, in execute
admin_1  | [admin]     return self._execute_text(object_, multiparams, params)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1178, in _execute_text
admin_1  | [admin]     parameters,
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
admin_1  | [admin]     e, statement, parameters, cursor, context
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1514, in _handle_dbapi_exception
admin_1  | [admin]     util.raise_(exc_info[1], with_traceback=exc_info[2])
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
admin_1  | [admin]     raise exception
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
admin_1  | [admin]     cursor, statement, parameters, context
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 609, in do_execute
admin_1  | [admin]     cursor.execute(statement, parameters)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
admin_1  | [admin]     res = self._query(query)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 317, in _query
admin_1  | [admin]     self._post_get_result()
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 352, in _post_get_result
admin_1  | [admin]     self._rows = self._fetch_row(0)
admin_1  | [admin]   File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 325, in _fetch_row
admin_1  | [admin]     return self._result.fetch_row(size, self._fetch_type)
admin_1  | [admin] decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]
admin_1  | [admin] unable to load app 0 (mountpoint='') (callable not found or import error)
admin_1  | [admin] *** no app loaded. going in full dynamic mode ***
admin_1  | [admin] uWSGI running as root, you can use --uid/--gid/--chroot options
admin_1  | [admin] *** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** 
admin_1  | [admin] *** uWSGI is running in multiple interpreter mode ***
admin_1  | [admin] spawned uWSGI master process (pid: 104)
admin_1  | [admin] spawned uWSGI worker 1 (pid: 127, cores: 4)

The application seems to start, but all requests cause the following error in the logs, of course:

admin_1  | [admin] --- no python application found, check your startup logs for errors ---

I think the behavior was slightly different, when I had started everything for the first time. I think the application was starting up fine, but I was getting a slightly different error about decimal conversation. Not sure about that, though, as I don't have the logs any more.

Please let me know if I can provide further information :-)

[aep]

hmm this might just be bad instructions? the database is empty at this point and i cant see what is supposed to create anything in there.

edit: there's multiple issues here.

• the decimal.InvalidOperation thing goes away by using mariadb:10 instead of 11
• the startup sequence is fragile. apparantly the python thing doesnt retry later if the mysql db happens to be not ready
• the docs are missing a step somewhere. the database is empty so you cant actually login

[aep]

this commit removes gromox-dbop

https://github.com/grommunio/gromox-container/commit/af0e6969f52b6cac570c5091fe02edce76cf9b01

which according to this manual is required to setup the database

https://docs.grommunio.com/admin/manual_install.html#gromox-user-database

however, i cant get to work either. not sure if that's maybe the reason it was removed

[grammola]

Hi @aep, thank you for using our containers. I am unable to replicate the issue you have with decimal.InvalidOperation. That might be specific to your platform or installation. Please can you give me further details about your platform?

hmm this might just be bad instructions? the database is empty at this point and i cant see what is supposed to create anything in there.

edit: there's multiple issues here.

* the decimal.InvalidOperation thing goes away by using `mariadb:10` instead of 11

* the startup sequence is fragile. apparantly the python thing doesnt retry later if the mysql db happens to be not ready

* the docs are missing a step somewhere. the database is empty so you cant actually login

About the startup sequence, if you check the docker-compose example, the containers depend on the DB starting up before they start. That's a hard requirement. So, please ensure that you have the DB ready before the gromox-core container and grommunio-admin container starts up.

About the database, the gromox-core container populates the DB. So you need that container to be up and running before you can do anything useful.

The commit you refer to removes the db population from the grommunio-admin container.

Do you have the gromox-core container up and running?

[grammola]

@aep, as a side note, I would suggest that you still use the this container for now. This has all the services in one container.

We are still handling some volume and shared storage issues on the v0.3 where we split out the services

[aep]

I am unable to replicate the issue you have with decimal.InvalidOperation

docker pull mariadb might help to reproduce. the compose file doesnt specify a version, so its whatever was last pulled on your machine, which might be 10. 10 works fine. 11 does not

the containers depend on the DB starting up before they start.

service_started doesnt indicate if mariadb is ready or not. it just means docker has started the service. by random chance this somtimes works out. if the software is unable to retry connection, what we usually do is either use service_healthy https://stackoverflow.com/a/41854997/1993710 or just or wrap the initscript in a port wait https://unix.stackexchange.com/a/5279/566568

About the database, the gromox-core container populates the DB.

it didnt. could you tell me where it does that so i can try to figure out why that doesnt work?

thanks

[grammola]

docker pull mariadb might help to reproduce. the compose file doesnt specify a version, so its whatever was last pulled on your machine, which might be 10. 10 works fine. 11 does not

Great catch, thank you. I will check this out.

service_started doesnt indicate if mariadb is ready or not. it just means docker has started the service. by random chance this somtimes works out. if the software is unable to retry connection, what we usually do is either use service_healthy https://stackoverflow.com/a/41854997/1993710 or just or wrap the initscript in a port wait https://unix.stackexchange.com/a/5279/566568

Thank you for the information here and we are aware of it. The docker compose setup is an example recipe. We noted that you may need to make changes to suit your environment / use case.

it didnt. could you tell me where it does that so i can try to figure out why that doesnt work?

The setup for the db is here. I found the issue that might have caused it. Some changes in v0.2 affected the dependencies. I will fix it today

[grammola]

A couple of new changes will hit the repo later today. Please use the v0.1 for now as I suggested. The new version will be out tomorrow or over the weekend

Edit: @aep There's an update now. It will reach this repo tomorrow but the images on dockerhub are already updated. You should not have the previous problems with the split containers. I am still testing so please feel free to let me know whatever issues you notice.

[grammola]

The issues pointed out here have been fixed. I will close the issue now. Please feel free to open another issue if anything surfaces again