sqlalchemy OperationalError: no such table: oils

[jajcayn]

Hi, I installed the OilLibrary on CentOS 7 into virtualenv (remote machine, sshed) with $venv/bin/python setup.py develop throwing no error and everything looks fine. but when I try the example: from oil_library import get_oil oil_obj = get_oil('BAHIA') it gives me the following error: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "oil_library/factory.py", line 70, in get_oil oil = session.query(Oil).filter(Oil.name == oil_).one() File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2749, in one ret = self.one_or_none() File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2719, in one_or_none ret = list(self) File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2790, in __iter__ return self._execute_and_instances(context) File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2813, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute return meth(self, multiparams, params) File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement compiled_sql, distilled_params File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context context) File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception exc_info File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context context) File "/root/PyGnome/venv/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.OperationalError: (pysqlite2.dbapi2.OperationalError) no such table: oils [SQL: u'SELECT oils.id AS oils_id, oils.imported_record_id AS oils_imported_record_id, oils.estimated_id AS oils_estimated_id, oils.name AS oils_name, oils.adios_oil_id AS oils_adios_oil_id, oils.api AS oils_api, oils.oil_water_interfacial_tension_n_m AS oils_oil_water_interfacial_tension_n_m, oils.oil_water_interfacial_tension_ref_temp_k AS oils_oil_water_interfacial_tension_ref_temp_k, oils.oil_seawater_interfacial_tension_n_m AS oils_oil_seawater_interfacial_tension_n_m, oils.oil_seawater_interfacial_tension_ref_temp_k AS oils_oil_seawater_interfacial_tension_ref_temp_k, oils.pour_point_min_k AS oils_pour_point_min_k, oils.pour_point_max_k AS oils_pour_point_max_k, oils.flash_point_min_k AS oils_flash_point_min_k, oils.flash_point_max_k AS oils_flash_point_max_k, oils.emulsion_water_fraction_max AS oils_emulsion_water_fraction_max, oils.bullwinkle_time AS oils_bullwinkle_time, oils.bullwinkle_fraction AS oils_bullwinkle_fraction, oils.adhesion_kg_m_2 AS oils_adhesion_kg_m_2, oils.solubility AS oils_solubility, oils.k0y AS oils_k0y, oils.nickel_ppm AS oils_nickel_ppm, oils.vanadium_ppm AS oils_vanadium_ppm, oils.saturates_fraction AS oils_saturates_fraction, oils.aromatics_fraction AS oils_aromatics_fraction, oils.resins_fraction AS oils_resins_fraction, oils.asphaltenes_fraction AS oils_asphaltenes_fraction, oils.polars_fraction AS oils_polars_fraction, oils.benzene_fraction AS oils_benzene_fraction, oils.sulphur_fraction AS oils_sulphur_fraction, oils.paraffins_fraction AS oils_paraffins_fraction, oils.wax_content_fraction AS oils_wax_content_fraction, oils.naphthenes_fraction AS oils_naphthenes_fraction \nFROM oils \nWHERE oils.name = ?'] [parameters: ('BAHIA',)]

not sure what is wrong, since I installed the OilLibrary yesterday into my laptop (macOS) and everything run smoothly

thanks, best

nikola

[dharanidharan12]

While using sudo python setup.py develop ImportError: cannot import name RelationshipProperty OilLibrary database generation returned: 1

I tried to install OilLibrary, but sqlalchemy providing error like cannot import name RelationshipProperty

While using sudo python setup.py install Getting error like: from sqlalchemy.orm.relationships import (RelationshipProperty, ImportError: cannot import name RelationshipProperty

How to solve this issue.. please Help!!!

[ChrisBarker-NOAA]

How are you installing the dependencies? in particular, SQLAlchemy?

This looks like an SQLAlchemy problem. Does it come with any tests? can you run them to make sure it is installed and working properly first?

-CHB

[jajcayn]

Dear Chris,

thanks a lot for your reply. I installed all the dependencies in virtualenv using pip, hence pip install SQLAlchemy.. unfortunately, the SQLAlchemy doesn't provide any testing suite, at least not the one I am aware of.

nevertheless, some googling pointed me that the most probable problem is path to database, some forums mentioning that one needs to use the full path (are you?) and that the path is case sensitive..

in any case the db seems to be there: (venv) [root@ims OilLibrary]# find /root/PyGnome/ -name '*.db' /root/PyGnome/py_gnome/OilLibrary/oil_library/OilLib.db /root/PyGnome/venv/lib/python2.7/site-packages/oil_library/OilLib.db

and as I was saying, just a few hours before I installed the OilLibrary on my personal laptop (macOS) without any problems the same way (virtualnenv, install all deps through pip in venv a and then install OilLib from source).. this has to be some weird problem regarding the difference between OSes...

thanks!

nikola

[ChrisBarker-NOAA]

Nikola,

I think we are very close here -- and I think you are right -- the path to the DB may be getting confused.

The "trick" here is that we are being "clever" and having the package build and install the DB when the setup.py is run. But that gets tricky as to where the final library may be living. It works for us, and with conda environments, and perhaps isn't doing the right thing for virtual environments -- there are lot of possible combinations here.

However, very odd that it worked with virtualenvironment on OS-X .. I"d expect that to be the same.

I see that it didn't work with either "develop" or "install" -- so that's not the problem.

Why are you using sudo? with virtualenv, you generally install into your user dir -- so sudo should not be required -- not that I think that's the problem.

@JamesMakela: could you maybe tell Nikola where he can either:

hack the lib to have the right path to the db or tell him where to move the db so it will be found.

One nagging question, though -- wouldn't we get an error when it tried to opent eh DB? not when it tries to access something? makes me think we have a blank or corrupted DB ...

@jajcayn -- how large is the DB file at:

/root/PyGnome/py_gnome/OilLibrary/oil_library/OilLib.db /root/PyGnome/venv/lib/python2.7/site-packages/oil_library/OilLib.db

On my system, it's about 2.7MB.

[ChrisBarker-NOAA]

hmm, looking some more, I see, in oil_library/init.py:

_oillib_path = os.path.dirname(__file__)
__module_folder__ = __file__.split(os.sep)[-2]
_db_file = 'OilLib.db'
_db_file_path = os.path.join(_oillib_path, _db_file)

which means that it is looking in the oil_libary pacagke dir for the DB -- and it looks like you do have it there:

(venv) [root@ims OilLibrary]# find /root/PyGnome/ -name '*.db' /root/PyGnome/py_gnome/OilLibrary/oil_library/OilLib.db /root/PyGnome/venv/lib/python2.7/site-packages/oil_library/OilLib.db

However, it looks like you have too copies -- one in your PyGnome install (from develop mode install?) and one in the venv.

one start is to clean out one of those.

if you can import oil_libary, you can do:

import oil_library print oil_library.file

to see which one is being used.

[ChrisBarker-NOAA]

I just moved the OIlLib.db file from my install, and got exactly the same error.

So I do think that's it.

do you have read permissions on the file?

@James: we should probably add some check when the module is first imported, or when the DB is first accessed, that give a meaningful message if the DB file is not found.

-CHB

[jajcayn]

Dear Chris!

thanks a lot! not really sure what happened - I removed one of the libs (the one in site-packages), tried it again without success, then I tried remove all the folders with 'OilLibrary' in them from site-packages, also run python setup.py cleanall and then finally tried to install once again and it worked! I used install not develop and the example with BAHIA oil and other one is working like a charm..

Now I'll try to install PyGnome finally, thanks a lot for your help!

p.s.: I was not using sudo, it's just I am ssh-login as root, our company policy when using virtual machines.. don't know why.)