Problem with Sqlalchemy 1.3.[0-4]

[thiagotei]

When using the SQLAlchemy version 1.3.4 and Python 2.7 I've got the following error:

(sqlalchemy.exc.InvalidRequestError) Can't resolve value for column configuration.id on object <Configuration at 0x7fe34b04f610>; no value has been set for this column
[SQL: SELECT count(*) AS count_1
FROM (SELECT result.id AS result_id, result.configuration_id AS result_configuration_id, result.machine_id AS result_machine_id, result.input_id AS result_input_id, result.tuning_run_id AS result_tuning_run_id, result.collection_date AS result_collection_date, result.collection_cost AS result_collection_cost, result.state AS result_state, result.time AS result_time, result.accuracy AS result_accuracy, result.energy AS result_energy, result.size AS result_size, result.confidence AS result_confidence, result.was_new_best AS result_was_new_best
FROM result
WHERE ? = result.tuning_run_id AND ? = result.configuration_id) AS anon_1]
[parameters: [{}]]
Traceback (most recent call last):
  File "/home/thiago/Documents/ice-personal/icelib/ice.py", line 406, in run
    self.searchtool.search(origmetric, origres)
  File "/home/thiago/Documents/ice-personal/icelib/tools/search/opentunertool_locus.py", line 334, in search
    oh = self._trm.main()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/opentuner-0.8.1-py2.7.egg/opentuner/tuningrunmain.py", line 201, in main
    self.search_driver.main()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/opentuner-0.8.1-py2.7.egg/opentuner/search/driver.py", line 274, in main
    if self.run_generation_techniques() > 0:
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/opentuner-0.8.1-py2.7.egg/opentuner/search/driver.py", line 172, in run_generation_techniques
    dr = self.root_technique.desired_result()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/opentuner-0.8.1-py2.7.egg/opentuner/search/metatechniques.py", line 43, in desired_result
    dr = technique.desired_result()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/opentuner-0.8.1-py2.7.egg/opentuner/search/technique.py", line 93, in desired_result
    cfg = self.desired_configuration()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/opentuner-0.8.1-py2.7.egg/opentuner/search/technique.py", line 201, in desired_configuration
    return next(self.gen)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/future-0.17.1-py2.7.egg/future/builtins/newnext.py", line 59, in newnext
    return iterator.next()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/opentuner-0.8.1-py2.7.egg/opentuner/search/technique.py", line 263, in call_main_generator
    if not self.driver.has_results(p):
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/opentuner-0.8.1-py2.7.egg/opentuner/search/driver.py", line 158, in has_results
    return self.results_query(config=config).count() > 0
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3567, in count
    return self.from_self(col).scalar()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3311, in scalar
    ret = self.one()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3281, in one
    ret = self.one_or_none()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3250, in one_or_none
    ret = list(self)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3323, in __iter__
    return self._execute_and_instances(context)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 3348, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
    distilled_params,
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    e, util.text_type(statement), parameters, None, None
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 681, in _init_compiled
    for grp, m in enumerate(parameters)
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/sql/compiler.py", line 692, in construct_params
    pd[self.bind_names[bindparam]] = bindparam.effective_value
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 1210, in effective_value
    return self.callable()
  File "/home/thiago/Documents/pyvirtenvs/envpy2.7/local/lib/python2.7/site-packages/sqlalchemy/orm/relationships.py", line 1587, in _go
    % (column, state_str(state))
StatementError: (sqlalchemy.exc.InvalidRequestError) Can't resolve value for column configuration.id on object <Configuration at 0x7fe34b04f610>; no value has been set for this column
[SQL: SELECT count(*) AS count_1
FROM (SELECT result.id AS result_id, result.configuration_id AS result_configuration_id, result.machine_id AS result_machine_id, result.input_id AS result_input_id, result.tuning_run_id AS result_tuning_run_id, result.collection_date AS result_collection_date, result.collection_cost AS result_collection_cost, result.state AS result_state, result.time AS result_time, result.accuracy AS result_accuracy, result.energy AS result_energy, result.size AS result_size, result.confidence AS result_confidence, result.was_new_best AS result_was_new_best
FROM result
WHERE ? = result.tuning_run_id AND ? = result.configuration_id) AS anon_1]

I printed the config.id inside opentuner/driverbase.py and it was None. I do not have any issues with Sqlalchemy version 1.2.[0-19].

--Thiago

[jansel]

I believe the issue might be similar to this one: https://stackoverflow.com/questions/620610/sqlalchemy-obtain-primary-key-with-autoincrement-before-commit/620831

If you add a session.flush() does it get an id?

[thiagotei]

I've added a self.session.flush() at the beginning of the results_query function and the execution finished normally. I'm not sure if I added it to the right place, but looks like you nailed the issue.

[jansel]

Great, feel free to submit a PR with the fix.

[qris]

If you end up here searching for this error message, it's likely that you're querying a relationship for objects that have not yet been added to a session, so they have no primary key. This is newly an error in SQLAlchemy 1.3; previously it would just return no results. See https://github.com/abrt/faf/issues/764#issuecomment-561702212 for details.