sqlalchemy.exc.ResourceClosedError when running hypersearch.py

[TalhaAsmal]

Hi,

I get the following error and stacktrace when running hypersearch.py. I'm using Python 3.6.4 64-bit on Windows 10 with Postgres 10.1.3 64-bit. I've installed all the requirements in the requirements.txt file with the same version as specified, but I still get the following error.

2018-02-02 22:59:09.443734: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX 2018-02-02 22:59:09.708364: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1105] Found device 0 with properties: name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.7845 pciBusID: 0000:01:00.0 totalMemory: 8.00GiB freeMemory: 6.64GiB 2018-02-02 22:59:09.708511: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1) row_count: 1574274 Traceback (most recent call last): File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1112, in _execute_context conn = self.__connection AttributeError: 'Connection' object has no attribute '_Connection__connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1114, in _execute_context conn = self._revalidate_connection() File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 429, in _revalidate_connection raise exc.ResourceClosedError("This Connection is closed") sqlalchemy.exc.ResourceClosedError: This Connection is closed

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "hypersearch.py", line 775, in main() File "hypersearch.py", line 771, in main y_list=Y File "I:\toolkits\tforce_btc_trader_clean\gp.py", line 193, in bayesian_optimisation2 y_list.append(loss_fn(params)) File "hypersearch.py", line 722, in loss_fn reward = hsearch.execute(vec2hypers(params)) File "hypersearch.py", line 549, in execute env.train_and_test(agent, self.cli_args.n_steps, self.cli_args.n_tests, -1) File "I:\toolkits\tforce_btc_trader_clean\btc_env.py", line 466, in train_and_test self.use_dataset(Mode.TEST) File "I:\toolkits\tforce_btc_trader_clean\btc_env.py", line 273, in use_dataset df = data.db_to_dataframe(self.conn, limit=limit, offset=offset, arbitrage=self.hypers.arbitrage) File "I:\toolkits\tforce_btc_trader_clean\data\data.py", line 201, in _db_to_dataframe_main df = pd.read_sql_query(query, conn).iloc[::-1] File "i:\python_venv\tforce_btc_trader\lib\site-packages\pandas\io\sql.py", line 332, in read_sql_query parse_dates=parse_dates, chunksize=chunksize) File "i:\python_venv\tforce_btc_trader\lib\site-packages\pandas\io\sql.py", line 1087, in read_query result = self.execute(args) File "i:\python_venv\tforce_btc_trader\lib\site-packages\pandas\io\sql.py", line 978, in execute return self.connectable.execute(args, **kwargs) File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 939, in execute return self._execute_text(object, multiparams, params) File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1097, in _execute_text statement, parameters File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1121, in _execute_context None, None) File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1402, in _handle_dbapi_exception exc_info File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\util\compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\util\compat.py", line 186, in reraise raise value.with_traceback(tb) File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1114, in _execute_context conn = self._revalidate_connection() File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 429, in _revalidate_connection raise exc.ResourceClosedError("This Connection is closed") sqlalchemy.exc.StatementError: (sqlalchemy.exc.ResourceClosedError) This Connection is closed [SQL: 'select coinbase.open as coinbase_open, coinbase.high as coinbase_high, coinbase.low as coinbase_low, coinbase.close as coinbase_close, coinbase.volume_btc as coinbase_volume_btc, coinbase.volume_currency as coinbase_volume_currency, coinbase.weighted_price as coinbase_weighted_price, coincheck.open as coincheck_open, coincheck.high as coincheck_high, coincheck.low as coincheck_low, coincheck.close as coincheck_close, coincheck.volume_btc as coincheck_volume_btc, coincheck.volume_currency as coincheck_volume_currency, coincheck.weighted_price as coincheck_weighted_price from coinbase\n left join lateral (\n select open, high, low, close, volume_btc, volume_currency, weighted_price\n from coincheck\n where coincheck.timestamp <= coinbase.timestamp\n order by coincheck.timestamp desc\n limit 1 \n ) coincheck on true\n order by coinbase.timestamp desc limit 157427 offset 1416846']

Is this a postgres version issue, or something related to windows? It seems to happen when doing the Select statement for the test data.

[lefnire]

Hmm... it seems like the database connection connects, but then closes. Can you make sure your tensorforce requirement is at #master? (currently their master is https://github.com/reinforceio/tensorforce/commit/f1c49de35633ad5b6866834200df75bd2b2dfc1c; I've specifically got my eye on this commit https://github.com/reinforceio/tensorforce/commit/f0876f55e2cfb0af789479237bed7d8fe66df515 not being present for your case)

What I would do is:

• uninstall tensorforce first pip uninstall tensorforce
• then git clone-it somewhere git clone https://github.com/reinforceio/tensorforce.git
• then pip install -e . from within the cloned dir

Incidentally, while I don't think this issue is Windows at all, we'll keep an eye on how far you can get with Win & this project. I've seen some heart-ache w/ the various deps. I'd dual-boot Ubuntu

[TalhaAsmal]

That worked perfectly, thanks! I'm training on windows, GTX 1070, --gpu-split 2 with my own dataset from bitfinex. Lets see how it goes