search

notadamking / RLTrader

A cryptocurrency trading environment using deep reinforcement learning and OpenAI's gym
https://discord.gg/ZZ7BGWh
GNU General Public License v3.0
1.73k stars 539 forks source link

Google colab Segmentation Fault Error after -!python cli.py optimize - command cell #129

Open glikoz opened 5 years ago

glikoz commented 5 years ago

I am newbie. I'm trying to use colab to execute your code. I'm getting this result after: !python cli.py optimize command. Thanks for help.

WARNING: Logging before flag parsing goes to stderr. 2019-07-25 22:41:31,271 - __main__ - initialize_data - DEBUG - Initialized Features: ['Date', 'Open', 'High', 'Low', 'Close', 'Volume'] WARNING: Logging before flag parsing goes to stderr. 2019-07-25 22:41:31,273 - __main__ - initialize_data - DEBUG - Initialized Features: ['Date', 'Open', 'High', 'Low', 'Close', 'Volume'] [I 2019-07-25 22:42:23,556] Using an existing study with name 'PPO2__MlpLnLstmPolicy__IncrementalProfit' instead of creating a new one. [I 2019-07-25 22:42:23,558] Using an existing study with name 'PPO2__MlpLnLstmPolicy__IncrementalProfit' instead of creating a new one. 2019-07-25 22:42:23,567 - __main__ - initialize_optuna - DEBUG - Initialized Optuna: 2019-07-25 22:42:23,569 - __main__ - initialize_optuna - DEBUG - Initialized Optuna: 2019-07-25 22:42:25,040 - __main__ - initialize_optuna - DEBUG - Best reward in (556) trials: -599.3695253077894 2019-07-25 22:42:25,040 - __main__ - __init__ - DEBUG - Initialize RLTrader: PPO2__MlpLnLstmPolicy__IncrementalProfit 2019-07-25 22:42:25,094 - __main__ - initialize_optuna - DEBUG - Best reward in (556) trials: -599.3695253077894 2019-07-25 22:42:25,094 - __main__ - __init__ - DEBUG - Initialize RLTrader: PPO2__MlpLnLstmPolicy__IncrementalProfit [W 2019-07-25 23:01:28,946] Setting status of trial#557 as TrialState.FAIL because of the following error: ValueError('The provided tag was already used for this event type',) Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/optuna/study.py", line 468, in _run_trial result = func(trial) File "/content/drive/My Drive/python/BTC1/lib/RLTrader.py", line 153, in optimize_params model.learn(n_steps_per_eval) File "/usr/local/lib/python3.6/dist-packages/stable_baselines/ppo2/ppo2.py", line 307, in learn writer=writer, states=mb_states)) File "/usr/local/lib/python3.6/dist-packages/stable_baselines/ppo2/ppo2.py", line 245, in _train_step writer.add_run_metadata(run_metadata, 'step%d' % (update * update_fac)) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/summary/writer/writer.py", line 264, in add_run_metadata raise ValueError("The provided tag was already used for this event type") ValueError: The provided tag was already used for this event type 2019-07-25 23:01:38,359 - __main__ - optimize - INFO - Finished trials: 558 2019-07-25 23:01:38,633 - __main__ - optimize - INFO - Best trial: -599.3695253077894 2019-07-25 23:01:38,633 - __main__ - optimize - INFO - Params: 2019-07-25 23:01:38,895 - __main__ - optimize - INFO - cliprange: 0.13144434574199731 2019-07-25 23:01:38,895 - __main__ - optimize - INFO - ent_coef: 6.845011861174882e-05 2019-07-25 23:01:38,895 - __main__ - optimize - INFO - gamma: 0.9686739342197893 2019-07-25 23:01:38,895 - __main__ - optimize - INFO - lam: 0.8772703435238995 2019-07-25 23:01:38,895 - __main__ - optimize - INFO - learning_rate: 0.005528318256310889 2019-07-25 23:01:38,895 - __main__ - optimize - INFO - n_steps: 311.0406063281854 2019-07-25 23:01:38,896 - __main__ - optimize - INFO - noptepochs: 1.0591429310422293 **[b6a85346cae0:00495] *** Process received signal *** **[b6a85346cae0:00495] Signal: Segmentation fault (11) [b6a85346cae0:00495] Signal code: Address not mapped (1) [b6a85346cae0:00495] Failing at address: 0x7fe8fd4a220d [b6a85346cae0:00495] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fe900757890] [b6a85346cae0:00495] [ 1] /lib/x86_64-linux-gnu/libc.so.6(getenv+0xa5)[0x7fe900396785] [b6a85346cae0:00495] [ 2] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(_ZN13TCMallocGuardD1Ev+0x34)[0x7fe900c01e44] [b6a85346cae0:00495] [ 3] /lib/x86_64-linux-gnu/libc.so.6(__cxa_finalize+0xf5)[0x7fe900397615] [b6a85346cae0:00495] [ 4] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(+0x13cb3)[0x7fe900bffcb3] [b6a85346cae0:00495] *** End of error message ******* WARNING: Logging before flag parsing goes to stderr. 2019-07-25 23:01:49,494 - __main__ - initialize_data - DEBUG - Initialized Features: ['Date', 'Open', 'High', 'Low', 'Close', 'Volume'] [I 2019-07-25 23:02:28,755] Using an existing study with name 'PPO2__MlpLnLstmPolicy__IncrementalProfit' instead of creating a new one. 2019-07-25 23:02:28,763 - __main__ - initialize_optuna - DEBUG - Initialized Optuna: 2019-07-25 23:02:30,123 - __main__ - initialize_optuna - DEBUG - Best reward in (558) trials: -599.3695253077894 2019-07-25 23:02:30,255 - __main__ - __init__ - DEBUG - Initialize RLTrader: PPO2__MlpLnLstmPolicy__IncrementalProfit [b6a85346cae0:00956] *** Process received signal *** [b6a85346cae0:00956] Signal: Segmentation fault (11) [b6a85346cae0:00956] Signal code: Address not mapped (1) [b6a85346cae0:00956] Failing at address: 0x7f316882a20d [b6a85346cae0:00956] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7f316badf890] [b6a85346cae0:00956] [ 1] /lib/x86_64-linux-gnu/libc.so.6(getenv+0xa5)[0x7f316b71e785] [b6a85346cae0:00956] [ 2] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(_ZN13TCMallocGuardD1Ev+0x34)[0x7f316bf89e44] [b6a85346cae0:00956] [ 3] /lib/x86_64-linux-gnu/libc.so.6(__cxa_finalize+0xf5)[0x7f316b71f615] [b6a85346cae0:00956] [ 4] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(+0x13cb3)[0x7f316bf87cb3] [b6a85346cae0:00956] *** End of error message ***

cyberslam commented 5 years ago

My Colab install was successful:

.... Successfully built optuna ta empyrical quantstats alembic yfinance Mako idna-ssl pyperclip Installing collected packages: pbr, stevedore, pyperclip, colorama, cmd2, cliff, colorlog, Mako, python-editor, alembic, optuna, ta, statsmodels, empyrical, multidict, yarl, pycares, aiodns, async-timeout, typing-extensions, idna-ssl, aiohttp, asn1crypto, cryptography, ccxt, configparser, yfinance, quantstats, tensorflow-gpu Found existing installation: statsmodels 0.10.0 Uninstalling statsmodels-0.10.0: Successfully uninstalled statsmodels-0.10.0 Successfully installed Mako-1.0.14 aiodns-1.1.1 aiohttp-3.5.4 alembic-1.0.11 asn1crypto-0.24.0 async-timeout-3.0.1 ccxt-1.18.988 cliff-2.15.0 cmd2-0.9.15 colorama-0.4.1 colorlog-4.0.2 configparser-3.7.4 cryptography-2.7 empyrical-0.5.0 idna-ssl-1.1.0 multidict-4.5.2 optuna-0.13.0 pbr-5.4.1 pycares-3.0.0 pyperclip-1.7.0 python-editor-1.0.4 quantstats-0.0.17 statsmodels-0.10.0rc2 stevedore-1.30.1 ta-0.4.5 tensorflow-gpu-1.14.0 typing-extensions-3.7.4 yarl-1.1.0 yfinance-0.1.43

then I used the following command to launch "optimize":

!python ./cli.py optimize

the following was the last bit of the error print-out:

..... .....

| approxkl | 1.2992781e-15 | | clipfrac | 0.0 | | ep_rewmean | nan | | eplenmean | nan | | explained_variance | 0.00413 | | fps | 14 | | nupdates | 17 | | policy_entropy | 0.0051370724 | | policy_loss | 7.4659073e-07 | | serial_timesteps | 5950 | | time_elapsed | 352 | | total_timesteps | 5950 | | value_loss | 0.00065950956 |

| approxkl | 0.0 | | clipfrac | 0.0 | | ep_rewmean | nan | | eplenmean | nan | | explained_variance | -0.00316 | | fps | 15 | | nupdates | 15 | | policy_entropy | 2.776626e-09 | | policy_loss | -4.196167e-07 | | serial_timesteps | 5625 | | time_elapsed | 326 | | total_timesteps | 5625 | | value_loss | 0.0054322067 |

2019-07-28 00:23:04,076 - main - optimize_params - INFO - Pruning trial for not making any trades: 1 [I 2019-07-28 00:23:04,076] Setting status of trial#552 as TrialState.PRUNED. 2019-07-28 00:23:09,920 - main - optimize - INFO - Finished trials: 554 2019-07-28 00:23:10,169 - main - optimize - INFO - Best trial: -57.909999999999854 2019-07-28 00:23:10,170 - main - optimize - INFO - Params: 2019-07-28 00:23:10,468 - main - optimize - INFO - cliprange: 0.14951451376750177 2019-07-28 00:23:10,468 - main - optimize - INFO - ent_coef: 8.755803215177085e-06 2019-07-28 00:23:10,469 - main - optimize - INFO - gamma: 0.9024819584559397 2019-07-28 00:23:10,469 - main - optimize - INFO - lam: 0.8627452728582746 2019-07-28 00:23:10,469 - main - optimize - INFO - learning_rate: 0.0617342341331879 2019-07-28 00:23:10,469 - main - optimize - INFO - n_steps: 241.9399431970005 2019-07-28 00:23:10,469 - main - optimize - INFO - noptepochs: 14.955931188308773 [79d169cd1242:09793] Process received signal [79d169cd1242:09793] Signal: Segmentation fault (11) [79d169cd1242:09793] Signal code: Address not mapped (1) [79d169cd1242:09793] Failing at address: 0x7fc383fe220d [79d169cd1242:09793] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fc386c8c890] [79d169cd1242:09793] [ 1] /lib/x86_64-linux-gnu/libc.so.6(getenv+0xa5)[0x7fc3868cb785] [79d169cd1242:09793] [ 2] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(_ZN13TCMallocGuardD1Ev+0x34)[0x7fc387136e44] [79d169cd1242:09793] [ 3] /lib/x86_64-linux-gnu/libc.so.6(__cxa_finalize+0xf5)[0x7fc3868cc615] [79d169cd1242:09793] [ 4] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(+0x13cb3)[0x7fc387134cb3] [79d169cd1242:09793] End of error message

| approxkl | 0.0 | | clipfrac | 0.0 | | ep_rewmean | nan | | eplenmean | nan | | explained_variance | 0.0134 | | fps | 19 | | nupdates | 16 | | policy_entropy | 3.3155612e-09 | | policy_loss | -6.1670937e-07 | | serial_timesteps | 6000 | | time_elapsed | 350 | | total_timesteps | 6000 | | value_loss | 0.00019280183 |

2019-07-28 00:23:28,090 - main - optimize_params - INFO - Pruning trial for not making any trades: 1 [I 2019-07-28 00:23:28,090] Setting status of trial#553 as TrialState.PRUNED. 2019-07-28 00:23:31,960 - main - optimize - INFO - Finished trials: 554 2019-07-28 00:23:32,134 - main - optimize - INFO - Best trial: -57.909999999999854 2019-07-28 00:23:32,134 - main - optimize - INFO - Params: 2019-07-28 00:23:32,308 - main - optimize - INFO - cliprange: 0.14951451376750177 2019-07-28 00:23:32,308 - main - optimize - INFO - ent_coef: 8.755803215177085e-06 2019-07-28 00:23:32,308 - main - optimize - INFO - gamma: 0.9024819584559397 2019-07-28 00:23:32,308 - main - optimize - INFO - lam: 0.8627452728582746 2019-07-28 00:23:32,308 - main - optimize - INFO - learning_rate: 0.0617342341331879 2019-07-28 00:23:32,308 - main - optimize - INFO - n_steps: 241.9399431970005 2019-07-28 00:23:32,308 - main - optimize - INFO - noptepochs: 14.955931188308773 [79d169cd1242:09792] Process received signal [79d169cd1242:09792] Signal: Segmentation fault (11) [79d169cd1242:09792] Signal code: Address not mapped (1) [79d169cd1242:09792] Failing at address: 0x7ff93680320d [79d169cd1242:09792] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7ff9394ad890] [79d169cd1242:09792] [ 1] /lib/x86_64-linux-gnu/libc.so.6(getenv+0xa5)[0x7ff9390ec785] [79d169cd1242:09792] [ 2] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(_ZN13TCMallocGuardD1Ev+0x34)[0x7ff939957e44] [79d169cd1242:09792] [ 3] /lib/x86_64-linux-gnu/libc.so.6(cxa_finalize+0xf5)[0x7ff9390ed615] [79d169cd1242:09792] [ 4] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(+0x13cb3)[0x7ff939955cb3] [79d169cd1242:09792] End of error message WARNING: Logging before flag parsing goes to stderr. 2019-07-28 00:23:36,255 - main - initialize_data - DEBUG - Initialized Features: ['Date', 'Open', 'High', 'Low', 'Close', 'Volume'] [I 2019-07-28 00:24:12,288] Using an existing study with name 'PPO2MlpLnLstmPolicyIncrementalProfit' instead of creating a new one. 2019-07-28 00:24:12,292 - main - initialize_optuna - DEBUG - Initialized Optuna: 2019-07-28 00:24:13,278 - main - initialize_optuna - DEBUG - Best reward in (554) trials: -57.909999999999854 2019-07-28 00:24:13,407 - main - init - DEBUG - Initialize RLTrader: PPO2MlpLnLstmPolicyIncrementalProfit [79d169cd1242:09990] Process received signal [79d169cd1242:09990] Signal: Segmentation fault (11) [79d169cd1242:09990] Signal code: Address not mapped (1) [79d169cd1242:09990] Failing at address: 0x7fe6bc31520d [79d169cd1242:09990] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7fe6befbf890] [79d169cd1242:09990] [ 1] /lib/x86_64-linux-gnu/libc.so.6(getenv+0xa5)[0x7fe6bebfe785] [79d169cd1242:09990] [ 2] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(_ZN13TCMallocGuardD1Ev+0x34)[0x7fe6bf469e44] [79d169cd1242:09990] [ 3] /lib/x86_64-linux-gnu/libc.so.6(cxa_finalize+0xf5)[0x7fe6bebff615] [79d169cd1242:09990] [ 4] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(+0x13cb3)[0x7fe6bf467cb3] [79d169cd1242:09990] End of error message

glikoz commented 5 years ago

This is exactly same error, I hope someone can explain the reason behind it.

cyberslam commented 5 years ago

Hi Zhouti29,

            Thank you for your suggested change.  I just add your change

AFTER the !pip install -r requirements.txt and it start to run. However, I still get a segment fault error after running for a while with losses shown:

........ ........

------------------------------------ | approxkl | 0.082762145 | | clipfrac | 0.25561348 | | explained_variance | 0.00134 | | fps | 10 | | n_updates | 15 | | policy_entropy | 2.2614372 | | policy_loss | 0.010807981 | | serial_timesteps | 5160 | | time_elapsed | 428 | | total_timesteps | 5160 | | value_loss | 268293.97 | ------------------------------------ ----------------------------------- | approxkl | 0.13534357 | | clipfrac | 0.34583 | | explained_variance | 0.00263 | | fps | 10 | | n_updates | 16 | | policy_entropy | 2.0661743 | | policy_loss | 0.06681823 | | serial_timesteps | 5504 | | time_elapsed | 460 | | total_timesteps | 5504 | | value_loss | 506792.7 | ----------------------------------- [I 2019-08-08 17:40:13,937] Finished trial#553 resulted in value: -3022.860003117472. Current best value is -3022.860003117472 with parameters: {'cliprange': 0.34998638936790727, 'ent_coef': 2.7406115907349536e-08, 'gamma': 0.9662285172094842, 'lam': 0.8814985022852209, 'learning_rate': 0.007102979083909665, 'n_steps': 319.69330758702245, 'noptepochs': 27.575389394851317}. 2019-08-08 17:40:14,192 - main - optimize - INFO - Finished trials: 554 2019-08-08 17:40:14,433 - main - optimize - INFO - Best trial: -3022.860003117472 2019-08-08 17:40:14,433 - main - optimize - INFO - Params: 2019-08-08 17:40:14,671 - main - optimize - INFO - cliprange: 0.34998638936790727 2019-08-08 17:40:14,671 - main - optimize - INFO - ent_coef: 2.7406115907349536e-08 2019-08-08 17:40:14,671

What should I do to correct this?

Cheers,

Samuel

On Sun, Aug 4, 2019 at 4:15 PM zhouti29 notifications@github.com wrote:

!pip install stable-baselines --upgrade should solve the issue

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/notadamking/RLTrader/issues/129?email_source=notifications&email_token=ACGVJNWWLFDYFYMZOIAFM43QC5PHPA5CNFSM4IG7J5VKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3QLTFQ#issuecomment-518044054, or mute the thread https://github.com/notifications/unsubscribe-auth/ACGVJNW2THEXI345R3KK7YDQC5PHPANCNFSM4IG7J5VA .

zhouti29 commented 5 years ago

Hi Samuel, I have the same segmentation fault error after running the script for a while. That's why I deleted my solution posted earlier. Now I am running the RL on my local machine. Sorry to give you the misleading information.

cyberslam commented 5 years ago

No problem my friend.  Did you have better success when you use your local PC instead of Colab?Do you know that AdamKing started a new project at Githib called Tensortrade....  I am just running it on Colab - it's rehash of RLtrader..  SamuelSent from my Samsung Galaxy smartphone. -------- Original message --------From: zhouti29 notifications@github.com Date: 2019-08-08 11:04 a.m. (GMT-08:00) To: notadamking/RLTrader RLTrader@noreply.github.com Cc: cyberslam sflam168@gmail.com, Comment comment@noreply.github.com Subject: Re: [notadamking/RLTrader] Google colab Segmentation Fault Error after -!python cli.py optimize - command cell (#129) Hi Samuel, I have the same segmentation fault error after running the script for a while. That's why I deleted my solution posted earlier. Now I am running the RL on my local machine. Sorry to give you the misleading information.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread. [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/notadamking/RLTrader/issues/129?email_source=notifications\u0026email_token=ACGVJNVGG5YQQPCCAVO3X7DQDRNZJA5CNFSM4IG7J5VKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD34N7AQ#issuecomment-519626626", "url": "https://github.com/notadamking/RLTrader/issues/129?email_source=notifications\u0026email_token=ACGVJNVGG5YQQPCCAVO3X7DQDRNZJA5CNFSM4IG7J5VKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD34N7AQ#issuecomment-519626626", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

cyberslam commented 5 years ago

Success on Tensortrade: no error!!!Sent from my Samsung Galaxy smartphone. -------- Original message --------From: zhouti29 notifications@github.com Date: 2019-08-08 11:04 a.m. (GMT-08:00) To: notadamking/RLTrader RLTrader@noreply.github.com Cc: cyberslam sflam168@gmail.com, Comment comment@noreply.github.com Subject: Re: [notadamking/RLTrader] Google colab Segmentation Fault Error after -!python cli.py optimize - command cell (#129) Hi Samuel, I have the same segmentation fault error after running the script for a while. That's why I deleted my solution posted earlier. Now I am running the RL on my local machine. Sorry to give you the misleading information.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread. [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/notadamking/RLTrader/issues/129?email_source=notifications\u0026email_token=ACGVJNVGG5YQQPCCAVO3X7DQDRNZJA5CNFSM4IG7J5VKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD34N7AQ#issuecomment-519626626", "url": "https://github.com/notadamking/RLTrader/issues/129?email_source=notifications\u0026email_token=ACGVJNVGG5YQQPCCAVO3X7DQDRNZJA5CNFSM4IG7J5VKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD34N7AQ#issuecomment-519626626", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]