search

nateemma / strategies

Custom trading strategies using the freqtrade framework
331 stars 94 forks source link

Windows 11 installation #51

Open jenovauh opened 8 months ago

jenovauh commented 8 months ago

Can I know if anyone manage to install and run on windows 11 native or WSL?

jenovauh commented 8 months ago

I am getting these error, keras should be installed along with tensorflow but somehow still unable to detect the version.

2024-03-01 14:04:10,656 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance" 2024-03-01 14:04:13,743 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'... 2024-03-01 14:04:24,005 - tensorflow - WARNING - From C:\Users\nelson.kwek\Downloads\freqtrade.venv\lib\site-packages\keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

2024-03-01 14:04:24,458 - tensorflow - WARNING - From C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\CompressionAutoEncoder.py:43: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

C:\Users\nelson.kwek\Downloads\freqtrade.venv\lib\site-packages\statsforecast\utils.py:237: FutureWarning: 'M' is deprecated and will be removed in a future version, please use 'ME' instead. "ds": pd.date_range(start="1949-01-01", periods=len(AirPassengers), freq="M"),

NNTC

2024-03-01 14:04:49,051 - freqtrade - ERROR - Fatal exception! Traceback (most recent call last): File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\main.py", line 42, in main return_code = args'func' File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\commands\optimize_commands.py", line 58, in start_backtesting backtesting.start() File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\optimize\backtesting.py", line 1401, in start min_date, max_date = self.backtest_one_strategy(strat, data, timerange) File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\optimize\backtesting.py", line 1318, in backtest_one_strategy preprocessed = self.strategy.advise_all_indicators(data) File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\strategy\interface.py", line 1378, in advise_all_indicators return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\strategy\interface.py", line 1378, in return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\strategy\interface.py", line 1410, in advise_indicators return self.populate_indicators(dataframe, metadata) File "C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\NNTC.py", line 394, in populate_indicators Environment.print_environment() File "C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\Environment.py", line 79, in print_environment keras_version = tf.keras.version AttributeError: module 'keras.api._v2.keras' has no attribute 'version'

jenovauh commented 8 months ago

Can someone kind enough to 'pip list' for me so I can try to troubleshoot the issue on windows?

nateemma commented 8 months ago

Hmm, when I run it (on MacOS), I get the following output:

NNTC

Software Environment:

freqtrade:  2023.12-dev-fcc87399c

OS Type:    darwin (MacOS), Version: macOS-14.2.1-arm64-arm-64bit

python:     ['3.11.6 (main, Nov  2 2023, 04:39:43) [Clang 14.0.3

(clang-1403.0.22.14.1)]']

sklearn:    1.1.3

tensorflow: 2.15.0,

devices:[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

keras:      2.15.0

pytorch:    2.0.1

lightning:  2.1.2

darts:      0.27.0

Can you check that you have the latest version? The error you have indicates the following error:

File "C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\Environment.py", line 79, in print_environment keras_version = tf.keras.version AttributeError: module 'keras.api._v2.keras' has no attribute 'version'

However, the latest version has Environment.py moved to the uitils directory, and the corresponding line is:

keras_version = keras.version

Cheers,

Phil

On Thu, Feb 29, 2024 at 10:06 PM jenovauh @.***> wrote:

I am getting these error, keras should be installed along with tensorflow but somehow still unable to detect the version.

2024-03-01 14:04:10,656 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance" 2024-03-01 14:04:13,743 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'... 2024-03-01 14:04:24,005 - tensorflow - WARNING - From C:\Users\nelson.kwek\Downloads\freqtrade.venv\lib\site-packages\keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

2024-03-01 14:04:24,458 - tensorflow - WARNING - From C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\CompressionAutoEncoder.py:43: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

C:\Users\nelson.kwek\Downloads\freqtrade.venv\lib\site-packages\statsforecast\utils.py:237: FutureWarning: 'M' is deprecated and will be removed in a future version, please use 'ME' instead. "ds": pd.date_range(start="1949-01-01", periods=len(AirPassengers), freq="M"),

NNTC

2024-03-01 14:04:49,051 - freqtrade - ERROR - Fatal exception! Traceback (most recent call last): File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\main.py", line 42, in main return_code = args'func' http://args File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\commands\optimize_commands.py", line 58, in start_backtesting backtesting.start() File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\optimize\backtesting.py", line 1401, in start min_date, max_date = self.backtest_one_strategy(strat, data, timerange) File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\optimize\backtesting.py", line 1318, in backtest_one_strategy preprocessed = self.strategy.advise_all_indicators(data) File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\strategy\interface.py", line 1378, in advise_all_indicators return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\strategy\interface.py", line 1378, in return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "C:\Users\nelson.kwek\Downloads\freqtrade\freqtrade\strategy\interface.py", line 1410, in advise_indicators return self.populate_indicators(dataframe, metadata) File "C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\NNTC.py", line 394, in populate_indicators Environment.print_environment() File "C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\Environment.py", line 79, in print_environment keras_version = tf.keras.version AttributeError: module 'keras.api._v2.keras' has no attribute 'version'

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/51#issuecomment-1972570455, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD4X57YXD32O2NSGAWEURTYWALHRAVCNFSM6AAAAABEAMCLHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZSGU3TANBVGU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

jenovauh commented 8 months ago

I guess the issue is tensorflow 2.15.0 stop supporting on windows cmd and cause a lot of weird issue that cannot be resolved at my skill level. Test tensorflow-cpu also and it doesnt work properly. Test with windows WSL ubuntu also giving similiar error. I will try to figure out how to slowly fix the issue. Been testing daily installing and uninstall trying to resolve it.

jenovauh commented 8 months ago

Can you check that you have the latest version? The error you have indicates the following error: File "C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\Environment.py", line 79, in print_environment keras_version = tf.keras.version AttributeError: module 'keras.api._v2.keras' has no attribute 'version' However, the latest version has Environment.py moved to the uitils directory, and the corresponding line is: keras_version = keras.version Cheers, Phil

Do you mean i have to replace Environment.py from uitils folder to "C:\Users\nelson.kwek\Downloads\freqtrade\user_data\strategies\binance\Environment.py"?

jenovauh commented 8 months ago

Anyway after my recent reinstallation for freqtrade and your git repo. I am getting this new error.

2024-03-07 12:11:09,856 - freqtrade.resolvers.iresolver - WARNING - Could not import /home/jenovauh/freqtrade/user_data/strategies/binanceus/NNPredictor_Wavenet2.py due to 'No module named 'tf''>

So what i did was to remove NNPredictor_Wavenet2.py and see what happen next and i get this error

NNTC

2024-03-07 12:14:30,729 - freqtrade - ERROR - Fatal exception! Traceback (most recent call last): File "/home/jenovauh/freqtrade/freqtrade/main.py", line 42, in main return_code = args'func' File "/home/jenovauh/freqtrade/freqtrade/commands/optimize_commands.py", line 58, in start_backtesting backtesting.start() File "/home/jenovauh/freqtrade/freqtrade/optimize/backtesting.py", line 1401, in start min_date, max_date = self.backtest_one_strategy(strat, data, timerange) File "/home/jenovauh/freqtrade/freqtrade/optimize/backtesting.py", line 1318, in backtest_one_strategy preprocessed = self.strategy.advise_all_indicators(data) File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1378, in advise_all_indicators return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1378, in return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1410, in advise_indicators return self.populate_indicators(dataframe, metadata) File "/home/jenovauh/freqtrade/user_data/strategies/binance/NNTC.py", line 394, in populate_indicators Environment.print_environment() AttributeError: module 'Environment' has no attribute 'print_environment'>

nateemma commented 8 months ago

are you using the latest version? I re-organised the directories a long time ago. For example, NNTC.py should be in user_data/strategies/ and Environment.py should be in user_data/strategies/utils

If you moved them all into user_data/strategies/binance, then you need to change the import statements (because they look in the utils directory)

Cheers,

Phil

On Wed, Mar 6, 2024 at 8:15 PM jenovauh @.***> wrote:

Anyway after my recent reinstallation for freqtrade and your git repo. I am getting this new error.

2024-03-07 12:11:09,856 - freqtrade.resolvers.iresolver - WARNING - Could not import /home/jenovauh/freqtrade/user_data/strategies/binanceus/NNPredictor_Wavenet2.py due to 'No module named 'tf'' So what i did was to remove NNPredictor_Wavenet2.py and see what happen next and i get this error NNTC

2024-03-07 12:14:30,729 - freqtrade - ERROR - Fatal exception! Traceback (most recent call last): File "/home/jenovauh/freqtrade/freqtrade/main.py", line 42, in main return_code = args'func' http://args File "/home/jenovauh/freqtrade/freqtrade/commands/optimize_commands.py", line 58, in start_backtesting backtesting.start() File "/home/jenovauh/freqtrade/freqtrade/optimize/backtesting.py", line 1401, in start min_date, max_date = self.backtest_one_strategy(strat, data, timerange) File "/home/jenovauh/freqtrade/freqtrade/optimize/backtesting.py", line 1318, in backtest_one_strategy preprocessed = self.strategy.advise_all_indicators(data) File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1378, in advise_all_indicators return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1378, in return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1410, in advise_indicators return self.populate_indicators(dataframe, metadata) File "/home/jenovauh/freqtrade/user_data/strategies/binance/NNTC.py", line 394, in populate_indicators Environment.print_environment() AttributeError: module 'Environment' has no attribute 'print_environment'

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/51#issuecomment-1982317376, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD4X5YMAMCS2D35P6PRVCTYW7SXBAVCNFSM6AAAAABEAMCLHGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBSGMYTOMZXGY . You are receiving this because you commented.Message ID: @.***>

jenovauh commented 8 months ago

are you using the latest version? I re-organised the directories a long time ago. For example, NNTC.py should be in user_data/strategies/ and Environment.py should be in user_data/strategies/utils

If you moved them all into user_data/strategies/binance, then you need to change the import statements (because they look in the utils directory)

Yes using the lastest version, I am updating by using git pull. Maybe I will delete all and git clone again and test it out.

jenovauh commented 8 months ago

I had remove and git clone your strategies again. After that i am using "zsh user_data/strategies/scripts/test_strat.sh " which will be "zsh user_data/strategies/scripts/test_strat.sh --short binance NNTC" If I dont move all the NNTC py file it will give me error that NNTC.py is not found in user_data/strategies/binance

So i move all the NNTC files into user_data/strategies/binance It run fine at first until Model is already trained then crashed. Below are the error encounter, not sure if you have any idea how to solve this.

(.venv) (freq) jenovauh@TPSLTAFTS03:~/freqtrade$ zsh user_data/strategies/scripts/test_strat.sh --short binance NNTC

zsh: /home/jenovauh/anaconda3/envs/freq/lib/libtinfo.so.6: no version information available (required by zsh) 2024-03-08 12:13:22,105 - freqtrade - INFO - freqtrade 2024.3-dev-cab38fb8c Exchange (binance) detected - using legacy mode

Using config file: user_data/strategies/binance/config_binance_short.json and Strategy dir: user_data/strategies/binance

Fri Mar 8 12:13:23 +08 2024 Testing strategy:NNTC for exchange:_binance... freqtrade backtesting --cache none --breakdown month --timerange=20230910-20240308 -c user_data/strategies/binance/config_binance_short.json --strategy-path user_data/strategies/binance --strategy-list NNTC 2024-03-08 12:13:24,045 - freqtrade - INFO - freqtrade 2024.3-dev-cab38fb8c 2024-03-08 12:13:24,047 - freqtrade.configuration.load_config - INFO - Using config: user_data/strategies/binance/config_binance_short.json ... 2024-03-08 12:13:24,048 - freqtrade.loggers - INFO - Verbosity set to 0 2024-03-08 12:13:24,048 - freqtrade.configuration.configuration - INFO - Using additional Strategy lookup path: user_data/strategies/binance 2024-03-08 12:13:24,048 - freqtrade.configuration.configuration - INFO - Using max_open_trades: 10 ... 2024-03-08 12:13:24,048 - freqtrade.configuration.configuration - INFO - Parameter --timerange detected: 20230910-20240308 ... 2024-03-08 12:13:24,049 - freqtrade.configuration.configuration - INFO - Using user-data directory: /home/jenovauh/freqtrade/user_data ... 2024-03-08 12:13:24,049 - freqtrade.configuration.configuration - INFO - Using data directory: /home/jenovauh/freqtrade/user_data/data/binance ... 2024-03-08 12:13:24,049 - freqtrade.configuration.configuration - INFO - Using strategy list of 1 strategies 2024-03-08 12:13:24,049 - freqtrade.configuration.configuration - INFO - Parameter --breakdown detected ... 2024-03-08 12:13:24,049 - freqtrade.configuration.configuration - INFO - Parameter --cache=none detected ... 2024-03-08 12:13:24,049 - freqtrade.configuration.configuration - INFO - Filter trades by timerange: 20230910-20240308 2024-03-08 12:13:24,049 - freqtrade.exchange.check_exchange - INFO - Checking exchange... 2024-03-08 12:13:24,056 - freqtrade.exchange.check_exchange - INFO - Exchange "binance" is officially supported by the Freqtrade development team. 2024-03-08 12:13:24,056 - freqtrade.configuration.configuration - INFO - Using pairlist from configuration. 2024-03-08 12:13:24,056 - freqtrade.configuration.config_validation - INFO - Validating configuration ... 2024-03-08 12:13:24,058 - freqtrade.commands.optimize_commands - INFO - Starting freqtrade in Backtesting mode 2024-03-08 12:13:24,058 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled 2024-03-08 12:13:24,058 - freqtrade.exchange.exchange - INFO - Using CCXT 4.2.47 2024-03-08 12:13:24,058 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'swap'}, 'enableRateLimit': True} 2024-03-08 12:13:24,063 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'swap'}, 'enableRateLimit': True, 'rateLimit': 200} 2024-03-08 12:13:24,067 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance" 2024-03-08 12:13:26,692 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'...

NNTC

Software Environment:

freqtrade:  2024.3-dev-cab38fb8c
OS Type:    linux, Version: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
python:     ['3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0]']
sklearn:    1.4.1.post1
tensorflow: 2.15.0, devices:[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
keras:      2.15.0
pytorch:    2.2.0+cu121
lightning:  2.1.2
darts:      0.27.2

Strategy Parameters/Flags

Dataset Type:           DatasetType.DEFAULT (0)
Signal Type:            SignalType.Profit (profit)
Classifier Type:        ClassifierType.LSTM
Lookahead:              1.0 hours (12 candles)
n_profit_stddevs:       2.0
n_loss_stddevs:         2.0
compress_data:          True
refit_model:            False
model_per_pair:         False
combine_models:         False
ignore_exit_signals:    False

AVAX/USDT:USDT adding indicators... training models... Compressed data 100 -> 64 (features) holds:40536 (78.00%) buys:5485 (10.55%) sells:5948 (11.45%) training samples: 37407 #buys: 3713 (9.93 %) #sells: 3943 (10.54 %) Loading existing model (/home/jenovauh/freqtrade/user_data/strategies/binance/models/NNTC_profit_LSTM/NNTC_profit_LSTM.keras)... Model is already trained 2024-03-08 12:13:50,317 - freqtrade - ERROR - Fatal exception! Traceback (most recent call last): File "/home/jenovauh/freqtrade/freqtrade/main.py", line 42, in main return_code = args'func' File "/home/jenovauh/freqtrade/freqtrade/commands/optimize_commands.py", line 58, in start_backtesting backtesting.start() File "/home/jenovauh/freqtrade/freqtrade/optimize/backtesting.py", line 1401, in start min_date, max_date = self.backtest_one_strategy(strat, data, timerange) File "/home/jenovauh/freqtrade/freqtrade/optimize/backtesting.py", line 1318, in backtest_one_strategy preprocessed = self.strategy.advise_all_indicators(data) File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1378, in advise_all_indicators return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1378, in return {pair: self.advise_indicators(pair_data.copy(), {'pair': pair}).copy() File "/home/jenovauh/freqtrade/freqtrade/strategy/interface.py", line 1410, in advise_indicators return self.populate_indicators(dataframe, metadata) File "/home/jenovauh/freqtrade/user_data/strategies/binance/NNTC.py", line 432, in populate_indicators self.train_models(curr_pair, dataframe, buys, sells) File "/home/jenovauh/freqtrade/user_data/strategies/binance/NNTC.py", line 715, in train_models preds = self.get_classifier_predictions(clf, tsr_test) File "/home/jenovauh/freqtrade/user_data/strategies/binance/NNTC.py", line 836, in get_classifier_predictions predictions = classifier.predict(df_tensor) File "/home/jenovauh/freqtrade/user_data/strategies/utils/ClassifierKerasTrinary.py", line 278, in predict preds = self.model.predict(df_tensor, verbose=0) File "/home/jenovauh/freqtrade/.venv/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/jenovauh/freqtrade/.venv/lib/python3.10/site-packages/tensorflow/python/eager/execute.py", line 53, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, tensorflow.python.framework.errors_impl.UnknownError: Graph execution error:

Detected at node CudnnRNN defined at (most recent call last):

Fail to find the dnn implementation. [[{{node CudnnRNN}}]] [[NNTC_profit_LSTM/lstm/PartitionedCall]] [Op:__inference_predict_function_1057]
ExaByt3s commented 8 months ago

It seems to be a configuration error in your env for tensorflow and the nvidia toolkit, try installing

pip install tensorflow-cpu==2.15.0

jenovauh commented 8 months ago

It seems to be a configuration error in your env for tensorflow and the nvidia toolkit, try installing

pip install tensorflow-cpu==2.15.0

Yes I am pretty sure it is both of them as well. Trying to see which version doesnt cause these error. Sometime it works sometime it doesnt. Also will cause difference strategies having different error. I have tried at least 8 times to redo the env without much success. Still trying when i am free.