Folder structure complexity leading to bot malfunction on update

[joe-nano]

Hello Team,

Quick question, might sound dumb, but it's there a way to keep the folder structure less cluttered. There seem to be way too much confusion for some of us lesser folk. :)

So currently when I print path to folder, I get this. I have files populated within this folder but I can't the structure

root@JDCD:~/freqtrade/freqtrade# tree /path/to/folder /path/to/folder [error opening dir]

0 directories, 0 files

I do get something when I use the find . -type d | sed -e "s/[^-][^\/]*\// |/g" -e "s/|([^ ])/|-\1/" . |-strategy | |-pycache |-edge | |-pycache |-plot |-pairlist | |-pycache |-templates | |-subtemplates |-configuration | |-pycache |-pycache |-optimize |-commands | |-pycache |-resolvers | |-pycache |-data | |-history | | |-pycache | |-pycache |-exchange | |-pycache |-vendor | |-qtpylib | | |-pycache | |-pycache |-rpc | |-pycache |-user_data | |-notebooks | |-plot | |-hyperopt_results | |-strategies | | |-pycache | |-hyperopts | |-data | | |-binance | |-backtest_results

If I use find | sed 's|[^/]*/|- |g' .

• wallets.py
• worker.py
• tradesv3.dryrun.sqlite
• exceptions.py
• strategy
• • init.py
• • pycache
• • • init.cpython-37.pyc
• • • interface.cpython-37.pyc
• • BBRSI.py
• • interface.py
• init.py
• edge
• • init.py
• • pycache
• • • init.cpython-37.pyc
• • • edge_positioning.cpython-37.pyc
• • edge_positioning.py
• plot
• • init.py
• • plotting.py
• pairlist
• • init.py
• • PrecisionFilter.py
• • pairlistmanager.py
• • StaticPairList.py
• • SpreadFilter.py
• • IPairList.py
• • pycache
• • • init.cpython-37.pyc
• • • StaticPairList.cpython-37.pyc
• • • IPairList.cpython-37.pyc
• • • pairlistmanager.cpython-37.pyc
• • • PrecisionFilter.cpython-37.pyc
• • PriceFilter.py
• • VolumePairList.py
• freqtradebot.py
• templates
• • sample_strategy.py
• • sample_hyperopt_loss.py
• • sample_hyperopt_advanced.py
• • sample_hyperopt.py
• • base_config.json.j2
• • strategy_analysis_example.ipynb
• • base_hyperopt.py.j2
• • subtemplates
• • • exchange_bittrex.j2
• • • sell_trend_full.j2
• • • indicators_minimal.j2
• • • hyperopt_sell_space_minimal.j2
• • • buy_trend_minimal.j2
• • • indicators_full.j2
• • • plot_config_minimal.j2
• • • sell_trend_minimal.j2
• • • hyperopt_sell_space_full.j2
• • • hyperopt_buy_guards_minimal.j2
• • • exchange_generic.j2
• • • hyperopt_sell_guards_full.j2
• • • hyperopt_sell_guards_minimal.j2
• • • exchange_kraken.j2
• • • hyperopt_buy_guards_full.j2
• • • buy_trend_full.j2
• • • exchange_binance.j2
• • • hyperopt_buy_space_minimal.j2
• • • hyperopt_buy_space_full.j2
• • • plot_config_full.j2
• • base_strategy.py.j2
• constants.py
• configuration
• • deprecated_settings.py
• • init.py
• • config_validation.py
• • timerange.py
• • config_setup.py
• • load_config.py
• • configuration.py
• • pycache
• • • load_config.cpython-37.pyc
• • • init.cpython-37.pyc
• • • configuration.cpython-37.pyc
• • • directory_operations.cpython-37.pyc
• • • config_validation.cpython-37.pyc
• • • config_setup.cpython-37.pyc
• • • deprecated_settings.cpython-37.pyc
• • • timerange.cpython-37.pyc
• • • check_exchange.cpython-37.pyc
• • directory_operations.py
• • check_exchange.py
• misc.py
• main.py
• pycache
• • worker.cpython-37.pyc
• • init.cpython-37.pyc
• • loggers.cpython-37.pyc
• • state.cpython-37.pyc
• • freqtradebot.cpython-37.pyc
• • persistence.cpython-37.pyc
• • wallets.cpython-37.pyc
• • misc.cpython-37.pyc
• • constants.cpython-37.pyc
• • exceptions.cpython-37.pyc
• • main.cpython-37.pyc
• config.json
• optimize
• • init.py
• • default_hyperopt.py
• • default_hyperopt_loss.py
• • backtesting.py
• • hyperopt_interface.py
• • hyperopt_loss_sortino_daily.py
• • hyperopt_loss_sharpe.py
• • hyperopt_loss_interface.py
• • edge_cli.py
• • hyperopt_loss_onlyprofit.py
• • hyperopt_loss_sortino.py
• • hyperopt.py
• • hyperopt_loss_sharpe_daily.py
• • optimize_reports.py
• commands
• • trade_commands.py
• • plot_commands.py
• • init.py
• • data_commands.py
• • pairlist_commands.py
• • hyperopt_commands.py
• • optimize_commands.py
• • build_config_commands.py
• • arguments.py
• • cli_options.py
• • pycache
• • • init.cpython-37.pyc
• • • cli_options.cpython-37.pyc
• • • deploy_commands.cpython-37.pyc
• • • hyperopt_commands.cpython-37.pyc
• • • build_config_commands.cpython-37.pyc
• • • plot_commands.cpython-37.pyc
• • • list_commands.cpython-37.pyc
• • • trade_commands.cpython-37.pyc
• • • pairlist_commands.cpython-37.pyc
• • • optimize_commands.cpython-37.pyc
• • • arguments.cpython-37.pyc
• • • data_commands.cpython-37.pyc
• • deploy_commands.py
• • list_commands.py
• main.py
• resolvers
• • init.py
• • hyperopt_resolver.py
• • strategy_resolver.py
• • exchange_resolver.py
• • iresolver.py
• • pycache
• • • init.cpython-37.pyc
• • • strategy_resolver.cpython-37.pyc
• • • iresolver.cpython-37.pyc
• • • pairlist_resolver.cpython-37.pyc
• • • exchange_resolver.cpython-37.pyc
• • pairlist_resolver.py
• state.py
• data
• • init.py
• • dataprovider.py
• • history
• • • init.py
• • • history_utils.py
• • • jsondatahandler.py
• • • pycache
• • • • init.cpython-37.pyc
• • • • history_utils.cpython-37.pyc
• • • • idatahandler.cpython-37.pyc
• • • idatahandler.py
• • pycache
• • • init.cpython-37.pyc
• • • converter.cpython-37.pyc
• • • dataprovider.cpython-37.pyc
• • converter.py
• • btanalysis.py
• persistence.py
• loggers.py
• exchange
• • ftx.py
• • init.py
• • kraken.py
• • binance.py
• • exchange.py
• • pycache
• • • init.cpython-37.pyc
• • • ftx.cpython-37.pyc
• • • common.cpython-37.pyc
• • • exchange.cpython-37.pyc
• • • binance.cpython-37.pyc
• • • kraken.cpython-37.pyc
• • • bibox.cpython-37.pyc
• • common.py
• • bibox.py
• vendor
• • init.py
• • qtpylib
• • • init.py
• • • pycache
• • • • init.cpython-37.pyc
• • • • indicators.cpython-37.pyc
• • • indicators.py
• • pycache
• • • init.cpython-37.pyc
• rpc
• • init.py
• • webhook.py
• • fiat_convert.py
• • pycache
• • • rpc_manager.cpython-37.pyc
• • • init.cpython-37.pyc
• • • fiat_convert.cpython-37.pyc
• • • telegram.cpython-37.pyc
• • • rpc.cpython-37.pyc
• • api_server.py
• • rpc.py
• • telegram.py
• • rpc_manager.py
• user_data
• • notebooks
• • • strategy_analysis_example.ipynb
• • plot
• • hyperopt_results
• • strategies
• • • sample_strategy.py
• • • pycache
• • • • BBRSI.cpython-37.pyc
• • • • sample_strategy.cpython-37.pyc
• • • BBRSI.py
• • hyperopts
• • • sample_hyperopt_loss.py
• • • sample_hyperopt_advanced.py
• • • sample_hyperopt.py
• • data
• • • binance
• • backtest_results

Regards Joe

[xmatthias]

well from a user perspective, the only directory that should concern you is user_data and it's contents.

by supplying --userdir /some/directory to freqtrade commands you can also place this anywhere on your disk you like. (Personally, i usually run mine with --userdir ../userdir to keep the userdirectory versioned seperately).

The internal structure of freqtrade may seem like "many files" - but everything within __pycache__ directories is automatically generated by python.

Also, if you're a developer / trying to change freqtrade internals, then these files may concern you. But if you have some experience developing you'll soon realize that having more, small files will make a project much easier to maintain than having only a few large files.

Long story short

• From a project maintainability perspective, we're most likely adding more files than removing files and combining functionality into one file.
• from a user perspective - only user_data should concern you - and user_data can be placed whever you like (you can get an empty template by using freqtrade create-userdir --userdir /some/directory).

[hroff-1902]

I did not get the point.

root@JDCD:~/freqtrade/freqtrade# tree /path/to/folder /path/to/folder [error opening dir]

0 directories, 0 files

What have you expected if you use the tree utility on a non-existing dir?

Are from the Udemy course and they teached you to place your custom strategies under freqtrade/strategies/ and custom hyperopts under freqtrade/optimize/?

[joe-nano]

I am from the Udemy course but that was not the issue

Hello some of the files seem to not be found and I'm having to install config files in multiple folders before the bot would run. I am not sure if this is supposed to be the case?

Not criticising I was just offering a suggestion and trying to understand why there are two freqtrade folders which makes it confusing to organise files. I printed out the directory structure to illustrate.

Regards, Joe

[xmatthias]

well if you're using the last version, you should be able to have the file in your user_data directory - so in user_data/config.json.

There is no other place the coniguration should be - so please remove it from all other places. Same for strategy / hyperopt files - all goes into user_data/<subfolder>.

[joe-nano]

Thank you Matthias I finally got it working, I have no intention of breaking it again. Thank you for your help.

[hroff-1902]

and trying to understand why there are two freqtrade folders which makes it confusing to organise files. I printed out the directory structure to illustrate.

If you are about the freqtrade dir created by git and the freqtrade subdir in this -- this is the standard layout for python projects.

For example:

• https://github.com/peerchemist/finta -- finta dir for the project and finta subdir in it
• https://github.com/pandas-dev/pandas -- pandas dir for the project and pandas subdir in it

This is just a common convention for pythonistas, freqtrade codebase also follows it; so I just did not get the point you are talking about above...