No module named 'ta' - Hercules strategy

aristosv commented 3 years ago

Describe your environment

Operating system: Debian 10
Python Version: Python 3.7.3
CCXT version: ccxt==1.48.76
Freqtrade Version: freqtrade develop-f138cca7

Your question

I am trying to backtest the Hercules strategy, but I get an error saying: 2021-05-04 08:08:48,794 - freqtrade.resolvers.iresolver - WARNING - Could not import /usr/local/freqtrade_USDT_aristos/user_data/strategies/freqtrade_USDT_aristos.py due to 'No module named 'ta''

I have added this to the configuration, as per the strategy instructions but still no luck {"method": "AgeFilter", "min_days_listed": 100}

xmatthias commented 3 years ago

You should be able to get this to run with pip install ta (a requirement for this strategy).

gnuton commented 3 years ago

should this package be already part of the docker image?

xmatthias commented 3 years ago

it's not - but installing custom dependencies is pretty easy also with docker.

gnuton commented 3 years ago

Just for helping others looking for a solution for this.. Here is mine:

1 Create docker-compose.local.yml

---
version: '3'
services:
  freqtrade:
    image: freqtrade:local
    build:
       context: .
       dockerfile: "./Dockerfile.local.dev"

2. Create a Dockerfile.local.dev

FROM freqtradeorg/freqtrade:develop
USER ftuser
RUN pip install --user ta

3. Then I do run it with:

docker-compose -f docker-compose.yml -f docker-compose.local.yml run --rm freqtrade backtesting --config user_data/config.json --strategy ${STRATEGY} --timerange ${TIME_RANGE} -i ${TIMEFRAME} -p ${PAIR}

The above command will use docker-compose.yaml settings but with the docker image containining "ta".

gnuton commented 3 years ago

BTW since "ta" is required by several new strategies in https://github.com/freqtrade/freqtrade-strategies.git I would still think that adding this dep to official dev docker image would still make sense (at least for me :D)

jfcarre commented 3 years ago

Thank you very much for sharing the solution @gnuton. I have just one question, in the Dockerfile.dev file, USER root and USER ftuser must be changed? If yes, what should be placed in there? Thank you!

xmatthias commented 3 years ago

@gnuton your instruction is wrong.

You should not install it as root - but instead use pip install --user ta - which is also the documented way in the examples. Otherwise you're mixing modules in local and global places - which is never a good idea.

gnuton commented 3 years ago

@jfcarre I fixed the Docker file above (hopefully) according to @xmatthias suggestion.

jfcarre commented 3 years ago

Thank you! Now everything works properly.

ThatFack commented 1 year ago

Hello ! I am completely new to docker and I am running into this issue :

I followed your help post, please see below my configs files and the commands I run :

Docker-compose.yml

---
version: '3'
services:
  freqtrade:
    image: freqtradeorg/freqtrade:stable
    # image: freqtradeorg/freqtrade:develop
    # Use plotting image
    # image: freqtradeorg/freqtrade:develop_plot
    # Build step - only needed when additional dependencies are needed
    # build:
    #   context: .
    #   dockerfile: "./docker/Dockerfile.custom"
    restart: unless-stopped
    container_name: freqtrade
    volumes:
      - "./user_data:/freqtrade/user_data"
    # Expose api on port 8080 (localhost only)
    # Please read the https://www.freqtrade.io/en/stable/rest-api/ documentation
    # before enabling this.
    ports:
      - "127.0.0.1:8080:8080"
    # Default command used when running `docker compose up`
    command: >
      trade
      --logfile /freqtrade/user_data/logs/freqtrade.log
      --db-url sqlite:////freqtrade/user_data/tradesv3.sqlite
      --config /freqtrade/user_data/config.json
      --strategy SampleStrategy

Docker-compose.local.yml

version: '3'
services:
  freqtrade:
    image: freqtrade:local
    build:
       context: .
       dockerfile: "./Dockerfile.env"

Dockerfile.env

FROM freqtradeorg/freqtrade:develop

RUN pip install market_profile

I did removed the --user parameter because I am running on windows 10.

When i run this : docker-compose -f docker-compose.yml -f docker-compose.local.yml run --rm freqtrade backtesting --config user_data/config.json --strategy ${STRATEGY} --timerange 20220901-20230401 -i 1m

I can see that the market_profile module is indeed downloaded and exported to the image, please see below the output :

[+] Building 0.8s (6/6) FINISHED
 => [internal] load build definition from Dockerfile.env                                                                                                    0.1s
 => => transferring dockerfile: 108B                                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                                           0.0s
 => => transferring context: 2B                                                                                                                             0.0s
 => [internal] load metadata for docker.io/freqtradeorg/freqtrade:develop                                                                                   0.6s
 => [1/2] FROM docker.io/freqtradeorg/freqtrade:develop@sha256:6e96cb3c05f10704914d99ee9f1e07917928b8e4563435bdafab3f72fa4852c6                             0.0s
 => CACHED [2/2] RUN pip install market_profile                                                                                                             0.0s
 => exporting to image                                                                                                                                      0.1s
 => => exporting layers                                                                                                                                     0.0s
 => => writing image sha256:0a1a2df9d7fd769b3e71372ba179a7eb9fa64688a4a70bdc407f93ced0f34a2e                                                                0.0s
 => => naming to docker.io/library/freqtrade:local                                                                                                          0.0s

Then I got this error about a missing python module :

2023-04-02 21:04:59,660 - freqtrade - INFO - freqtrade docker-2023.4.dev-19e112f3
2023-04-02 21:04:59,673 - freqtrade.configuration.load_config - INFO - Using config: user_data/config.json ...
2023-04-02 21:04:59,680 - freqtrade.loggers - INFO - Verbosity set to 0
2023-04-02 21:04:59,681 - freqtrade.configuration.configuration - INFO - Parameter -i/--timeframe detected ... Using timeframe: 1m ...
2023-04-02 21:04:59,681 - freqtrade.configuration.configuration - INFO - Using max_open_trades: 10 ...
2023-04-02 21:04:59,681 - freqtrade.configuration.configuration - INFO - Parameter --timerange detected: 20220901-20230401 ...
2023-04-02 21:04:59,941 - freqtrade.configuration.configuration - INFO - Using user-data directory: /freqtrade/user_data ...
2023-04-02 21:04:59,946 - freqtrade.configuration.configuration - INFO - Using data directory: /freqtrade/user_data/data/bybit ...
2023-04-02 21:04:59,946 - freqtrade.configuration.configuration - INFO - Overriding timeframe with Command line argument
2023-04-02 21:04:59,946 - freqtrade.configuration.configuration - INFO - Parameter --cache=day detected ...
2023-04-02 21:04:59,946 - freqtrade.configuration.configuration - INFO - Filter trades by timerange: 20220901-20230401
2023-04-02 21:04:59,948 - freqtrade.exchange.check_exchange - INFO - Checking exchange...
2023-04-02 21:04:59,968 - freqtrade.exchange.check_exchange - WARNING - Exchange "bybit" is known to the the ccxt library, available for the bot, but not officially supported by the Freqtrade development team. It may work flawlessly (please report back) or have serious issues. Use it at your own discretion.
2023-04-02 21:04:59,968 - freqtrade.configuration.configuration - INFO - Using pairlist from configuration.
2023-04-02 21:04:59,968 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2023-04-02 21:04:59,975 - freqtrade.commands.optimize_commands - INFO - Starting freqtrade in Backtesting mode
2023-04-02 21:04:59,976 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled
2023-04-02 21:04:59,976 - freqtrade.exchange.exchange - INFO - Using CCXT 3.0.37
2023-04-02 21:04:59,976 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'swap'}, 'enableRateLimit': True}
2023-04-02 21:04:59,996 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'swap'}, 'enableRateLimit': True}
2023-04-02 21:05:00,019 - freqtrade.exchange.exchange - INFO - Using Exchange "Bybit"
2023-04-02 21:05:01,453 - freqtrade.exchange.exchange - INFO - Using cached leverage_tiers.
2023-04-02 21:05:01,453 - freqtrade.exchange.exchange - INFO - Done initializing 208 markets.
2023-04-02 21:05:01,461 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Bybit'...
2023-04-02 21:05:01,518 - freqtrade.resolvers.iresolver - WARNING - Could not import /freqtrade/user_data/strategies/MarketProfileStrategy.py due to 'No module named 'market_profile''
2023-04-02 21:05:01,584 - freqtrade - ERROR - Impossible to load Strategy 'MarketProfileStrategy'. This class does not exist or contains Python code errors.

I know that my strategy has no Python code error as I can backtest it on my virutalenv but I got a weird error saying that futures trading is not supported on bybit even thought I am currently running another strategy in live on bybit. This is why I am trying to run it on docker so I have the latest version for sure of freqtrade.

Thank you very much to anyone taking the time to read my issue, really apprieciate your help @xmatthias , seen you on every issue raised and helped me a lot getting into freqtrade.

tuwid commented 1 year ago

it's not - but installing custom dependencies is pretty easy also with docker.

but why? the whole point of docker is to pack everything you need in the image, not do hacks..

xmatthias commented 1 year ago

so we should best install all of pypi - just in case someone needs any module? It'll only blow the container up to 200 GB or so - but then nobody will ever have this problem.

it's easy to pack into an image - but that doesn't mean it makes sense. Common dependencies are in the docker-file - the rest can be installed as needed - which is pretty well documented.

freqtrade / freqtrade-strategies