search

Drakkar-Software / OctoBot

Open source crypto trading bot
https://www.octobot.cloud
GNU General Public License v3.0
3.24k stars 751 forks source link

question: portfolio optimization #1112

Open Amarchuk opened 4 years ago

Amarchuk commented 4 years ago

Hi,

It is not a feature request and more like a question. I check all the docs and try to find examples, but still not sure about. My usecase is simple to formulate: I have a portfolio with some weights (for example [0.25, 0.5, 0.25, 0.0]) and on each step I have new prediction of the weights and thus need to sell/buy several pairs to rebalance according to prediction. Previously I did this manually calculating limit prices and tradable amount and it works more or less correctly. And I try to understand if it possible to develop such approach here, because Octobot looks superuseful and web-interface is especially fun to work with.

If I can read a specific code part - link is appreciated. Thanks!

GuillaumeDSM commented 4 years ago

Hi, Thanks for you interest ! You can use OctoBot for that having evaluators to compute the weights and then make OctoBot balance your portfolio for you however this will require a bit of code since it is not integrated in the bot just yet. We are at the final phase of a major refactor and therefore I suggest you have a look at the 0.4 branch of this repo. You can see examples of evaluators here https://github.com/Drakkar-Software/OctoBot-Tentacles/blob/0.4.0/Evaluator/TA/momentum_evaluator/momentum_evaluator.py

Then the idea would be to create a trading mode that creates trades using the current portfolio balance. Example of a trading mode: https://github.com/Drakkar-Software/OctoBot-Tentacles/blob/0.4.0/Trading/Mode/daily_trading_mode/daily_trading_mode.py

GitHub
Drakkar-Software/OctoBot-Tentacles
This repository contains public tentacles (packages) for the OctoBot project - Drakkar-Software/OctoBot-Tentacles
GitHub
Drakkar-Software/OctoBot-Tentacles
This repository contains public tentacles (packages) for the OctoBot project - Drakkar-Software/OctoBot-Tentacles
Amarchuk commented 4 years ago

Thanks @GuillaumeDSM for these examples! I start to investigate the possibilities to implement my strategy using 0.4.0 as a reference and will return with more questions.

One particular question: will it be possible to backtest my portfolio optimization using Octobot web UI or other options? I play with other tentacles recently downloading data for coins and backtest already implemented examples, but seems like this is suitable for one symbol at a time only.

Herklos commented 4 years ago

Thanks @GuillaumeDSM for these examples! I start to investigate the possibilities to implement my strategy using 0.4.0 as a reference and will return with more questions.

One particular question: will it be possible to backtest my portfolio optimization using Octobot web UI or other options? I play with other tentacles recently downloading data for coins and backtest already implemented examples, but seems like this is suitable for one symbol at a time only.

Hi @Amarchuk, you mean backtesting on multiple symbols at the same time to simulate your full portfolio optimization ?

Amarchuk commented 4 years ago

Hi @Herklos, yes, exactly as you formulate

Herklos commented 4 years ago

Hi @Herklos, yes, exactly as you formulate

Backtesting on multiple symbol is not yet available but it's in our 0.4.0 roadmap.

GuillaumeDSM commented 4 years ago

It should be available very soon, probably this week.

magick93 commented 4 years ago

Hi @GuillaumeDSM - can you please give an update on the status of Backtesting on multiple symbol?

Thanks

GuillaumeDSM commented 4 years ago

Hi @magick93, It's working in the latest versions of 0.4 !

Amarchuk commented 4 years ago

Hi @GuillaumeDSM, is it correct that it is not yet available in the Web UI? And what does this actually mean - we can run same evaluator on just multiple symbols at once, or evaluator/strategy can now decide using prices from multiple symbols?

GuillaumeDSM commented 4 years ago

image image It's available on 0.4 from the web interface, you might need to re-install your tentacles to get the update.

Strategies and evaluators can chose to be evaluating every symbols or only one, same for strategies and trading modes, it's the purpose of this method:

@classmethod
def get_is_symbol_wildcard(cls) -> bool:

Ex: default is True for evaluators meaning that an evaluator will be waken up multiple times, once for each symbol. If it's returning False an instance of this evaluator will be created for each symbol and will only be waken up for its own symbol update

Amarchuk commented 4 years ago

Thanks, good to know about - I can't find it by myself.

Hm, interesting - I try to pull 0.4.0 and different ways to install it. Which way you suggest to update to 0.4 and test these new features? Will latest launcher works with new code as well (I try it, but ui shows v0.3.9 and I actually not test if it contains new features or not)?

GuillaumeDSM commented 4 years ago

I suggest you have a look at this page of the wiki for how to install octobot 0.4.

The current 0.3 launcher is not displaying OctoBot 0.4 because 0.4 is in alpha tests for now, no public version has been released yet. You will find more info about 0.4 on this page if you are curious about it.

Amarchuk commented 4 years ago

Hm, I did not touch docker yet, but experienced second time the same problem with python install (the first one was yesterday when I ask previous comment): 

> python3.7 -m  pip install --prefer-binary -r requirements.txt

Collecting cython==0.29.17 (from -r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/dc/6c/198da24bf0a1d5b5ee6119e891c5300bb1b0b8904832fac9ff62d4e381c9/Cython-0.29.17-cp37-cp37m-manylinux1_x86_64.whl
Collecting OctoBot-Commons<1.4,>=1.3.31 (from -r requirements.txt (line 5))
  Downloading https://files.pythonhosted.org/packages/1f/6f/7a0e372063aa47b9e8d8a97ef43d8036dde54566405348c55412a48f2c40/OctoBot-Commons-1.3.33.tar.gz (556kB)
     |████████████████████████████████| 563kB 715kB/s 
Collecting OctoBot-Trading<1.7,>=1.6.20 (from -r requirements.txt (line 6))
  Downloading https://files.pythonhosted.org/packages/c7/f4/400fbb6c1068dfe3943b4d339ab65f8983096974b30947977143f1353634/OctoBot-Trading-1.6.22.tar.gz (6.6MB)
     |████████████████████████████████| 6.7MB 6.9MB/s

And after that last line nothing happens (I wait around 20 minutes or so). This sounds like a bug for separated issue

Herklos commented 4 years ago

Hm, I did not touch docker yet, but experienced second time the same problem with python install (the first one was yesterday when I ask previous comment):

> python3.7 -m  pip install --prefer-binary -r requirements.txt

Collecting cython==0.29.17 (from -r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/dc/6c/198da24bf0a1d5b5ee6119e891c5300bb1b0b8904832fac9ff62d4e381c9/Cython-0.29.17-cp37-cp37m-manylinux1_x86_64.whl
Collecting OctoBot-Commons<1.4,>=1.3.31 (from -r requirements.txt (line 5))
  Downloading https://files.pythonhosted.org/packages/1f/6f/7a0e372063aa47b9e8d8a97ef43d8036dde54566405348c55412a48f2c40/OctoBot-Commons-1.3.33.tar.gz (556kB)
     |████████████████████████████████| 563kB 715kB/s 
Collecting OctoBot-Trading<1.7,>=1.6.20 (from -r requirements.txt (line 6))
  Downloading https://files.pythonhosted.org/packages/c7/f4/400fbb6c1068dfe3943b4d339ab65f8983096974b30947977143f1353634/OctoBot-Trading-1.6.22.tar.gz (6.6MB)
     |████████████████████████████████| 6.7MB 6.9MB/s

And after that last line nothing happens (I wait around 20 minutes or so). This sounds like a bug for separated issue

Hi @Amarchuk, if you're not using the current supported architecture and python version python will have to recompile some wheels (it can takes some hours depending on your computer CPU).

Amarchuk commented 4 years ago

@GuillaumeDSM Thanks for fast reply! I think that I actually use a supported architecture, but will give it another longer try anyway.

Amarchuk commented 4 years ago

Hi there,

First, another thanks for the great project! I can finally successfully install 0.4.0 using fresh virtualenv and also implement my own tentacle using some tricky way for previous version. Now I have several independent questions:

  1. I start octobot 0.3.9 using launcher and need some additional python packages in my tentacle, but can't find proper python version where to install them. I add a print in log and see that octobot starts with python 3.7.2, but I can't locate it on my machine. Can we control somehow the python version when start it using launcher? (note: bash python3 command will run 3.6.6, python3.7 command - 3.7.4)

  2. How long can backtesting takes? It runs already for an hour for default 'DailyTradingMode' for one coin. There is Backtesting in progress sign, but the progressbar didn't move. Also is it expected behaviour that backtesting stops when you leave the /backtesting url?

  3. In 0.4.0 if I need to backtest my tentacle for multiple currencies at once - what should I change? Can you point to some functions to pay attention and starts from? (for example I am interested if self.exchange.get_symbol_prices(symbol, TimeFrames.THIRTY_MINUTES, limit=None, return_list=True) will still work for obtaining previous prices)

  4. About my own tentacle for 0.3.9: the code is tricky and kinda awkward because I need to manage portfolio as whole (so symbols are not independent from each other, because I need to sell some first and only then buy for other) and came so the solution when in my implementation of AbstractTradingModeCreator.create_new_order I create new orders not only for symbol parameter, but for other symbols as well. It is messy and I am not sure why it works, but seems like I get what I want. The question here is that for some symbols I can create all orders I want and they are executed correctly (so for symbol=ZEC/BTC I also add in created_orders orders for all other coins and return this list in create_new_order(ZEC/BTC, ...), and sometimes some of them (for other symbols) are silently cancelled. I am 100% sure that the orders are correct and can be executed. I know that it is difficult to understand what is going on here, but maybe you have a clue why this is happening.

Thanks

Amarchuk commented 4 years ago

Addition to 2: I notice in Chrome console this one, maybe it is related

Uncaught ReferenceError: get_websocket is not defined
    at init_backtesting_status_websocket (backtesting_util.js:151)
    at HTMLDocument.<anonymous> (backtesting.js:90)
    at e (jquery-3.4.0.min.js:2)
    at t (jquery-3.4.0.min.js:2)
GuillaumeDSM commented 4 years ago

Hi there,

First, another thanks for the great project! I can finally successfully install 0.4.0 using fresh virtualenv and also implement my own tentacle using some tricky way for previous version. Now I have several independent questions:

  1. I start octobot 0.3.9 using launcher and need some additional python packages in my tentacle, but can't find proper python version where to install them. I add a print in log and see that octobot starts with python 3.7.2, but I can't locate it on my machine. Can we control somehow the python version when start it using launcher? (note: bash python3 command will run 3.6.6, python3.7 command - 3.7.4)

To control the python version, the best is to use the code directly or a docker version. You can probably force a python version to run with the launcher by adding the python you want on the front of you PATH environment variable though.

  1. How long can backtesting takes? It runs already for an hour for default 'DailyTradingMode' for one coin. There is Backtesting in progress sign, but the progressbar didn't move. Also is it expected behaviour that backtesting stops when you leave the /backtesting url?

If you get this error "Uncaught ReferenceError: get_websocket is not defined" it very likely means that your browsers is using js files from 0.3 as well as 0.4, you can use ctrl+f5 or shift+f5 (depending on your browser) to reset the local cache, it should fix this problem. Backtesting time should be a few seconds.

  1. In 0.4.0 if I need to backtest my tentacle for multiple currencies at once - what should I change? Can you point to some functions to pay attention and starts from? (for example I am interested if self.exchange.get_symbol_prices(symbol, TimeFrames.THIRTY_MINUTES, limit=None, return_list=True) will still work for obtaining previous prices)

You just have to select select multiple data files with different currencies. Then if your evaluator get_is_symbol_wildcard(cls) returning True (default value), it will be called for each symbol. If returning false, an specific instance of this evaluator will be handling each symbol. Data will automatically come from ohlcv_callback, like on live mode.

  1. About my own tentacle for 0.3.9: the code is tricky and kinda awkward because I need to manage portfolio as whole (so symbols are not independent from each other, because I need to sell some first and only then buy for other) and came so the solution when in my implementation of AbstractTradingModeCreator.create_new_order I create new orders not only for symbol parameter, but for other symbols as well. It is messy and I am not sure why it works, but seems like I get what I want. The question here is that for some symbols I can create all orders I want and they are executed correctly (so for symbol=ZEC/BTC I also add in created_orders orders for all other coins and return this list in create_new_order(ZEC/BTC, ...), and sometimes some of them (for other symbols) are silently cancelled. I am 100% sure that the orders are correct and can be executed. I know that it is difficult to understand what is going on here, but maybe you have a clue why this is happening.

If you are using the DailyTradingMode as a super class, it has await self.cancel_symbol_open_orders(symbol) in async def _set_state that can cancel orders. It might be the reason. If you want to know for sure from where a cancel is triggered, you can either add a debug breakpoint on each trader.py cancel methods or raise an exception to print the current traceback when a cancel is happening (add it in trader.py cancel methods):

Thanks

Amarchuk commented 4 years ago

Thanks so much @GuillaumeDSM! I don't be able to take a look at everything yet, but this is definitely helps (and ctrl+f5 works, can backtest changes now).

Amarchuk commented 4 years ago

Hi!

Another minor question (or probably bug report?) related to this issue (still for 0.3.9). I try to fill my portfolio with needed coins in Trading tab, but, for example, there is no way to add Etherium (ETH) from dropdown list (no such option). But I can go to user/config and add it manually there as one of the coins and it will appears in OctoBot after start. Is it ok?

GuillaumeDSM commented 4 years ago

Hi, Yes, to add currencies, add them directly into your user/config.json (web interface is not yet handling currency addition)

Amarchuk commented 4 years ago

@GuillaumeDSM Thanks again for all clarifications! I have one minor new question: it seems like in 0.4.0 all Advanced folders were removed and when I try to create them and move files here anyway - I get a bunch of errors about missed metadata.json. So it seems like the way I test my code before is not applicable anymore. I try to get some insight about this from changelog and reading code, but only found some web-related changes. Can you please clarify this as well or suggest the proper way to test my tentacle in 0.4?

GuillaumeDSM commented 4 years ago

In 0.4 there is no advanced / default folders anymore, everything is bundled together in the related tentacle type, to integrate your code, you can just copy/paste an existing tentacle folder (like volatility_evaluatorfor ex) and replace everything (in every sub-file/folder) related to StochasticRSIVolatilityEvaluator and the names of the files with your tentacle

Amarchuk commented 4 years ago

@GuillaumeDSM I have now two more:

  1. So I continue trying to start using 0.4 and this is tough since there are so many changes. I finally replace almost every part and fill all configs, but actually can't turn on my strategy because I keep get errors about required_time_frames property missed, and I really don't know what to do about it because it is included in config files
 2020-06-25 22:31:38,849 INFO     OctoBot Launcher     Disclaimer accepted by user.
 2020-06-25 22:31:38,873 DEBUG    EvaluatorProducer    Pausing...
--- Logging error ---
Traceback (most recent call last):
  File "octobot4.0/OctoBot/octobot/commands.py", line 110, in start_bot
    await bot.initialize()
  File "octobot4.0/OctoBot/octobot/octobot.py", line 103, in initialize
    await self.start_producers()
  File "octobot4.0/OctoBot/octobot/octobot.py", line 113, in start_producers
    await self.evaluator_producer.run()
  File "octobot4.0/OctoBot/octobot/channels/octobot_channel.py", line 40, in run
    await self.start()
  File "octobot4.0/OctoBot/octobot/producers/evaluator_producer.py", line 39, in start
    self.matrix_id = await initialize_evaluators(self.octobot.config, self.tentacles_setup_config)
  File "p37/lib/python3.7/site-packages/octobot_evaluators/api/evaluators.py", line 167, in initialize_evaluators
    _init_time_frames(config, tentacles_setup_config)
  File "p37/lib/python3.7/site-packages/octobot_evaluators/api/evaluators.py", line 178, in _init_time_frames
    init_time_frames_from_strategies(config, tentacles_setup_config)
  File "p37/lib/python3.7/site-packages/octobot_evaluators/api/initialization.py", line 29, in init_time_frames_from_strategies
    for time_frame in strategies_eval_class.get_required_time_frames(config):
  File "p37/lib/python3.7/site-packages/octobot_evaluators/evaluator/strategy_evaluator.py", line 387, in get_required_time_frames
    raise Exception(f"'{STRATEGIES_REQUIRED_TIME_FRAME}' is missing in configuration file")
Exception: 'required_time_frames' is missing in configuration file
  1. Ok, I turn off my strategy and use default one instead, and webUI can now start, but there is no trading and if I try to go into Trading section I get this one:
  File "p37/lib/python3.7/site-packages/octobot_tentacles_manager/loaders/tentacle_loading.py", line 55, in get_tentacle
    return _tentacle_by_tentacle_class[klass.get_name()]
KeyError: 'PGPDailyTradingMode'

It feels like I can't somehow register tentacle/mode, but really can't find out why because fill any config/init which found.

GuillaumeDSM commented 4 years ago

Hi @Amarchuk ! We will soon make a guide on how to create a 0.4 tentacle since it's different from 0.3 as you noticed. About 1.

Tentacles config are stored in your user/tentacles_config/specific_config folder. When installing a tentacle, its config is copied from the tentacle folder (config sub-dir) to there. json files in your tentacle/config folder are default values for your config that are re-applied when factory resetting the tentacle config (from web interface for example). Your issue is probably that you are editing the config file in your tentacle/config folder, which is not the one your bot will be using when starting (since it's the default values one). You can copy-paste it into your user/tentacles_config/specific_config folder and replace the previous version, that should fix your issue.

About 2.

The tentacle loading process is using the metadata.json file to register tentacles: it's registering tentacles according to their class name such as in this (from web_interface/metadata.json):

{
  "version": "1.2.0",
  "origin_package": "OctoBot-Default-Tentacles",
  "tentacles": ["WebInterface"],
  "tentacles-requirements": ["web_service"]
}

Here the data about the WebInterface tentacle class (which is located into the web_interface folder) will be stored under the WebInterface key into _tentacle_by_tentacle_class because "tentacles": ["WebInterface"] is in metadata.json. Then when OctoBot will require data about this WebInterface thing (for example to display details about it to the user, such as its version or any other thing), OctoBot will use this _tentacle_by_tentacle_class using the name of the tentacle class it is trying to get data from. Therefore in your tentacle metadata.json file, you need to have this syntax "tentacles": ["WebInterface"], with the tentacles class(es) that are located into your tentacle folder into the tentacles key list for them to be registered and available for later uses in OctoBot.
I hope that fixes your issue, this is the most likely reason you are having this 2. issue.

GuillaumeDSM commented 4 years ago

About 2, I'm updating the code to first not interrupt the page loading and the display an easier to understand error (in https://github.com/Drakkar-Software/OctoBot-Tentacles/pull/271)

Amarchuk commented 4 years ago

Hi @GuillaumeDSM !

We will soon make a guide on how to create a 0.4 tentacle since it's different from 0.3 as you noticed.

Very good news!

It seems like for some reason configs were not copied into user/tentacles_config/specific_config and manual copying solves my first problem, thanks.

About second: I really appreciate the quality and fasteness of your answers, but unfortunately it is really difficult to understand this WebInterface example, because it is unusual one. I double-check all folders and still it is seems like I have metadata.json in every place I need about. So I have evaluator, strategy and mode, and each metadata looks like this:

{
  "version": "1.1.6",
  "origin_package": "OctoBot-Default-Tentacles",
  "tentacles": ["PGPTradingMode"],
  "tentacles-requirements": []
}

and the name for "tentacles" is the same as valuator/strategy/mode so they all are different. Note that in __init__.py I don't use check_tentacle_version('1.2.0',... and just import everything.

Maybe I miss something eventually and besides all tentacles_config.json I also need to additionally register tentacles somehow?

GuillaumeDSM commented 4 years ago

Your trading mode, eval and strategy classes have to be listed in tentacles_config.json for OctoBot to handle them properly yes. Updating tentacles_config.json, the __init__.py files (both inside the tentacle if missing and in the parent folder) and importing config are normally handled automatically by the tentacle manager (we also need to write a wiki article about this). To sum up, the required elements for a 0.4 tentacle are:

  1. the tentacle folder in the right place in the tentacle architecture (next to similar tentacles)
  2. in this folder:
    • a .py file with the code of your tentacle
    • importing the tentacle class of your .py file in __init__.py
    • listing this (or these) tentacle class(es) in metadata.json under "tentacles":
  3. import * from this folder in the __init__.py from the parent folder (for python to see your tentacle)
  4. add your tentacle class(es) into user/tentacles_config/tentacles_config.json (True for activation, False for not activation, please note that only one trading mode can be activated at a time)
  5. copy your tentacle condig into user/tentacles_config/specific_config if any json config file

If you did all this and it still doesn't work, then I will need more info (startup errors / warnings / etc)

Amarchuk commented 4 years ago

Hi @GuillaumeDSM,

Thanks again for the last answer - I re-check everything and finally was able to import and run my tentacle (indeed there was an error in one of json). After several hours of fixing compatibility issues I successfully run it and everything works as expected thus I switch my attention to the purpose doing it on the first place - backtesting. I try to collect new data for it, but get an error "Can't collect data for ETC/BNB on binance (data collector error: Please ensure your tentacles configuration file is valid and at least one trading mode is activated)". Unfortunately I can't backtrack the source of the error, can you please generously help me with this?

P.S. Workaround: I can switch to 0.3.9, download OHLC data from it and then import them using Import from data file, but of course it is not that useful.

GuillaumeDSM commented 4 years ago

Hi @Amarchuk !

There is an issue with the current version regarding data collection for exchange, you can fix it this way: Replace the content of your tentacles/Backtesting/collectors/exchanges/exchange_history_collector/exchange_history_collector.py with the content of this version (updated one): https://github.com/Drakkar-Software/OctoBot-Tentacles/blob/0.4.0/Backtesting/collectors/exchanges/exchange_history_collector/exchange_history_collector.py

You can import 0.3.9 data files into 0.4 one using the local file selector in data collector (this will transform 0.3.9 data into 0.4)

GitHub
Drakkar-Software/OctoBot-Tentacles
This repository contains public tentacles (packages) for the OctoBot project - Drakkar-Software/OctoBot-Tentacles
Amarchuk commented 4 years ago

It works perfectly, thanks

Amarchuk commented 4 years ago

Hi @GuillaumeDSM,

Since the last time I was able to fix issues in my mode for 0.4 and finally was able to run backtesting successfully. Thanks again for all help and suggestions!

I want to ask for one more advice: it will be ideally to add a model as a parameter for trading mode, so I can easily configure it in UI and run backtesting for different models. The problems is that different models optimize different portfolios (the names and number of coins can be different). Thus, the problem is that I need to change coins in user/config.json and make running instance of OctoBot recognize these changes. As a possible workaround I can add in json as many coins as possible and just filter those used in particular model, but maybe there is a better way to do this (I try to find some hooks used in Add a new pair button, but seems like this is not a correct option).

And one small question: if on this stage I am interested in backtesting feature only, is there any way to switch off simulated trading?

GuillaumeDSM commented 4 years ago

Hi @Amarchuk !

Glad that you are ,ow able to use 0.4 as you wish !

For now there is no way to change the currently traded pairs while OctoBot is running, it will only update the config file for the next start. We could add this in the future if there is a demand on it but for now it's not in the top priorities unfortunatelly a restart will remain necessary for now. Regarding your trading config, we could introduce something like a profile setup though (something like a set of traded pairs and evaluator/strategy/trading mode configuration) so that it would be easy to "save" a config and switch to another one etc. What do you think about this concept ?

Now if you are interested in backtesting, you can start your OctoBot with the --backtesting argument, it will only start the minimal stuff and run a backtesting on the data files that are in your config.json or that are given on your octobot command argument (traded pair will be the ones in your data files). Use OctoBot --help to get more details about how to use OctoBot from the command line. This will start the bot, run a backtesting on the given files, print results in your terminal and exit. If you still want a web interface (to display graphs for example), you will have to start a normal OctoBot and run backtestings from the web interface.

Amarchuk commented 4 years ago

Ok, good to be sure that this is not easily possible and that restart is needed.

Regarding your trading config, we could introduce something like a profile setup though (something like a set of traded pairs and evaluator/strategy/trading mode configuration) so that it would be easy to "save" a config and switch to another one etc. What do you think about this concept ?

Hm, interesting. I think this is actually can be very useful for real traders, but can help in my situation too. Maybe add a simple way to load(import) portfolio coins in OctoBot. And obviously everybody with similar needs will benefit from #1244.

And additional thanks for pointing me to terminal commands, I completely forget about them. Maybe the reasonable way to improve them is to add smth like --user_config option instead of suggested profile setups. In any case, OctoBot have such a great UI and it is a pleasure to work with it, so I prefer it before shell.

Amarchuk commented 4 years ago

One minor question: if I want to connect OctoBot to my account on one of the exchanges to do actual trading instead of simulation - will it automatically pick up portfolio info? Or just get amount for coins listed explicitly in user/config.json?

GuillaumeDSM commented 4 years ago

It will pick the exchange portfolio, the one in config is only for simulations

Amarchuk commented 3 years ago

Hi @GuillaumeDSM

I return back to this project after some time and have several questions/notes regarding latest changes (some of them just cosmetic issues):

About README in https://github.com/Drakkar-Software/OctoBot/tree/0.4.0

  1. (cosmetic) two times mentioned part with With python sources (unix)
  2. pip3 install -r pre_requirements.txt - there is no pre_requirements.txt in tree
  3. wrong dependence in requirements: OctoBot-Trading==1.10.0 ERROR: Could not find a version that satisfies the requirement OctoBot-Trading==1.10.0 (from -r requirements.txt (line 6)) (from versions: 1.4.27, 1.5.7, 1.6.23, 1.7.2, 1.8.10, 1.9.4, 1.10.1, 1.11.0, 1.11.1, 1.11.2) ERROR: No matching distribution found for OctoBot-Trading==1.10.0 (from -r requirements.txt (line 6))

I install other OctoBot-Trading, but can't start octobot to test my previously developed code:

 python3.7 start.py --simulate
 2020-10-13 13:11:53,105 INFO     OctoBot Launcher     Version : 0.4.0a23
 2020-10-13 13:11:53,105 DEBUG    OctoBot Launcher     Running on Linux:4.15.0-118-generic:x86_64 with python
 2020-10-13 13:11:53,105 INFO     OctoBot Launcher     Loading config files...
VERSION: 3.7.7 (default, Mar 10 2020, 17:25:08) 
[GCC 5.4.0 20160609]
/mnt/sdb2/anton/octobot4.0/p37/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/mnt/sdb2/anton/octobot4.0/p37/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/mnt/sdb2/anton/octobot4.0/p37/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/mnt/sdb2/anton/octobot4.0/p37/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/mnt/sdb2/anton/octobot4.0/p37/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/mnt/sdb2/anton/octobot4.0/p37/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
 2020-10-13 13:11:56,502 WARNING  tensorflow           From /mnt/sdb2/anton/octobot4.0/p37/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
 2020-10-13 13:11:56,519 INFO     OctoBot Launcher     Disclaimer accepted by user.
--- Logging error ---
Traceback (most recent call last):
  File "/mnt/sdb2/anton/octobot4.0/OctoBot3/OctoBot/octobot/commands.py", line 105, in start_bot
    await ensure_setup_configuration(bot_install_dir=OCTOBOT_FOLDER)
TypeError: ensure_setup_configuration() got an unexpected keyword argument 'bot_install_dir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/logging/__init__.py", line 1025, in emit
    msg = self.format(record)
  File "/usr/lib/python3.7/logging/__init__.py", line 869, in format
    return fmt.format(record)
  File "/mnt/sdb2/anton/octobot4.0/p37/lib/python3.7/site-packages/colorlog/colorlog.py", line 123, in format
    message = super(ColoredFormatter, self).format(record)
  File "/usr/lib/python3.7/logging/__init__.py", line 608, in format
    record.message = record.getMessage()
  File "/usr/lib/python3.7/logging/__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/usr/lib/python3.7/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/sdb2/anton/octobot4.0/OctoBot3/OctoBot/octobot/task_manager.py", line 63, in run_bot_in_thread
    self.async_loop.run_forever()
  File "/usr/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
    self._run_once()
  File "/usr/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
    handle._run()
  File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/mnt/sdb2/anton/octobot4.0/OctoBot3/OctoBot/octobot/commands.py", line 115, in start_bot
    logger.exception(e, True, f"OctoBot Exception : {e}")
Message: TypeError("ensure_setup_configuration() got an unexpected keyword argument 'bot_install_dir'")
Arguments: (True, "OctoBot Exception : ensure_setup_configuration() got an unexpected keyword argument 'bot_install_dir'")
--- Logging error ---
Traceback (most recent call last):
  File "/mnt/sdb2/anton/octobot4.0/OctoBot3/OctoBot/octobot/commands.py", line 105, in start_bot
    await ensure_setup_configuration(bot_install_dir=OCTOBOT_FOLDER)
TypeError: ensure_setup_configuration() got an unexpected keyword argument 'bot_install_dir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.7/logging/handlers.py", line 69, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python3.7/logging/handlers.py", line 185, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib/python3.7/logging/__init__.py", line 869, in format
    return fmt.format(record)
  File "/usr/lib/python3.7/logging/__init__.py", line 608, in format
    record.message = record.getMessage()
  File "/usr/lib/python3.7/logging/__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/usr/lib/python3.7/threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/sdb2/anton/octobot4.0/OctoBot3/OctoBot/octobot/task_manager.py", line 63, in run_bot_in_thread
    self.async_loop.run_forever()
  File "/usr/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
    self._run_once()
  File "/usr/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
    handle._run()
  File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/mnt/sdb2/anton/octobot4.0/OctoBot3/OctoBot/octobot/commands.py", line 115, in start_bot
    logger.exception(e, True, f"OctoBot Exception : {e}")
Message: TypeError("ensure_setup_configuration() got an unexpected keyword argument 'bot_install_dir'")
Arguments: (True, "OctoBot Exception : ensure_setup_configuration() got an unexpected keyword argument 'bot_install_dir'")

I try to figure out this right now, guess something changes in config files since then, but maybe you can help with that.

Also I want to know if I it is possible to run Docker image for 0.4 with docker-compose.yaml mentioned here.

Thanks, appreciate your help

GitHub
Drakkar-Software/OctoBot
Cryptocurrency trading bot: high frequency, daily trading, social trading, ... - Drakkar-Software/OctoBot
GuillaumeDSM commented 3 years ago

Hi ! Thanks for the feedback, readme.md will be fixed here About the OctoBot trading dependency, this is an issue with the current dependency system, it will be fixed in the next version. To nevertheless use this version, you can install OctoBot-Trading from the sources or simply clone it and add it to your PYTHONPATH before starting your bot (no need to compile it this way). With this method, you need the exact version associated to 1.1.0 though, that is to say this commit: https://github.com/Drakkar-Software/OctoBot-Trading/commit/77ed41340d4d0667b38740744e6fa5c2f3bf23ee

Amarchuk commented 3 years ago

Thanks for the answer, glad it helps to fix small typos.

Will try today suggested way to deal with OctoBot-Trading dependency - but what about docker? We are really planning to use OctoBot for trading (maybe in a month or so), and I think this will benefit you too as we will practically work as testers of new changes.

GuillaumeDSM commented 3 years ago

OctoBot 0.4 is available on docker ! https://github.com/Drakkar-Software/OctoBot/wiki/OctoBot-0.4#manual-installation

GitHub
Drakkar-Software/OctoBot
Cryptocurrency trading bot: high frequency, daily trading, social trading, ... - Drakkar-Software/OctoBot