search

DeviaVir / zenbot

Zenbot is a command-line cryptocurrency trading bot using Node.js and MongoDB.
MIT License
8.21k stars 2.04k forks source link

Bot trades with loss all the time. #189

Closed krilson closed 6 years ago

krilson commented 7 years ago

Gave this a try, exchange poloniex, default settings, live trading (not sim).

The bot consistently without exception buys at price X and sells at a lower price. It is doing the opposite of what it should be doing. ( I am not kidding, every trade was with a loss, even in this crazy bull market we have now)

Even tried disabling c.sell_stop_pct = 0 but still it persists in doing this.

Example: Buy NXT at 0.00003540 at 2017-05-24 02:13:05 Sell NXT at 0.00003449 at 2017-05-24 13:02:01

I will leave it running and keep up with the updates if the behavior changes maybe.

Edit: To also mention, I have it running on 10 pairs. Every trade that happened on any of the pairs was this way. Sold at a lower price. And it is running now for about 5 days. Maybe the long term result will be different.

dukye commented 6 years ago

@krilson Could you share your file with modifications ?

Thx

krilson commented 6 years ago

@dukye Sure, here it is, very simple modification: https://github.com/krilson/zenbot/commit/66fc7fae22987adb1162794f3bf97fae70d19e25#diff-c081f5fa7634cfc3a1254e982a748f9c

My conf looks like this: // Optional stop-order triggers:

// sell if price drops below this % of bought price (0 to disable) c.sell_stop_pct = 0 // buy if price surges above this % of sold price (0 to disable) c.buy_stop_pct = 0 // enable trailing sell stop when reaching this % profit (0 to disable) c.profit_stop_enable_pct = 0 // maintain a trailing stop this % below the high-water mark of profit c.profit_stop_pct = 1

// Order execution rules:

// avoid trading at a slippage above this pct c.max_slippage_pct = 5 // buy with this % of currency balance c.buy_pct = 50 // sell with this % of asset balance c.sell_pct = 90 // ms to adjust non-filled order after c.order_adjust_time = 30000 // avoid selling at a loss below this pct c.max_sell_loss_pct = 0.1 // ms to poll order status c.order_poll_time = 5000 // ms to wait for settlement (after an order cancel) c.wait_for_settlement = 5000 // ms to wait for settlement (after a funds on hold error) c.wait_more_for_settlement = 60000 // % to mark up or down price for orders c.markup_pct = 0

// Misc options:

// default # days for backfill and sim commands c.days = 14 // ms to poll new trades at c.poll_trades = 60000 // amount of currency to start simulations with c.currency_capital = 1000 // amount of asset to start simulations with c.asset_capital = 0 // for sim, reverse time at the end of the graph, normalizing buy/hold to 0 c.symmetrical = false // number of periods to calculate RSI at c.rsi_periods = 14 // period to record balances for stats c.balance_snapshot_period = '15m'

And I start the bot with this: ./zenbot.sh trade --period=2h --min_periods=60 --trend_ema=30 --neutral_rate=0.25 poloniex.LTC-XMR

I am experimenting now one pair with a 24h period. Experiment with the neutral rate a bit, depends how soon you want it to buy/sell.

Myenny commented 6 years ago

Thank you for sharing @krilson Would you mind sharing your statistics? My bot doesn't seem to be trading with very much with these settings

ether-btc commented 6 years ago

@krilson when I install zenbot for the first time, are your changes already included or do I have to edit them in myself? Setting up a Raspi fresh, so asking in advance. thanks!

dukye commented 6 years ago

@krilson thanks :) @ether-btc yourself check krilson's prior post 9 days ago :) he join the diff link for trend_ema strategie.

MartinHlavna commented 6 years ago

will the fix be merged?

mathiasaerts commented 6 years ago

I was going to try this bot but then noticed this issue. Checking the code made me wonder, in the file mentioned above (extensions/strategies/trend_ema/strategy.js), the description at the top says:

Buy when (EMA - last(EMA) > 0) and sell when (EMA - last(EMA) < 0).

_(I'm assuming s.period.trend_ema_rate represents EMA and s.period.trend_ema_stddev represents last(EMA) here. Please correct me if I'm wrong..)_

However, the code at line 59 doesn't seem to reflect this statement and it seems a logic error was made while putting it into code:

Let's say EMA = s.period.trend_ema_rate = a and last(EMA) = s.period.trend_ema_stddev = b In this case, the description says we should sell when a - b < 0, however it's clear that when we rewrite it to match the current format in code, this does not add up:

a - b < 0 (a - b) + b < 0 + b a < b, while in code, it currently says a < -b

So, I'm going to change line 59 to the following and then give this bot a spin!

else if (!s.cancel_down && s.period.trend_ema_rate < s.period.trend_ema_stddev) {

Edit: Commit on hotfix branch: https://github.com/mathiasaerts/zenbot/commit/59eed42cb816effe1397d87187e208978d91e2ea#diff-c081f5fa7634cfc3a1254e982a748f9c

kazdimenator commented 6 years ago

Edit: Commit on hotfix branch: mathiasaerts/zenbot@59eed42#diff-c081f5fa7634cfc3a1254e982a748f9c

With this change in the simulation mode, the bot shows the worst result ...

MT00x commented 6 years ago

@mathiasaerts Great!. Did you already make some simulations comparing the result before and after your fix (same period, same options)? Can you do it ?

smarzola commented 6 years ago

(I'm assuming s.period.trend_ema_rate represents EMA and s.period.trend_ema_stddev represents last(EMA) here. Please correct me if I'm wrong..)

Nope, you're wrong trend_ema_rate is the trend_ema expressed as a rate:

(EMA - last(EMA)) / last(EMA) * 100 https://github.com/carlos8f/zenbot/blob/55d877a1998e65171c3efa544d8b3476ee11e61f/extensions/strategies/trend_ema/strategy.js#L30

trend_ema_rate is positive if EMA - last(EMA) > 0 so the signal should be buy, negative if EMA - last(EMA) < 0 so the signal should be sell

trend_ema_stddev is a threshold value that avoid trades if -trend_ema_stddev < trend_ema_rate < trend_ema_stddev, by default is the standard deviation of the trend_ema on the last 10 periods, but it could be overridden by a static value passing the option neutral_rate.

So https://github.com/carlos8f/zenbot/blob/55d877a1998e65171c3efa544d8b3476ee11e61f/extensions/strategies/trend_ema/strategy.js#L51-L64 conditions are correct, the issue is not here.

What I don't get is s.acted_on_trend variable, looking at the code it should always be false or undefined so the ternary operator https://github.com/carlos8f/zenbot/blob/55d877a1998e65171c3efa544d8b3476ee11e61f/extensions/strategies/trend_ema/strategy.js#L64 always resolve the first expression -> it always acts. This does not sound right to me, I'll have a look and get back.

interbiznw commented 6 years ago

How did that work out?

On Dec 12, 2017, 5:08 PM, at 5:08 PM, Mathias Aerts notifications@github.com wrote:

I was going to try this bot but then noticed this issue. Checking the code made me wonder, in the file mentioned above (extensions/strategies/trend_ema/strategy.js), the description at the top says:

Buy when (EMA - last(EMA) > 0) and sell when (EMA - last(EMA) < 0).

_(I'm assuming s.period.trend_ema_rate represents EMA and s.period.trend_ema_stddev represents last(EMA) here. Please correct me if I'm wrong..)_

  • On line 51, s.period.trend_ema_rate > s.period.trend_ema_stddev does match (EMA - last(EMA) > 0) (→ Buy)

However, the code at line 59 doesn't seem to reflect this statement and it seems a logic error was made while putting it into code:

  • s.period.trend_ema_rate < (s.period.trend_ema_stddev * -1) does not seem to match (EMA - last(EMA) < 0) (→ Sell)

Let's say EMA = s.period.trend_ema_rate = a and last(EMA) = s.period.trend_ema_stddev = b In this case, the description says we should sell when a - b < 0, however it's clear that when we rewrite it to match the current format in code, this does not add up:

a - b < 0 (a - b) + b < 0 + b a < b, while in code, it currently says a < -b

So, I'm going to change line 59 to the following and then give this bot a spin!

else if (!s.cancel_down && s.period.trend_ema_rate < s.period.trend_ema_stddev) {

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/carlos8f/zenbot/issues/189#issuecomment-351246947

mathiasaerts commented 6 years ago

@smarzola Thanks for clearing that up. I was obviously drawing a conclusion way too fast. @kazdimenator Thanks for trying it out anyway! It seems like I need to take a better look first..

danielkhalilov commented 6 years ago

@smarzola The "acted_on_trend" part of the code does not make sense to me either. It seems like the code is trying to say "if the trend in the last period was not up, then we did not act on it", ie "act" on the first period with a buy signal. The way the code currently is, it seems like the acted_on_trend variable is a dummy variable that is not really doing anything. In live trading, the moment the buy threshold is hit (emarate > trend_ema_stddev), the console output signals a "buy"; then on the next period if emarate>trend_ema_stddev still holds, it says "buying" and attempts to make the trade until it says "bought". (similar for "sell", "selling", "sold" statuses). Not sure if this helps you in any way, just trying to spark some ideas.

@krilson it would be very interesting to see this play out so that the bot does not whipsaw as much and does not enters/exit so suddenly. i may try playing with this (ema current - ema previous = close to 0) = subsignal 1 (S1) (ema current - ema previous = positive) = subsignal 2 (S2) - ema going up (ema current - ema previous = negative) = subsignal 3 (S3) - ema going down Buy on consecutive S2 signals, or when pattern goes from S3-->S1-->S2 (reversal) Sell on consecutive S3 signals, or when the pattern goes from S2-->S1-->S3 (reversal)

Observations for neutral_rate: For using small periods like 2m or 5m, neutral_rate="auto" works well, but causes too many whipsaws. For larger periods (15m, 1h), neutral_rate="auto" does not trigger often thus some set the neutral_rate to an arbitrary .1 or .2 (i guess because this .1 or .2 is bigger than the trend_ema_stddev using a long period). I wonder if multiplying the trend_ema_stddev by a variable based on period size would work instead of setting it manually. To generalize, since neutral_rate and period are correlated, perhaps there is some variable "k" such that k*trend_ema_stddev smoothes the whipsaws, where k is larger for small periods and smaller for large periods. This would help filter out whipsaws for small-period trials and help pick up signals better for large periods trials using a variable neutral_rate.

I hope the above is helpful in some way. If i implement the new trend or new neutral_rate strategy mentioned above I will post the code snippets (despite having ever coded in node.js). Thank you for devoting the time to developing and improving Zenbot, very amazing tool.

EDIT (results): I tested multiplying the stddev by a constant - while it filters out whipsaws a bit for low periods, it makes the buy/sell trigger peaks less accurate, so there will always be that tradeoff; concluded that the default neutral_rate is great without the multiplier for low periods and ideally an arbitrary constant close to 0 for larger periods. Also briefly tested krilson's strategy - one can accomplish this using s.period.trend_ema_rate_1 = (s.lookback[0].trend_ema - s.lookback[1].trend_ema) / s.lookback[1].trend_ema 100 s.period.trend_ema_rate_2 = (s.lookback[1].trend_ema - s.lookback[2].trend_ema) / s.lookback[2].trend_ema 100 .......... (for t periods, depending on how sensitive you want the signal to be) Then for all of them do: if s.period.trend_ema_rate_t > neutral_rate, S2, if s.period.trend_ema_rate_t < -neutral_rate, S3, else S1 Store these rates in an array {rate_1, rate_2, .... , rate_t} And code the strategy: for example, if {S3 S3 S1 S2 S2} then buy, if {S2 S2 S1 S3 S3} then sell I personally got too lazy to implement all the variations for this last step; if someone has a proposition to nail these combinations, perhaps I can finish the code. I imagine this working very well at picking up momentum in low-period (1m, 2m) simulations

shr00mie commented 6 years ago

this might just be a layperson's observations of what's going on behind the scenes, but this is what i'm observing.

current:

the way it SHOULD be:

other thoughts: this is less of a concern for people using strategies that are <5m in period_length, but if you're over 5/10m, let's say 45m, and current period data indicates an oversold/buy/sell 10m into your current period, it's gonna wait 35m to act on that?...that don't work.

strategies are also the wrong approach. strategies should be replaced with a library of technical indicators which one should be allowed to use in order to create strategies via simple equation logic. maybe provide a handful of out of the box "strategies" to show how they are utilized, but not depend upon them for continued or base utility.

Ayagami commented 6 years ago

No news about this?

kenorb commented 6 years ago

Related:

carlos8f commented 6 years ago

Hey guys! I'm working on a new bot: Bot18, which will be more flexible, better execution, not TA-dependent. Hopefully will lead to much better profitability! However, exchange support will take a while to port over and implement for the new socket-based, realtime approach. Stay tuned!

https://github.com/DeviaVir/zenbot/issues/1589

bot18_icon

~Carlos

carlos8f commented 6 years ago

@shr00mie, those are my thoughts entirely, and I've implemented that approach already in Bot18. It's gonna be 🔥 !

shr00mie commented 6 years ago

@carlos8f so it sounds like this is going to be a lot more event driven than the on_period nature of zenbot (yay!). the approach i'm currently working around is standardization of api/websocket components to hopefully allow plug and play exchange connectivity. commands and URLs would be the variables while standardization of API/websocket/order workflow would be ubiquitous.

what would you be replacing TA with any why? not following on the desire or move away from a standardized computational lib. something hand made or custom seems like it would be considerably more error prone.

looking forward to seeing what you're doing with as well as how data is going to be stored compared to zenbot (which was pretty painful). i feel like the DB could have been considerably more utilized for things like trade history, user account storage (user app login&pass, encrypted API keys, etc).

carlos8f commented 6 years ago

@shr00mie yes, bot18 will need to standardize the ws API workflow. bot18 doesn't have exchange abstraction yet, so i can focus on the maximum potential of the gdax api, my primary target. abstraction will come as soon as i get the foundation laid.

I'm not replacing TA with anything, just moving TA to the background, one potential way to generate orders. The engine will be more low-level, so basically the commands are:

  1. "set limit order at X price X size", then "cancel when conditions are..."
  2. "maintain limit order at X price depth", cancel when...
  3. "execute taker order with X funds or X asset"

Then, if we're mimicking what Zenbot 4 does by default, the strategy says every 2m,

That way, the engine input is changed to 3 mix-and-match commands (you can now buy and sell at the same time with multiple orders on each side), instead of a vague "signal" which invalidates signals before it and doesn't differentiate between command 2 and 3 (you have to provide taker or maker as an arg)

And the addition of command 1, which doesn't exist in Zenbot yet, will make market-making possible, and strategies will be able to maintain trailing stops directly.