MACD emits the same advice event multiple times

askmike commented 10 years ago

It adviced to take a long position 3 times in a row:

2014-01-22 09:31:39 (INFO): We have new trading advice!
2014-01-22 09:31:39 (INFO):    Position to take: long
2014-01-22 09:31:39 (INFO):    Market price: 814.98
2014-01-22 09:31:39 (INFO):    Based on market time: 2014-01-22 09:30:23

2014-01-22 09:31:39 (INFO): attempting to BUY 0.06995263687452452 BTC at Bitstamp
2014-01-22 09:32:10 (INFO): BUY was succesfull
2014-01-22 12:00:31 (INFO): Trader Received advice to go long Buying  BTC

2014-01-22 12:00:31 (INFO): We have new trading advice!
2014-01-22 12:00:31 (INFO):    Position to take: long
2014-01-22 12:00:31 (INFO):    Market price: 818.95
2014-01-22 12:00:31 (INFO):    Based on market time: 2014-01-22 11:59:33

2014-01-22 12:00:32 (INFO): wanted to buy USD but the amount is to small (0) at Bitstamp
2014-01-22 16:00:11 (INFO): Trader Received advice to go long Buying  BTC

2014-01-22 16:00:11 (INFO): We have new trading advice!
2014-01-22 16:00:11 (INFO):    Position to take: long
2014-01-22 16:00:11 (INFO):    Market price: 816.22
2014-01-22 16:00:11 (INFO):    Based on market time: 2014-01-22 15:59:20

2014-01-22 16:00:11 (INFO): wanted to buy USD but the amount is to small (0) at Bitstamp
2014-01-22 20:30:50 (INFO): Trader Received advice to go short Selling  BTC
2014-01-22 20:30:50 (INFO): (PROFIT REPORT) original simulated balance:  908.00000 USD
2014-01-22 20:30:50 (INFO): (PROFIT REPORT) current simulated balance:   914.39584 USD
2014-01-22 20:30:50 (INFO): (PROFIT REPORT) simulated profit:    6.39584 USD (0.70439%)

2014-01-22 20:30:50 (INFO): We have new trading advice!
2014-01-22 20:30:50 (INFO):    Position to take: short
2014-01-22 20:30:50 (INFO):    Market price: 819.00
2014-01-22 20:30:50 (INFO):    Based on market time: 2014-01-22 20:29:59

2014-01-22 20:30:50 (INFO): (PROFIT REPORT) original simulated balance:  908.00000 USD
2014-01-22 20:30:50 (INFO): (PROFIT REPORT) current simulated balance:   914.39584 USD
2014-01-22 20:30:50 (INFO): (PROFIT REPORT) simulated profit:    6.39584 USD (0.70439%)
2014-01-22 20:30:50 (INFO): attempting to SELL 0.14090079 BTC at Bitstamp
2014-01-22 20:31:21 (INFO): SELL was succesfull

askmike commented 10 years ago

I'll look into this, but maybe @djmuk knows if this was intended or not?

djmuk commented 10 years ago

I'd noticed this and would say it is 'by design'- what is happening is that the threshold for a buy (long) is being activated multiple times - The threshold is being exceeded (triggering an advice), the trigger then goes below the threshold which resets the trend to 'hold' but then exceeds the threshold again which triggers another 'buy' advice.

I would expect any of the methods to do the same as I am not aware there is any mechanism for forcing alternate long/short advice...

It would be possible to implement some form of latch that didn't issue a buy until a sell had been advised, but what do you do on startup? Equally well it may be valid to have multiple consecutive buy advices - Eg if you were adding additional funds, selling outside of gekko (eg setting sells to lock the profit) or perhaps if an 'only trade x% of available funds each time' was implemented - so each buy would trade 20% (say) of available currency into the asset.

So my view is that it is intended behaviour - and doesn't cause any issues apart from the 'unable to trade' error?

askmike commented 10 years ago

The trading methods always say what position you should take, they always have an active advice for you. Only when the advice changes we trigger the advice event.

Hold is just an internal state you can use to check if the advice is still the same as before (or it the calculated advice is neither short nor long). If the advice was long and you come up with a hold, the advice is still to go/stay long since the last detected trend was an uptrend and this has not changed. Sending multiple advice events with long all the time makes no sense: imagine you have the mail plugin and using that to buy/sell all your funds. If you get a mail to go short you sell all you assets, and after 3 hours you get another email to go short?

It would be possible to implement some form of latch that didn't issue a buy until a sell had been advised, but what do you do on startup?

Yes, all trading methods need this. On startup you always advice, even if it matches your current portfolio already because that portfolio might not having anything todo with any trading position.

Equally well it may be valid to have multiple consecutive buy advices - Eg if you were adding additional funds, selling outside of gekko (eg setting sells to lock the profit) or perhaps if an 'only trade x% of available funds each time' was implemented - so each buy would trade 20% (say) of available currency into the asset.

In your current MACD implementation getting 3 long advices in a row doesn't mean you are more likely to make money if you take a long position. If you add funds when Gekko advised to go long 5 hours ago and you follow its advice you should move those funds into a long position, instead of waiting for a random new threshold crossing which might not happen anymore at all (not getting this does not mean the trend is invalid).

So my view is that it is intended behaviour - and doesn't cause any issues apart from the 'unable to trade' error?

All plugins act upon Gekko changing the advice, not some indicator specific details like whether something is a hold or not, you are either advised to go long or to go short.

cykedev commented 10 years ago

Same on PPO

kuzetsa commented 10 years ago

The trading methods always say what position you should take, they always have an active advice for you. Only when the advice changes we trigger the advice event.

^quote from above...

@askmike This sort of behavior is something which should be configurable.

I had to remove multiple checks for no reason than to "un-fix" the "fixed" (broken, from the perspective of a GHS bull), hardcoded logic here:

https://github.com/kuzetsa/gekko/commit/916aab03d4ca3c8ece15c47fb2499e5c4d1af2af#commitcomment-4803839

Hardcoding it to "only trigger event when the advice changes" messes up the buy and hold strategy which is useful for cex.io GHS bulls.

askmike commented 10 years ago

@kuzetsa I don't agree with you BUT I have a solution that will probably satisfy your needs as well.

So my idea is that as long as a trading method suggests to go long you should always make sure the portfolio is in a long position. If you add funds yourself you are advised to line them up with the advice Gekko gave you.

However this doesn't really work on cex.io because your funds increase (always in currency -> a short position) because of how cex.io pays you out. What I think would be better is:

If advice is long, check every minute if you earned payout, if you did swap them to the asset (GHS).

This also solves that if you want to have daily candles it does not only swap the payout to GHS once per day, but once per minute. These two things should be decoupled from each other imo.

I hope you guys agree with my solution, because I think this is how I'll implement it.

EDIT: I am going to make this interval configurable, but it should be something separate unrelated to candleticks.

botweb commented 10 years ago

Sounds great :+1:

djmuk commented 10 years ago

@askmike - So you don't need anything changed in the advice methods? Perhaps this lives in the advice / trade actors anyway?

I would also say that if you added funds then you might NOT want to go long on them if there is no current trend, even if the last advice (some time ago) was to go long - we could be at the top of the upswing and no method will detect the instant it starts to go down so you run the risk of going long at a high and the short coming in below the buy price. Whereas another long advice being issued would suggest that there is another upswing starting - there is a difference between issuing 'long' advice all through an upswing (which isn't done) and issuing 'long' when the upswing has 'paused', gone to 'no trend' and then restarted.

askmike commented 10 years ago

TLDR:

I'm going to adjust all methods so they don't repeat the same advice. For now nothing except the trading method is aware of flat (sideways) markets.

Long version:

So the problem/confusing around this is:

There are 3 types of trends a market can be in: uptrend / downtrend / sideways (you can call this no trend).
Gekko's advice is based on the idea that if the market goes into an uptrend or a downtrend you can act on this change (changing position like buying/selling etc).
In some scenarios you might want to know in what state the market is in (not just those 2 but also the flat state), because those two advices never tell you when the market is going sideways (not up or down), like when you want to buy more assets non stop once new stuff comes in. Though I think you can't make an exception for this in case of CEX.io because:
1. Than: should the mailer also email you when the market goes flat after an uptrend? Because according to that logic it means you shouldn't take a long position anymore and the mailer doesn't know when you act on this information (the same thing as positioning CEX.io payouts imo) and you don't know when the uptrend is done.
2. Same goes for the real trader: what if at the time the uptrend is done and the market goes flat you have not been able to 100% switch to long yet? Currently on all exchanges (except gox) on a long advice Gekko will create limit orders at the price the of the lowest ask. However in volatile markets this number is not the lowest ask anymore when it hits the matching engine at the exchange (especially when you are not riding the trend yet but chasing it). If after 30 seconds the order is not completely done, we retarget on the ask again. Switching ~1 BTC on CEX.io has taken me a 2-3 minutes before, so if someone has 10BTC and after a couple of minutes the market goes flat again: should he stop buying GHS?

Note that flat does not mean the trend is about to reverse. It is considered a downtrend when it hits the sellThreshold and not a second before that. It might as well be an uptrend for 1H, be flat for 1H and go up again after that, I don't want to make those assumptions in any of Gekko's code that does not deal with making assumptions (the only code that deals with that are the trading methods itself).

EDIT: the last thing I want to say on this in regards to CEX.io: buying GHS with all payout doesn't cost a penny on CEX.io currently (the only exchange we wanted this specific behaviour for), it only adds to do speculative position all your other funds are already in.

djmuk commented 10 years ago

"Note that flat does not mean the trend is about to reverse" - neither does it mean that the trend is continuing. ALL you can say if the market is flat is that there is NO advice. Therefore if the market then starts to rise at that point you can validly issue a 'long' advice and not before - you certainly can't assume that any previous 'long' advice is still valid.

The mailer shouldn't mail you when it goes flat - the assumption is/must be that any previous long position has been fully taken BUT if it hasn't (due to chasing the market) then your question 2 applies - and yes you probably should stop buying and wait and see if the upswing continues or reverses.

djmuk commented 10 years ago

Mike - I know this is exchange specific and probably unique to cexio... but

Cexio can do sideways for long periods (several hours and longer) with little or no movement, however during this time it makes sense to be invested in GHS because they will mine for you.

So it is logical to make a manual buy to go back into the asset (GHS) if you are not invested at this point. BUT if you are not invested then that implies the last advice from Gekko was to Short, however you WOULD want a (duplicate from Gekko's viewpoint) short notification if the market started to move downwards.

askmike commented 10 years ago

The mailer shouldn't mail you when it goes flat - the assumption is/must be that any previous long position has been fully taken BUT if it hasn't (due to chasing the market) then your question 2 applies - and yes you probably should stop buying and wait and see if the upswing continues or reverses.

Why shouldn't the mailer tell you the advice just became invalid (according to your logic)? What if you wake up and have an email from 3 hours ago, do you really need to check charts / gekko logs to see if it is still valid? It's about riding trend until it reverses IMO. If you can't assume the message is valid anymore what's the point of sending an email at all?

Cexio can do sideways for long periods (several hours and longer) with little or no movement, however during this time it makes sense to be invested in GHS because they will mine for you.

If you want to be long in a flat line you should configure the thresholds so that flat / up = long and down = short. No reason to put this logic into Gekko IMO.

That said I agree the current setup can be improved by adjusting the events the trading methods can send (eg. adding new events to reflect a flat line state). Though I want to do that after 0.1.0 (you can work on it and send a PR before that). But it needs to start with precise idea of all possible events and this needs to be reflected in the spec before implementing anything.

djmuk commented 10 years ago

isn't the 'soft advice' / hold exactly that flat line state?

I think that this debate may be academic (at least for PPO/MACD). Having re-read the docs while tuning my bot I realised that the main trigger for MACD/PPO is when the histogram (ppo - ppoema) CROSSES 0, if the value>0 then it is an upswing, if it <0 it is a downswing. the threshold is an addition that I bought in from the EMA method and should probably be set to a very small value...

askmike commented 10 years ago

The soft advice is:

A way for the trading advisor to emit statistics (like MACD calculated numbers) on every tick.
Not implemented yet, it is emitted from the methods but not propagated through Gekko because it's not thought out correctly and not in the spec.

kuzetsa commented 10 years ago

The way I originally coded this on my own branch (pre-localDB) was to have my cex.io gekko bot use a special timer which wasn't used by any other modules, and did nothing other than refresh the BTC & GHS balance. The only other change I did was to disable (hardcoded disable, but turned off just the same) any of the checks which tried to prevent the bot from acting multiple consecutive times in a row...

Those changes are useful, but ONLY for what I wanted it for, and I DID NOT mean to imply that it could be useful on other trading methods, exchanges, or most of the various "actors" or plugin modules or whatever (mailer, etc. would send rather a lot of messages, or mess things up if there were fees on the exchange, etc.)

I don't see any reason why the logic to "check with every new candle" or "check every N minutes" couldn't be configurable like I had already planned before the localDB branch was written.

We can make the logic for "buy and hold so long as there is no downtrend" behavior be something written as conditional code, and configurable from the config.js like I had originally intended. I never meant for this logic to be applied unconditionally to any & every module in gekko. It was only ever meant to be useful for miners who wanted to easily reinvest their mining income.

(the thing I'm describing is useful regardless of if the income was due to the miner's self-hosted hardware pointed at the ghash.io / cex.io pool, manually deposited to the cex.io deposit address, payouts deposited from another pool, or just the income associated with the user's held GHS balance)

askmike commented 10 years ago

@kuzetsa I agree, when I merged your change into Gekko I thought it was handy for re investing earnings. But it is causing to much trouble for all other plugins using the new event system.

so TLDR:

I'm all open for a change so we also propagate a flat trend besides up and down. Though it needs to be thought out, put in the spec and implemented.

mlmurray commented 10 years ago

Added in potential fixes for this in my fork. Added two new configuration options:

tradeOnStart: set to 'true', the system will trade on the first advice it gets when it starts up (or recovers from a crash). Set to false, it sets the direction first, and then trades when that direction changes.
tradeAfterFlat: if set to 'true', system goes from a direction to 'none' if the system drops to a 'flat' market position (so it trades on the next change). Set to false, it does @askmike's current solution.

askmike / gekko

MACD emits the same advice event multiple times #171