search

askmike / gekko

A bitcoin trading bot written in node - https://gekko.wizb.it/
MIT License
10.07k stars 3.94k forks source link

[Binance] 'error creating:' 'Error -2013: Order does not exist.' #2328

Closed PhunkyBob closed 6 years ago

PhunkyBob commented 6 years ago

This is a bug following this one : https://github.com/askmike/gekko/issues/2308

I'm submitting a ... [x] bug report [ ] question about the decisions made in the repository

Action taken (what you did) My strategy tried to order long.

Expected result (what you hoped would happen) I expect the order to be placed, filled, and my strategy to continue.

Actual result (unexpected outcome) The order was placed, filled but my strategy crashed:

2018-07-17 20:08:02 (DEBUG):    Processing 39 new trades. From 2018-07-17 18:07:41 UTC to 2018-07-17 18:08:01 UTC. (a few seconds)
2018-07-17 20:08:02 (INFO):     Trader Received advice to go long. Buying  XRP
2018-07-17 20:08:02 (DEBUG):    Creating order to buy 243.14055460837704 XRP
2018-07-17 20:08:07 (DEBUG):    [ORDER] statusChange: SUBMITTED
2018-07-17 20:08:07 (DEBUG):    [ORDER] statusChange: OPEN
2018-07-17T18:08:09.217Z 'error creating:' 'Error -2013: Order does not exist.'
/home/gekko/gekko/exchange/orders/sticky.js:185
        throw err;
        ^

Error: Error -2013: Order does not exist.
    at /home/gekko/gekko/exchange/wrappers/binance.js:72:15
    at Request.request [as _callback] (/home/gekko/gekko/exchange/node_modules/binance/lib/rest.js:90:25)
    at Request.self.callback (/home/gekko/gekko/exchange/node_modules/request/request.js:185:22)
    at emitTwo (events.js:126:13)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (/home/gekko/gekko/exchange/node_modules/request/request.js:1157:10)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
    at IncomingMessage.<anonymous> (/home/gekko/gekko/exchange/node_modules/request/request.js:1079:12)
    at Object.onceWrapper (events.js:313:30)

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, etc)

Unlike this issue: https://github.com/askmike/gekko/issues/2308 the order was placed before the strategy crashed.

I'm on commit 449c4243a48c73ab4e525a69d244aa0b51a473d2

askmike commented 6 years ago

@PhunkyBob sorry about all of this! I actually run into the same error in my live tests overnight. This branch has fixed that BUT binance might have more problems.

Thanks a lot for helping me - but I wouldn't call Binance very stable right now. as soon as I merge #2330 it should be stable! You are free to keep helping me (I would like it) but no pressure since it's not stable right now.

askmike commented 6 years ago

If you want to keep helping me you need to run that branch as it's not part of develop yet.

PhunkyBob commented 6 years ago

I'm working on an external function that would "buy market" instead of "loop for a buy limit". I'll see if it's interesting to add it in Gekko.

Pseudo code:

- Get information on the pair (lot size, mize amount, step amount, ...).
- Get current balance.
- Get current price.
- Deduce the quantity and *floor it* (not "round it") to the correct amount with correct number of decimal.
- Buy / sell market
- Maybe make a loop with some "await" to try until it's OK

I notice market orders are way faster to be filled.

askmike commented 6 years ago

We need to add a new order to gekko broker next to the sticky. Note that some exchange (like binance) have a native market order function that has advantages (smaller amounts).

Right now gekko broker uses callbacks internally, I don't want to change this to await just yet! Unless we change all the code to await style.

On Wed, 18 Jul 2018, 15:11 PhunkyBob, notifications@github.com wrote:

I'm working on an external function that would "buy market" instead of "loop for a buy limit". I'll see if it's interesting to add it in Gekko.

Pseudo code:

  • Get information on the pair (lot size, mize amount, step amount, ...).
  • Get current balance.
  • Get current price.
  • Deduce the quantity and floor it (not "round it") to the correct amount with correct number of decimal.
  • Buy / sell market
  • Maybe make a loop with some "await" to try until it's OK

I notice market orders are way faster to be filled.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/askmike/gekko/issues/2328#issuecomment-405832796, or mute the thread https://github.com/notifications/unsubscribe-auth/AA7MD85fFsJdhZecVGKKnq3fZ5Eo72NPks5uHt-ugaJpZM4VTgUI .

PhunkyBob commented 6 years ago

Obviously too much change...

askmike commented 6 years ago

With all the code I meant code limited to gekko broker, which I tried as a separate library (meaning only content inside the /exchange folder).

On Wed, Jul 18, 2018 at 5:04 PM, PhunkyBob notifications@github.com wrote:

Obviously too much change...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/askmike/gekko/issues/2328#issuecomment-405863167, or mute the thread https://github.com/notifications/unsubscribe-auth/AA7MDwaWr1eBfe6PE5k_7ZXLUc6Mjucjks5uHvoqgaJpZM4VTgUI .

-- PGP key at keybase.io/mikevanrossum https://keybase.io/mikevanrossum/key.asc

PhunkyBob commented 6 years ago

Some trades went OK. Others not:

2018-07-18 21:21:06 (DEBUG):    Processing 60 new trades. From 2018-07-18 19:20:46 UTC to 2018-07-18 19:21:05 UTC. (a few seconds)
2018-07-18 21:21:06 (INFO):     Trader Received advice to go long. Buying  XRP
2018-07-18 21:21:06 (DEBUG):    Creating order to buy 251.7838197169445 XRP
2018-07-18 21:21:11 (DEBUG):    [ORDER] statusChange: SUBMITTED
handleCreate Error -2010: Account has insufficient balance for requested action.
/home/gekko/gekko/exchange/orders/sticky.js:132
      throw err;
      ^

Error: Error -2010: Account has insufficient balance for requested action.
    at /home/gekko/gekko/exchange/wrappers/binance.js:72:15
    at Request.request [as _callback] (/home/gekko/gekko/exchange/node_modules/binance/lib/rest.js:90:25)
    at Request.self.callback (/home/gekko/gekko/exchange/node_modules/request/request.js:185:22)
    at emitTwo (events.js:126:13)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (/home/gekko/gekko/exchange/node_modules/request/request.js:1157:10)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
    at IncomingMessage.<anonymous> (/home/gekko/gekko/exchange/node_modules/request/request.js:1079:12)
    at Object.onceWrapper (events.js:313:30)
PhunkyBob commented 6 years ago

And again...

2018-07-20 08:15:18 (DEBUG):    Processing 39 new trades. From 2018-07-20 06:14:59 UTC to 2018-07-20 06:15:15 UTC. (a few seconds)
2018-07-20 08:15:18 (INFO):     Trader Received advice to go long. Buying  BCC
2018-07-20 08:15:18 (DEBUG):    Creating order to buy 0.15797258578681178 BCC
2018-07-20 08:15:23 (DEBUG):    [ORDER] statusChange: SUBMITTED
handleCreate Error -2010: Account has insufficient balance for requested action.
/home/gekko/gekko/exchange/orders/sticky.js:132
      throw err;
      ^

Error: Error -2010: Account has insufficient balance for requested action.
    at /home/gekko/gekko/exchange/wrappers/binance.js:72:15
    at Request.request [as _callback] (/home/gekko/gekko/exchange/node_modules/binance/lib/rest.js:90:25)
    at Request.self.callback (/home/gekko/gekko/exchange/node_modules/request/request.js:185:22)
    at emitTwo (events.js:126:13)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (/home/gekko/gekko/exchange/node_modules/request/request.js:1157:10)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
    at IncomingMessage.<anonymous> (/home/gekko/gekko/exchange/node_modules/request/request.js:1079:12)
    at Object.onceWrapper (events.js:313:30)
PhunkyBob commented 6 years ago

Does the script make a refresh of the account balance before calculating the amount to buy?

askmike commented 6 years ago

Yes after a order completes:

https://github.com/askmike/gekko/blob/24685ac3d58f229022a7b4331d8a6151c0a148e6/plugins/trader/trader.js#L230

and also after it cancelled an order:

https://github.com/askmike/gekko/blob/24685ac3d58f229022a7b4331d8a6151c0a148e6/plugins/trader/trader.js#L285

What happened to orders before you got that error (completed normally, cancel, error, or something else)?

Just so you know, I am running a gekko on binance and it's swapping money all day without much issues:

TRADE RECEIVED BY processTrade: { id: 'trade-25',
adviceId: 'advice-25',
action: 'sell',
cost: 0.11152103232,
amount: 0.014912,
price: 7478.61,
portfolio: { currency: 117.21934145, asset: 0.00078548 },
balance: 123.0890528296,
date: moment("2018-07-20T09:04:15.696"),
feePercent: 0.1,
effectivePrice: 7471.13139 }

(That is trade number 25 after I restarted it a few hours back to test latest version on #2330.

PhunkyBob commented 6 years ago

Let's suppose the following scenario:

Will Gekko try to buy for 1000 USDT because it was the portfolio amount fetched 2 weeks ago or will it check for the CURRENT portfolio to know the amount he can buy (900 USDT)?

I think my problem is there: Gekko don't update my portfolio before placing an order. As I run multiple instances of Gekko (1 instance / pair), if I lose money from a trade, all others instance will use a wrong value as portfolio (because not updated) and will fail. It works only when I keep winning...

askmike commented 6 years ago

I think my problem is there: Gekko don't update my portfolio before placing an order. As I run multiple instances of Gekko (1 instance / pair), if I lose money from a trade, all others instance will use a wrong value as portfolio (because not updated).

Great point, this might be the reason. We should definitely check the portfolio once in a while to make sure what Gekko thinks is the situation is actually the situation.

Though note: You should definitely not be trading on the same market gekko is active on, this will interfere with Gekko and will cause it to miscalculate balances and potentially profit as well. But it would be better to periodically check your portfolio to make sure nothing strange is happening.

On Fri, Jul 20, 2018 at 8:27 PM, PhunkyBob notifications@github.com wrote:

Let's suppose the following scenario:

  • I have 1000 USDT on Binance.
  • I run an instance of Gekko (it displays "1000 USDT").
  • 1 week later
  • I manually make an order that costs me 100 USDT. My balance is now 900 USDT.
  • 1 week later
  • My Gekko strategy find an opportunity to buy BNB and try to place an order.

Will Gekko try to buy for 1000 USDT because it was the portfolio amount fetched 2 weeks ago or will it check for the CURRENT portfolio to know the amount he can buy?

I think my problem is there: Gekko don't update my portfolio before placing an order. As I run multiple instances of Gekko (1 instance / pair), if I lose money from a trade, all others instance will use a wrong value as portfolio (because not updated).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/askmike/gekko/issues/2328#issuecomment-406585937, or mute the thread https://github.com/notifications/unsubscribe-auth/AA7MD9UYjAdloF2pzIplqxzMDEGO_Y56ks5uIcy4gaJpZM4VTgUI .

-- PGP key at keybase.io/mikevanrossum https://keybase.io/mikevanrossum/key.asc

PhunkyBob commented 6 years ago

But it would be better to periodically check your portfolio to make sure nothing strange is happening.

Is there a function to check the portfolio and update the current instance?

You should definitely not be trading on the same market gekko is active on, this will interfere with Gekko

I have multiple instances of Gekko using the same base currency (and the same strategy): USDT/BNB, USDT/NEO, USDT/ETH, USDT/LTC, ... When I open an order, I create a lock file meaning "USDT balance is currently used by an instance". When I close the order, I delete the lock file.

In my strategy, I can place an order only if there is no lock file. If I could run my strategy on only 1 pair, it would be super-annoying.

askmike commented 6 years ago

Is there a function to check the portfolio and update the current instance?

Yes! It's inside the trader this.sync(), it updates the following:

I have multiple instances of Gekko using the same base currency (and the same strategy): USDT/BNB, USDT/NEO, USDT/ETH, USDT/LTC, ...

Just wondering: how accurate do you expect profit reporting to be? If you have a strat that triggers a buy and later a sell, but the sell can't happen because another gekko already sold it the strat might seem to perform poorly.

On Fri, Jul 20, 2018 at 9:19 PM, PhunkyBob notifications@github.com wrote:

But it would be better to periodically check your portfolio to make sure nothing strange is happening.

Is there a function to check the portfolio and update the current instance?

You should definitely not be trading on the same market gekko is active on, this will interfere with Gekko

I have multiple instances of Gekko using the same base currency (and the same strategy): USDT/BNB, USDT/NEO, USDT/ETH, USDT/LTC, ... When I open an order, I create a lock file meaning "USDT balance is currently used by an instance". When I close the order, I delete the lock file.

In my strategy, I can place an order only if there is no lock file. If I could run my strategy on only 1 pair, it would be super-annoying.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/askmike/gekko/issues/2328#issuecomment-406598383, or mute the thread https://github.com/notifications/unsubscribe-auth/AA7MD9DYU4mwHIRktCjxwkOR-TXjRwvuks5uIdjogaJpZM4VTgUI .

-- PGP key at keybase.io/mikevanrossum https://keybase.io/mikevanrossum/key.asc

askmike commented 6 years ago

Another issue with this is that you might run into ratelimits: Gekko uses 1 gekko broker, which makes sure not to hit the API more than allowed. If you run multiple we can't be sure that all of them combined stay under the limit.

Not saying you can't run multiple gekkos, but generally speaking this is not the best idea. I'll push a fix that syncs the balance every 5 minutes.

On Fri, Jul 20, 2018 at 9:25 PM, Mike van Rossum mike@mikevanrossum.nl wrote:

Is there a function to check the portfolio and update the current instance?

Yes! It's inside the trader this.sync(), it updates the following:

  • portfolio balances
  • ticker
  • fee

I have multiple instances of Gekko using the same base currency (and the same strategy): USDT/BNB, USDT/NEO, USDT/ETH, USDT/LTC, ...

Just wondering: how accurate do you expect profit reporting to be? If you have a strat that triggers a buy and later a sell, but the sell can't happen because another gekko already sold it the strat might seem to perform poorly.

On Fri, Jul 20, 2018 at 9:19 PM, PhunkyBob notifications@github.com wrote:

But it would be better to periodically check your portfolio to make sure nothing strange is happening.

Is there a function to check the portfolio and update the current instance?

You should definitely not be trading on the same market gekko is active on, this will interfere with Gekko

I have multiple instances of Gekko using the same base currency (and the same strategy): USDT/BNB, USDT/NEO, USDT/ETH, USDT/LTC, ... When I open an order, I create a lock file meaning "USDT balance is currently used by an instance". When I close the order, I delete the lock file.

In my strategy, I can place an order only if there is no lock file. If I could run my strategy on only 1 pair, it would be super-annoying.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/askmike/gekko/issues/2328#issuecomment-406598383, or mute the thread https://github.com/notifications/unsubscribe-auth/AA7MD9DYU4mwHIRktCjxwkOR-TXjRwvuks5uIdjogaJpZM4VTgUI .

-- PGP key at keybase.io/mikevanrossum https://keybase.io/mikevanrossum/key.asc

-- PGP key at keybase.io/mikevanrossum https://keybase.io/mikevanrossum/key.asc

PhunkyBob commented 6 years ago

Just wondering: how accurate do you expect profit reporting to be?

By "profit reporting", you mean the tool that calculates the theoretical profit? I don't care. As soon my strategy is making profit, I don't care what Gekko thinks my profit should be.

If you have a strat that triggers a buy and later a sell, but the sell can't happen because another gekko already sold it the strat might seem to perform poorly.

The sell can always happen. A buy can only be done if there is no other instance trading, and a sell can only be done if there was a buy by this strategy.

Pseudo code of my strategy that will maybe explain better: 

status = '';
function onEveryCandle()
{
    is_bankroll_locked = file_exists('/tmp/binance_usdt.lock');
    condition_to_buy = complicated_function_to_see_if_its_right_time_to_buy();
    condition_to_sell = complicated_function_to_see_if_its_right_time_to_sell();

    if (status == '' && condition_to_buy == true && is_bankroll_locked == false)
    {
        create_file('/tmp/binance_usdt.lock');  // will prevent other instances to trade
        advise('long');
        status = 'long';
    }

    if (status == 'long' && condition_to_sell == true)
    {
        advise('short');
        status = '';
        delete_file('/tmp/binance_usdt.lock');  // will allow other instances to trade
    }
}