Bitfinex support for 4.x

[nedievas]

I prepared the extension to add Bitfinex to upcoming version, but I see that it will be necessary to change engine.js for it to work.

[carlos8f]

please elaborate?

[nedievas]

APIs are different in trading or even taking products/pairs from exchanges. I do not know how to get trading from exchange.js to engine.js, as I see engine trade function is made to work with gdax API (buy and sell). Bitfinex API has new_order to execute trading of both type. Maybe I should dig deeper, but I have poor programming skills as it's my hobby not the main resource to make living;)

[carlos8f]

Bitfinex API has new_order to execute trading of both type

not a problem, just have the buy/sell functions in exchange.js each send a different order type to new_order. Nothing that would necessitate changing engine.js yet.

There are bound to be some weird differences between exchange api's, and I'm currently dealing with one of them: some API's return historical data backwards and some forwards in time. Some API's don't have historical data, which I'm trying to fall back to a 3rd party API if possible. These things are why I haven't released 4.0 yet because the zenbot API needs to be stable (and account for exchange differences) at 4.0 release, even if the exchanges themselves aren't supported yet.

[nedievas]

I also noticed that it's difficult for me to take balances from wallet to get it to work in engine.js, as Bitfinex wallets have available amount and GDAX wallet has hold amount. I know that it is possible to take and make some mathematics to get correct values, but I want a simpler way because of my limited knowledge in coding.

P.S. Some comments in code would be of great help to know what is what ;)

[carlos8f]

I think this would work:

{
  "type":"exchange",
  "currency":"usd",
  "amount":"1",
  "available":"1"
}

held = amount - available so held = 0 in the above example

[tuxitor]

Something like this should do for the buy(sell:

` buy: function (opts, cb) var direction = 'buy' client.new_order("BTCUSD", 100, "market", direction" 0)

sell: function (opts, cb) var direction = 'sell' client.new_order("BTCUSD", 100, "market", direction, 0)`

For a more complete example, I would do something like this (for Bitstamp). I don't know if it will work yet. I may have problems with the callbacks and error handling:

    buy: function (opts, cb) {
      var client = authedClient()
      if (typeof opts.type === 'undefined' ) {
        opts.type = limit
        }
      if (opts.type === 'limit') {
        var limit_price = opts.price // Fix me
        //  GDAX client.buy(opts, function (err, resp, body) {
        client.buy(opts.product_id, opts.size, opts.price, limit_price, function (err, resp, body) {
          if (!err) err = statusErr(resp, body)
          if (err) return cb(err)
          cb(null, body)
        })
      } else {
        //  GDAX client.buy(opts, function (err, resp, body) {
        client.buyMarket(opts.product_id, opts.size, function (err, resp, body) {
          if (!err) err = statusErr(resp, body)
          if (err) return cb(err)
          cb(null, body)
        })
      }
    },

[nedievas]

I end up with: error: could not find product "BTCUSD"

[nedievas]

Uploaded files. Created pull request.

[nedievas]

Completed Bitfinex extension support. Looks like everything working except backfilling. Please, check pull request.

[nedievas]

OK. I have prepared Bitfinex extension for Zenbot 4. You may find it https://github.com/nedievas/bitfinex

Currently testing, to see any errors in trading.

Backfill is not supported. If this extension will work as intended, next step is to add it via https://cryptowat.ch/ API.

[nedievas]

paper trading working! Proof:

[tuxitor]

I started it a few minutes ago. Areminder if you are not aware of it, to get adwantage of the maker fee, you need to place the order BELOW the higest bid for buy orders. The same goes for sell orders, you need to place the sell order ABOVE the lowest ask to get the maker fee. Bitfinex claims that the reason is that they want you to increase the liquidity of the market. Maybe that needs to be handled by "engine.js".

[nedievas]

@eigilb I agree. The engine.js is to handle for @carlos8f . I did not want to make separate zenbot4 for bitfinex like I did with zenbot3.

[tuxitor]

After starting trade in papertrading mode, this shows up every now and then:

  last_period_id: 'm24933317',
  acted_on_stop: false,
  action: null,
  signal: null,
  acted_on_trend: false,
  trend: 'down',
  cancel_down: false,
  start_capital: 1000,
  start_price: 1996.1,
  last_signal: 'sell',
  quote: { bid: 2050, ask: 2051.7 },
  last_buy_price: '2050.0000',
  last_trade_worth: -0.008878048780487827,
  last_sell_price: '2044.0000' }

{ Error: 429
    at Request._callback (/home/eigilb/Prosjekt/Z/zenbot-carlos8f/extensions/bitfinex/node_modules/bitfinex-api-node/rest.js:82:23)
    at Request.self.callback (/home/eigilb/Prosjekt/Z/zenbot-carlos8f/extensions/bitfinex/node_modules/request/request.js:188:22)
    at emitTwo (events.js:106:13)
    at Request.emit (events.js:191:7)
    at Request.<anonymous> (/home/eigilb/Prosjekt/Z/zenbot-carlos8f/extensions/bitfinex/node_modules/request/request.js:1171:10)
    at emitOne (events.js:96:13)
    at Request.emit (events.js:188:7)
    at IncomingMessage.<anonymous> (/home/eigilb/Prosjekt/Z/zenbot-carlos8f/extensions/bitfinex/node_modules/request/request.js:1091:12)
    at IncomingMessage.g (events.js:291:16)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:185:7) desc: 'could not execute sell: error fetching quote' }

2017-05-28 21:18:00  2037.00 BTC-USD   +0.0%       85    --       -0.0310     sell  0.48461 BTC     9.94 USD    -0.3%   -2.3%
2017-05-28 21:18:32  2032.90 BTC-USD   -0.2%        0    --       -0.0464  selling  0.48461 BTC     9.94 USD    -0.5%   -2.3%

[nedievas]

Sorry. My mistake on sell. Corrected.

[tuxitor]

Still same message!

[nedievas]

@eigilb pull and restart

[tuxitor]

@nedievas Have pulled after update. It is still happening

[nedievas]

@eigilb show me from the beginning: $ zenbot trade --paper ?? Mine is still running, still I have no sell signal.

[nedievas]

The problem is when I ctrl-z, my paper trading balance resets to default :( The same is on default GDAX.

[tuxitor]

[eigilb@tb zenbot-carlos8f]$ ./zenbot.sh trade --paper  --trend_ema 21 --period 5m --max_slippage 0.001 --order_adjust_time 5000 --oversold_rsi_periods 13 --oversold_rsi 14 bitfinex.BTC-USD
fetching pre-roll data:
cannot backfill bitfinex.BTC-USD: exchange does not offer historical data
               DATE            PRICE     DIFF            VOL         RSI                         ACTIONS                      BAL                PROFIT
2017-05-28 18:05:00  2004.20 BTC-USD               36              0.0000         
2017-05-28 18:10:00  1957.90 BTC-USD   -2.3%      144              0.0000         
2017-05-28 18:15:00  1982.90 BTC-USD   +1.3%       94              0.0000         
2017-05-28 18:20:00  1975.00 BTC-USD   -0.4%       43              0.0000         
2017-05-28 18:25:00  1974.00 BTC-USD   -0.1%       46              0.0000         
2017-05-28 18:30:00  1996.70 BTC-USD   +1.1%       74              0.0000         
2017-05-28 18:35:00  2002.20 BTC-USD   +0.3%       49              0.0000         
2017-05-28 18:40:00  2011.20 BTC-USD   +0.4%       61              0.0000         
2017-05-28 18:45:00  2035.00 BTC-USD   +1.2%       87              0.0000         
2017-05-28 18:50:00  2043.20 BTC-USD   +0.4%      108              0.0000         
2017-05-28 18:55:00  2043.80 BTC-USD   +0.0%      163              0.0000         
2017-05-28 19:00:00  2018.90 BTC-USD   -1.2%       66              0.0000         
2017-05-28 19:05:00  2028.00 BTC-USD   +0.5%      104              0.0000         
2017-05-28 19:10:00  2041.00 BTC-USD   +0.6%       35    --        0.0000         
2017-05-28 19:15:00  2054.70 BTC-USD   +0.7%      100     -        0.0000         
2017-05-28 19:20:00  2046.00 BTC-USD   -0.4%       61    --        0.0000         
2017-05-28 19:25:00  2055.00 BTC-USD   +0.4%       47     -        0.0000         
2017-05-28 19:30:00  2057.10 BTC-USD   +0.1%       75     -        0.0000         
2017-05-28 19:35:00  2038.70 BTC-USD   -0.9%       33    --        0.0000         
2017-05-28 19:40:00  2030.80 BTC-USD   -0.4%       89    --        0.0000         
2017-05-28 19:45:00  2015.80 BTC-USD   -0.7%       86    --        0.0000         
2017-05-28 19:50:00  2007.70 BTC-USD   -0.4%      170    --        0.0000         
2017-05-28 19:55:00  1979.60 BTC-USD   -1.4%      188   ---       -0.1754     sell
2017-05-28 20:00:00  2017.60 BTC-USD   +1.9%      266    --        0.0117      buy
2017-05-28 20:05:00  2040.70 BTC-USD   +1.1%       76      +       0.1149      buy
2017-05-28 20:10:00  2015.90 BTC-USD   -1.2%      119    --       -0.0074     sell
2017-05-28 20:15:00  2003.00 BTC-USD   -0.6%      154    --       -0.0649     sell
2017-05-28 20:20:00  2016.20 BTC-USD   +0.7%      107    --        0.0005      buy
2017-05-28 20:25:00  2024.00 BTC-USD   +0.4%       52     -        0.0356      buy
2017-05-28 20:30:00  2050.00 BTC-USD   +1.3%      225      +       0.1496      buy
2017-05-28 20:35:00  2045.20 BTC-USD   -0.2%       71      +       0.1142      buy
2017-05-28 20:40:00  2037.20 BTC-USD   -0.4%       60      +       0.0677      buy
2017-05-28 20:45:00  2053.20 BTC-USD   +0.8%       25      +       0.1334      buy
2017-05-28 20:50:00  2042.10 BTC-USD   -0.5%       53      +       0.0713      buy
2017-05-28 20:55:00  2036.40 BTC-USD   -0.3%        5     -        0.0392      buy
2017-05-28 21:00:00  2039.90 BTC-USD   +0.2%       22      +       0.0513      buy
2017-05-28 21:05:00  2042.40 BTC-USD   +0.1%       13      +       0.0578      buy
2017-05-28 21:10:00  2047.70 BTC-USD   +0.3%        8      +       0.0763      buy
2017-05-28 21:15:00  2048.90 BTC-USD   +0.1%       37      +       0.0746      buy
2017-05-28 21:20:00  2033.70 BTC-USD   -0.7%       82     -       -0.0001     sell
2017-05-28 21:25:00  2037.30 BTC-USD   +0.2%       26     -        0.0160      buy
2017-05-28 21:30:00  2022.30 BTC-USD   -0.7%       68    --       -0.0525     sell
2017-05-28 21:35:00  2026.50 BTC-USD   +0.2%       12     -       -0.0290     sell
2017-05-28 21:40:00  2025.90 BTC-USD    0.0%       24     -       -0.0290     sell
---------------------------- STARTING PAPER TRADING ----------------------------
2017-05-28 21:45:00  2026.80 BTC-USD   +0.0%      101     -       -0.0224     sell  0.00000 BTC  1000.00 USD    +0.0%    0.0%
2017-05-28 21:45:15  2025.00 BTC-USD   -0.1%        0    --       -0.0284           0.00000 BTC  1000.00 USD    +0.0%   +0.0%{ options: 
   { paper: true,
     trend_ema: 21,
     period: '5m',
     max_slippage: 0.001,
     order_adjust_time: 5000,
     oversold_rsi_periods: 13,
     oversold_rsi: 14,
     strategy: 'trend_ema',
     sell_stop_pct: 0,
     buy_stop_pct: 0,
     profit_stop_enable_pct: 0,
     profit_stop_pct: 1,
     max_slippage_pct: 5,
     buy_pct: 99,
     sell_pct: 99,
     max_sell_loss_pct: 25,
     markup_pct: 0,
     poll_trades: 30000,
     currency_capital: 1000,
     asset_capital: 0,
     rsi_periods: 13,
     stats: true,
     selector: 'bitfinex.BTC-USD',
     mode: 'paper',
     min_periods: 52,
     neutral_rate: 0 },
  selector: 'bitfinex.BTC-USD',
  exchange: { name: 'bitfinex', makerFee: 0.1 },
  product_id: 'BTC-USD',
  asset: 'BTC',
  currency: 'USD',
  product: 
   { asset: 'BTC',
     currency: 'USD',
     min_size: 0.01,
     max_size: 2000,
     increment: 0.0001,
     label: 'BTC/USD' },
  balance: { asset: 0, currency: 1000 },
  ctx: {},
  day_count: 0,
  my_trades: [],
  vol_since_last_blink: 0,
  strategy: 
   { name: 'trend_ema',
     description: 'Buy when (EMA - last(EMA) > 0) and sell when (EMA - last(EMA) < 0). Optional buy on low RSI.' },
  last_day: 'd17314',
  period: 
   { period_id: '5m4986669',
     size: '5m',
     time: 1496000700000,
     open: 2025.4,
     high: 2025.4,
     low: 2025,
     close: 2025,
     volume: 0.15505502999999998,
     close_time: 1496000715000,
     trend_ema: 2030.7716775437582,
     oversold_rsi_avg_gain: 3.8526839199808203,
     oversold_rsi_avg_loss: 4.667562223678089,
     oversold_rsi: 45,
     trend_ema_rate: -0.028413029879712344,
     trend_ema_stddev: 0,
     id: 'ede4d7ad',
     selector: 'bitfinex.BTC-USD',
     session_id: '5221de76',
     rsi_avg_gain: 3.8526839199808203,
     rsi_avg_loss: 4.667562223678089,
     rsi: 45 },
  last_period_id: '5m4986669',
  acted_on_stop: false,
  action: null,
  signal: null,
  acted_on_trend: false,
  trend: 'down',
  cancel_down: false,
  start_capital: 1000,
  start_price: 2025.9,
  last_signal: 'sell' }

{ Error: 429
    at Request._callback (/home/eigilb/Prosjekt/Z/zenbot-carlos8f/extensions/bitfinex/node_modules/bitfinex-api-node/rest.js:82:23)
    at Request.self.callback (/home/eigilb/Prosjekt/Z/zenbot-carlos8f/extensions/bitfinex/node_modules/request/request.js:188:22)
    at emitTwo (events.js:106:13)
    at Request.emit (events.js:191:7)
    at Request.<anonymous> (/home/eigilb/Prosjekt/Z/zenbot-carlos8f/extensions/bitfinex/node_modules/request/request.js:1171:10)
    at emitOne (events.js:96:13)
    at Request.emit (events.js:188:7)
    at IncomingMessage.<anonymous> (/home/eigilb/Prosjekt/Z/zenbot-carlos8f/extensions/bitfinex/node_modules/request/request.js:1091:12)
    at IncomingMessage.g (events.js:291:16)
    at emitNone (events.js:91:20) desc: 'could not execute sell: error fetching quote' }

2017-05-28 21:50:00  2029.80 BTC-USD   +0.1%      168     -       -0.0069     sell  0.00000 BTC  1000.00 USD    +0.0%   -0.2%
2017-05-28 21:50:49  2029.80 BTC-USD   +0.0%        1     -       -0.0063           0.00000 BTC  1000.00 USD    +0.0%   -0.2%{ options: 
   { paper: true,
     trend_ema: 21,
     period: '5m',
     max_slippage: 0.001,
     order_adjust_time: 5000,
     oversold_rsi_periods: 13,
     oversold_rsi: 14,
     strategy: 'trend_ema',
     sell_stop_pct: 0,

Then it repeats

[nedievas]

@eigilb I suppose you've got an API block. Error: 429 TOO MANY REQUESTS Reference: https://httpstatuses.com/429

[nedievas]

245