ctubio / tribeca

Self-hosted crypto trading bot (automated high frequency market making) in node.js, angular, typescript and c++
https://127.0.0.1:3000
Other
95 stars 26 forks source link

Bitfinex ? :P #63

Closed Camille92 closed 7 years ago

Camille92 commented 7 years ago

48

Hi Carles,

I'm going to pay you some tips for that :)

I'm very interested in working with Bitfinex, and you should be too after my explanation.

In theory, Bitfinex can be the best exchange for Tribeca now that trading on all currencies works. It's the only exchange that has both high volumes and allows for "triangular" markets for Europeans.

Ie: Trading BTC - Something and Something - USD.

This allows having 3 Tribeca working together to get most of a market. Let's take the example of ETH. If I have Tribeca trading on BTC/USD, ETH/USD and ETH/BTC and properly set I can be sure that:

If both ETH and BTC are going down I'll be full of USD. If BTC is down and ETH up, I'll be full of ETH and vice versa.

Bitfinex is great because it allows for doing this stuff on ETH, ETC, LTC, DASh, XMR and ZEC with high liquidity! In other words, I'm nearly assured that at least one market will be up at any moment.

And sure the 0.1% fee is an issue but it can be easily overcome by triangular trading and with a bit of luck trading enough to get the 0.08% or 0.06% fee!

As well, not very mature markets such as XMR and ZEC are usually good for market making because the width can be big (and thus easily overcome the fee).

Ps: Do you accept tips for working on #46 as well? It's, in my opinion, the best idea I had to improve Tribeca and I would love to see it live :D.

Cheers and have a lovely weekend!

ctubio commented 7 years ago

ok soo well xD about Bitfinex (you convinced me) i don't really need 0.02, i need the minimum possible to be able to trade 1 order in both sides (so 2 concurrent minimum orders are OK for test; i think that value is 0.02, but maybe you know better than me)

theeen about #46, im waiting for myself to fully reread it (and reresearch about) a few more times to fully understand it cos i didnt finnished that last exam of maths in highschool xD but im on my way'¡ if you support Bitfinex, we can consider us all tipped to move forward #46 with plenty of chocolated milk supply

Camille92 commented 7 years ago

Sent! 😃 I'll comment for #46 under #46

ctubio commented 7 years ago

many thanks for your collaboration'¡ will let it run for a while and see if any issues happens, but looks good for now :D

let me know if you feel like something is missing

ctubio commented 7 years ago

oh well, the issue of #48 seems still happening, not while calculating live the amounts, but when reading from the exchange

also trades executed are not displayed in the UI

Camille92 commented 7 years ago

Thanks for working hard on the project!

I'll have a look and tell you when I have 5mn!

Cheers

ctubio commented 7 years ago

question! did you notice that the trades complete do not have decimals in bitfinex (all are .00)? is because of the fee? or is there any way to see the decimals? thanks'¡

ctubio commented 7 years ago

finally moved from API v1 with http calls to API v2 with websockets (that is still in beta), so it may be updated and therefore crash for us anytime soon but i will try to maintain

for now seems working aaaaalmost super OK :D only self open orders are not highlighted in the market levels and also rarely open orders are possible to be accumulated in quantities greater than 1 even when not using AK-47 in both sides; but is super synchronized tribeca and the exchange :D hope is the same for you'¡ (also hope we can fix this 2 little issues soOn :P)

ah yea, also the completed trades have no decimals, and i do not know yet why :sob:

ctubio commented 7 years ago

to make it work, a new config option is needed see https://github.com/ctubio/tribeca/commit/91ff55339f7597ae0a508bd820fa7993245bc6fe

Camille92 commented 7 years ago

Hello,

I haven't had the time to try Bitfinex so far! I'm going to and I'll tell you if i find something not working.

Thank you for the work, and having WebSocket is great

Camille92 commented 7 years ago

Hi Carles,

Today is a big day as I'm trying to make Bitfinex working 💯 I'm going to follow with a few comments to help to debug on this new exchange.

First impression, the width on many pairs is very big, it easily reaches 1% so it should be VERY profitable and overcome the fees ! 😃

Other things I need to find where is the option to deactivate the notification because it becomes unreadable hahaha.

capture d ecran 2017-05-08 a 16 21 07

Camille92 commented 7 years ago

I've been checking for the following pairs and here are the results.

There is mostly an issue with the decimal shown in Tribeca / sent to the exchange / available on the exchange and it doesn't work with ETC and DASH.

BTC/USD ETH/USD ETH/BTC ETC/USD ETC/BTC ZEC/USD ZEC/BTC XMR/USD XMR/BTC LTC/USD LTC/BTC DASH/USD DASH/BTC

For convenience, I'm going to do give for each pair what is the problem. I'm going to write 3 numbers for each pair that represent the decimals. The first number is the minimum decimal Tribeca "sees" on the UI. The second number is the decimal Tribeca send orders to the exchange The third number is the decimal allowed on the exchange for this pair.

To function properly, all numbers should equal the third one :)

I can give screenshots for each pair but it might be a bit too big. Please ask me if you need more information.

state pair display send allowed
:fire: BTC/USD 0.01 1 0.1
:fire: ETH/USD 0.1 1 0.001
:fire: ETH/BTC 0.1 ? 0.000001
:fire: ETC/USD ~Tribeca doesn't start:~ ? ? 0.0001
:fire: ETC/BTC ~Tribeca doesn't start:~ ? ? 0.0000001
:fire: ZEC/USD 0.0001 1 0.01
:fire: ZEC/BTC 0.0001 ? 0.000001
:fire: XMR/USD 0.1 1 0.001
:fire: XMR/BTC 0.1 ? 0.000001
:fire: LTC/USD 0.1 1 0.001
:fire: LTC/BTC 0.1 ? 0.000001
:fire: DSH/USD ~Tribeca doesn't start:~ ? ? 0.001
:fire: DSH/BTC ~Tribeca doesn't start:~ ? ? 0.000001

I'm sorry if it might sound a bit depressing but I guess it's something to go though when there is new pair or new exchanges! The same problem arose when LTC trading started :)

I hope that it helps so you don't have to look for each number yourself and that you can save time!

ctubio commented 7 years ago

so, as per your report, currently there isnt any pair that correctly matches all 3, no? all seems broken xD

for exampleee the first one: it sends 1, and is allowed to have 0.1 (that is 1, no?) so, what we need to do for BTC/USD is to display 1 decimal less? because now it displayes 0.01 but the allowed is 0.1? is this correct? thanks'¡

Camille92 commented 7 years ago

Haha sadly yes no pair matching perfectly.

Yes here a couple of screenshots for the first one: Tribeca thinks it sends at 1657.89 but it is actually an order at 1658 at the exchange (maybe Bitfinex round invalid orders). capture d ecran 2017-05-08 a 19 06 43

While the exchange allows for 0.1 capture d ecran 2017-05-08 a 19 09 10

But yes exactly as you describe it :)

ctubio commented 7 years ago

ok i experiencied too what you say for BTC/USD, and while developing the integration, i noticed that Bitfinex rounds all orders (see my previous comment in this thread), im not sure if this is because of the fees or why, but efectively bitfinex truncates what we send and it doesnt display any decimals in the website of bitfinex, moreover, in the api when we ask for the status of an order, bitfinex returns the price without decimals too, so i belive bitfinex doesn't have saved orders with decimals, that means that or bitfinex ignores the decimals, or we are not sending them in the way that bitfinex want, cos is in fact is ignoring them.

about other currencies, tribeca the first thing that does when connect to exchange is read what is the decimal p`recision and what is the minimum order price; so it is dinamic, and should work.

I belive we may have this problems because of the beta version of the API, because while debugging, really seems we are doing all OK in our side, but we do not get the expected information back from the exchange.

let me reinvestigate to discard that the problem is not in our side, so we can peacefully wait for the stable api or to migrate to v1 (i preffer to wait for stable v2 if you ask me xD and fix/live with this little issues meanwhilke)

also the api v2 is currently missing the cancel reject event i think :sob: but i made it work without this response :D (when canceling an order that doesnt exists, bitfinex shows a beautiful notification in the corner saying Order Not Found, but it doesnt says nothing throw the websocket, so you can wait forever waiting to cancel an unexistent order xD [maybe they put a meaning to the "silent", but i preffer explicit communication xD])

ty for the screenshot'¡

ctubio commented 7 years ago

before wait and before migrate to v1 (ok lets face it, we are not gonna go back to v1 xD), we should try to play setting the DEC_S flag (see https://bitfinex.readme.io/v2/reference#config), or to contact bitfinex support :dancer: maybe they suggest us to go to v1 and then i will shut up and proceed xD

Camille92 commented 7 years ago

Some info I found on the website for API.

Price Precision The precision level of all trading prices is calculated based on significant figures.

For all pairs on Bitfinex we use 5 significant digits. Examples of five significant digits would be 12.123, 1.1234, 123.43, and 1234.5.

This mimics how traditional global markets dynamically handle precision of small, medium, and larger values. The rationale behind this is that the higher the amount the less relevant the number of decimals becomes. The corollary is true for very small amounts, where more precision is more useful.

Note: API will truncate price with precision > 5

So we can also have this dynamic rule to not make mistakes :) and if prices moves (above or under $1000 or $100).

I would be very surprised if Bitfinex releases an API even in Beta not allowing for decimal trading because it means that it is impossible to trade all sub 1 markets (Something/BTC market).

I didn't get exactly what is DEC_S doing haha.

But yes agreed no need to go to look for V1.

Camille92 commented 7 years ago

I've been thinking about a solution of why this issue is arising.

First of all, I want to precise that Bitfinex doesn't show by default the decimal but if you click on this little + you should get them :) capture d ecran 2017-05-08 a 22 54 00

The most logical explanation I can come with is that as Bitfinex minimal precision is dynamic, the number Tribeca receives might not be the correct one which causes orders to not be well topped.

This would make sense as in the Bitfinex gateway the price of an order is related to min tick increment.

https://github.com/ctubio/tribeca/blob/master/src/server/gateways/bitfinex.ts L252.

Given that the price is always taking into account 5 significant digit there is an easy way to define min tick from the price. if price =/> 0.1 -> min tick = 0.00001 if price =/> 1 -> min tick = 0.0001 if price =/> 10 -> min tick = 0.001 if price =/> 100 -> min tick = 0.01 If price =/> 1000 -> min tick = 0.1

And so on!

I don't know if it's the source of the mistake but imho there might be a good chance.

ctubio commented 7 years ago

Hello Camille, in case of bitfinex the mintick comes from https://api.bitfinex.com/v1/symbols_details as price_precision but instead we use minimum_order_size and always round up; so we never send orders under the minimum limit (even while using %); anyway whatever we send bitfinex always ignores the decimals.. will try to investigate why (but currently the decimals are already dynamic and found to be working as seen in the UI, is bitfinex that is ignoring the decimals in the websocket)

many thanks for the hint of bitfinex ui to display the decimals'¡

Camille92 commented 7 years ago

Thank you for explaining me.

Then we can identify 2 different issues: _ The first one beeing Bitfinex always ignoring decimal.

_ The second one is that minimum order size is definitely not a good proxy for decimal precision (but goes in line with what we've seen testing Tribeca) as shown in the post with the table 😛 . Minimum order size defines a quantity, not the precision of the price.

Well, in the end, I'm not sure if I understand clearly what you meant or you from me but I was talking in the previous post about decimals and price precision, not the minimum quantity.

Have a good night I'll try to understand that better tomorrow with a clear mind.

ctubio commented 7 years ago

Hello Camile, yea we are talking about the same, you will understand now xD (but please enlight me later xD) i used minimum_order_size instead of price_precision, because all pairs have the value of 5 (see https://api.bitfinex.com/v1/symbols_details); so when using price_precision, everywhere in the UI the USD values for example have 5 decimals as well as ETH and others, thats why i used minimum quanity to calculate the decimals instead, cos seems nicer option, but maybe i was wrong xD

looking at your table, i see that is wrong if we put 5 decimals everywhere no? but is what https://api.bitfinex.com/v1/symbols_details says if we use price_precision soo.. im a bit confused at this point :P from where did you take the allowed values of your table? to me doesn't look bad to use the values of minimum_order_size (with the formula -1 * Math.floor(Math.log10(minimum_order_size)) seems fine the result, for example 0.01 returns 2; and 0.001 returns 3; but again, i may be wrong).

btw found the problem with DASH, also looking at https://api.bitfinex.com/v1/symbols_details, seems like the market is called DSH instead. about ETC, it doesnt work because needs to be added to tribeca, will fix ETC in next commit'

Camille92 commented 7 years ago

Hello hello,

I got what you tried to do!

It took me some time to understand what's going on here but I have the answer.

I think that we need first to define price precision and minimum order size to be clear about what we're talking about. Minimum order size is the minimum quantity of a coin you can send to the market it's the minimum for the BidSz and AskSz in Tribeca. price precision is the minimum price difference you can have between two orders.

Ie: For BTC/USD Minimum order size is 0.01 so it means that I can't send smaller orders to the market. But Prince precision is 0.1 -> The minimum value separating orders in the order book.

I've seen what you're trying to do and it works well IRL because the minimum price difference is equal to the price precision. An object will have a price with two decimals maximum and my coins allow me to count with two decimals only.

But in the crypto world, it doesn't work quite well haha: minimun_order_size is hard coded and is actually a negative correlation with price_precision. Let me explain why: You want to have price_precision to be a sizeable logical amount that most people use and that roughly the minimum orders have the same value around your exchange: Ie: 0.1 ETH or 9USD or 0.01BTC or 16USD..

But the higher the price is, the less price_precision you want (because sending orders separated by 0.001$ is useless on BTC/USD!) You want the orders to be roughly separated by the same %. 12.345 / 12.346 is separated by the same % than 12345 and 12346.

Now let's see how Bitfinex did it:

First, for my table, I checked the results manually for each pair. But now I understood the system they are using reading the "Readme": http://docs.bitfinex.com/docs/readme

Here is the system they use: All pair has a price precision of 5 it only means that the price is defined by 5 numbers: For instance: 123.45 (price precision: 0.01) or 0.12345 or 12345 or 123450 (price precision 10) and everything you can imagine with a combination of 5 numbers.

What is important here is to know is that the price is always defined by 5 numbers so the first decimal (tens, hundreds, thousands ...) defining the minimum decimal.

So to sum up: Minimun_size is not a good way to guess price precision -> the lower minimun_size is the bigger the price is. Price precision for each market in Bitfinex is defined by the price of the asset (and change each time the price change decimal). It's what is was trying to explain in the previous post talking about min tick. If an asset cost 10$ then we take the first five number and we get the minimum price precision: 10,000 -> min diff: 0.001

If this asset goes to 9.99 the price precision will be: 9.9999 -> min diff: 0.0001

Lastly on the read me Bitfinex says: "Note: API will truncate price with precision > 5"

So I guess what's happening here for instance in BTC/USD is that Tribeca sends orders with two decimals and get truncated. And that Bitfinex truncates even when the last number is 0. For instance: Sent order of 1600.10 -> API Sees 6 numbers and cuts it to just 1600 and not 1600.1.

I think we're close to close this issue but I agree it's a bit hard to understand and explain on paper.

Read carefully http://docs.bitfinex.com/docs/readme and when you understand it blows your mind for how logical it is (at least it did for me)!

ctubio commented 7 years ago

ok coOl (awesome xD) my little brain got bigger today xD

will try to apply your (now mine) knowledge, many many thanks'¡!¡

Camille92 commented 7 years ago

Haha happy to hear that!

I don't know how to code, so it's the least I can do to help!

teamwork

ctubio commented 7 years ago

:sunny: many many thanks for all your help Camille'¡

Camille92 commented 7 years ago

Many thanks for working on it :)