search

stellar-deprecated / kelp

Kelp is a free and open-source trading bot for the Stellar DEX and 100+ centralized exchanges
https://kelpbot.io
Other
1.09k stars 262 forks source link

error while getting price from exchange feed #92

Open sacarlson opened 5 years ago

sacarlson commented 5 years ago

// see a sample bug report here: https://github.com/interstellar/kelp/issues/1

Describe the bug

I'm seeing this in the returned log event 2019/01/14 16:29:35 ---------------------------------------------------------------------------------------------------- 2019/01/14 16:29:36 (base) assetA=native, maxA=199.9998900, trustA=math.MaxFloat64 2019/01/14 16:29:36 (quote) assetB=FUNT:GBUYUAI75XXWDZEKLY66CFYKQPET5JR4EENXZBUZ3YXZ7DS56Z4OKOFU, maxB=20.0000000, trustB=922337203685.4775391 2019/01/14 16:29:38 liabilities after resetting 2019/01/14 16:29:39 asset=base , balance=199.9998900, trust=math.MaxFloat64, minAccountBal=26.5000000, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000 2019/01/14 16:29:40 asset=quote , balance=20.0000000, trust=922337203685.4775391, minAccountBal=0.0200000, buyingLiabilities=0.0000000, sellingLiabilities=0.0000000 2019/01/14 16:29:40 returning kraken API key at index 0 2019/01/14 16:29:41 price from exchange feed (kraken/XXLM/ZUSD): bidPrice=0.1037580, askPrice=0.1039720, centerPrice=0.1038650 2019/01/14 16:29:41 feedPair prices: feedA=1.0000000, feedB=0.1038650; centerPrice=9.6278823 2019/01/14 16:29:41 center price (adjusted): 12.1349664 2019/01/14 16:29:41 returning kraken API key at index 0 2019/01/14 16:29:42 error: center price couldn't be loaded! | error while getting price from exchange feed: Could not execute request #5! (Response Content-Type is 'text/html', but should be 'application/json'.) 2019/01/14 16:29:42 levels couldn't be loaded: error while getting price from exchange feed: Could not execute request #5! (Response Content-Type is 'text/html', but should be 'application/json'.) 2019/01/14 16:29:42 error in selling sub-strategy: error while getting price from exchange feed: Could not execute request #5! (Response Content-Type is 'text/html', but should be 'application/json'.) 2019/01/14 16:29:42 not deleting any offers, deleteCycles (=1) needs to exceed deleteCyclesThreshold (=1) 2019/01/14 16:29:42 ---------------------------------------------------------------------------------------------------- 2019/01/14 16:29:42 sleeping for 4m53.430648196s...

I'm guessing from this part: error: center price couldn't be loaded! | error while getting price from exchange feed: Could not execute request #5! (Response Content-Type is 'text/html', but should be 'application/json'.) That the kraken api is getting multiple requests for the same price. Kraken will detect too many attepts to read a prices over some time for each user. we only need the first price it gets for this pair you don't need to try and get the same pair price 5 time to expand levels of trade. I haven't looked at the code to verify this is really the case as I'm not a very good Go programer but if I see no progress and I have time I'll take a closer look.

Expected behavior

// A succinct description of what you expected to happen.

I think the correct behavior in this situation is to run at least 90% of the time with 5 level trade at frequency of 30 seconds

Frequency

it was doing it only 1 out of 2 times at first until I started doing more trial and error testing at a faster rate of like less then 1 minute between runs with 5 levels of trade. I can get 10 out of 10 failures if I attempt to run back to back 5 sec or less apart.

Steps To Reproduce

I would assume you can reproduce it with at least 5 levels of trade and run the program at about 20 seconds or less apart. If you need an example cfg file I can provide it.

Possible Solution

make sure you don't read kraken exchange feed or most any exchange feed more than 1 time for the same trading pair per run

Your Environment

// Run kelp version to get the version and build information and paste it here version: /kelp-v1.3.0-linux-amd64.tar running on Linux Mint version: master:v1.3.0 git branch: master git hash: e19784fc9cdb113d5b51c8f421c8ab2d49752dc0 build date: 20190111T004556Z GOOS: linux GOARCH: amd64

...

Context

// A succinct description of how has this bug has affected you or prevented you from accomplishing what you wanted. I am unable to run at frequency of less then 10 minutes at levels more then 5 deep and not sure I can even do that. I also run my own ruby bots that use kraken api exchange feed. I sure don't want to be blacklisted from using the kraken feed because this app is hitting it too hard.

Attachments

// Attach any relevant configuration files, logs, tx hashes, etc. here.

nikhilsaraf commented 5 years ago

When using Kraken as a price feed, the bot hits their API around 2 times per cycle, so this should not exhaust rate limit by itself.

When using the Kraken API, the first few requests always return an incorrect content type like what you are seeing. These issues are at the kraken go sdk layer and there's not much we can do to fix that without possibly making deep changes to the sdk.

In order to accommodate such transient issues you should set your DELETE_CYCLES_THRESHOLD config param to be > 0 (available since Kelp v1.2.0) to tolerate a few cycles with such transient errors. It would be useful to experiment with different values for this parameter to see what works best for you.

Please let me know if this fixes it for you.

This issue is similar to https://github.com/interstellar/kelp/issues/93 in that they are both related to transient network issues.

sacarlson commented 5 years ago

i think I've isolated the problem to a bug in the kraken-go-api-client you are using in this case for the kraken feed. I was able to duplicate the problem in a test that just runs just this client. I put in a bug report to the developer at https://github.com/beldur/kraken-go-api-client/issues/43. Also in my searches I found another go package that might be even better as an option that supports many more feed points including Kraken https://github.com/nntaoli-project/GoEx. I'm thinking we might want to see how hard that might be to incorporate as an option. I might give it a wack. I've also increased Delete_cycles_threshold = 6 that I can tolerate for some time. 6 failures in a row still happen at times as I have seen.

nikhilsaraf commented 5 years ago

Ok, leaving this issue open for now. Once https://github.com/beldur/kraken-go-api-client/issues/43 is addressed, we can upgrade our dependency to use the latest version of their library with the fix, thereby resolving this issue.