Crypto-toolbox / bitex

Crypto-currency Exchange API Framework
MIT License
485 stars 136 forks source link

Suggestions on where to find a list of currency pairs for each exchange? #123

Closed bebosudo closed 6 years ago

bebosudo commented 6 years ago

Hi Nils, thanks for your work on this library. I'm using it only for getting tickers from different exchanges ATM, and I had some hard time in finding the correct pairs and format that each exchange API requires (and many still don't work).

Do you have any suggestion on a link of where to find the correct pairs?

Moreover, it seems that any pair in Poloniex is wrong.

>>> from bitex import Poloniex
>>> resp = Poloniex().ticker("BTC_BCN")
DEBUG:bitex.api.REST.api:Initialized API Client for URI: https://poloniex.com; Will request on API version: 
DEBUG:bitex.api.REST.api:Making request to: https://poloniex.com/public?command=returnTicker, kwargs: {'params': {}}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): poloniex.com
DEBUG:urllib3.connectionpool:https://poloniex.com:443 "GET /public?command=returnTicker HTTP/1.1" 200 None
DEBUG:bitex.api.REST.api:Made GET request made to https://poloniex.com/public?command=returnTicker, with headers {'User-Agent': 'python-requests/2.18.4', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'} and body None. Status code 200
ERROR:bitex.utils:Error while applying formatter!
Traceback (most recent call last):
  File "/home/achiusole/.venvs/crypto-env/lib/python3.6/site-packages/bitex/utils.py", line 61, in wrapper
    r.formatted = formatter(data, *args, **kwargs)
  File "/home/achiusole/.venvs/crypto-env/lib/python3.6/site-packages/bitex/formatters/poloniex.py", line 17, in ticker
    data = data[args[0]]
KeyError: <bitex.interfaces.poloniex.Poloniex object at 0x7fa38b5ffe48>
>>> resp.json().keys()
dict_keys(['BTC_BCN', 'BTC_BELA', 'BTC_BLK', 'BTC_BTCD', 'BTC_BTM', 'BTC_BTS', 'BTC_BURST', 'BTC_CLAM', 'BTC_DASH', 'BTC_DGB', 'BTC_DOGE', ..., 'BTC_OMG', 'ETH_OMG', 'BTC_GAS', 'ETH_GAS', 'BTC_STORJ'])

I'm going to manually take the info from the json for the Poloniex ticker, because I found no other way.

And also, an optimization for the ticker could be to accept a list of pairs to ask at the same time, because e.g. the Kraken API accepts a list of pairs: https://api.kraken.com/0/public/Ticker?pair=XBTUSD,XBTEUR I'd like to contribute to this project: would you accept a PR to accept multiple pairs with a single ticker?

Alberto

deepbrook commented 6 years ago

Hey @bebosudo , Thanks for contributing!

The formatting issue has been handled in the current dev branch - it introduces a formatter class which takes care of formatting a currency pair correctly depending on the exchange. Have a look at that! You can install it via pip install --pre bitex. Just be mindful that some things still have some hiccups.

Finding the current currency pairs available is exchange dependent - some offer a pair or symbols endpoint, others list them in the docs or don't list them at all. I've found that most exchanges offer at least one endpoint though that allows parsing of all pairs from its content. Again, the dev branch tries to fetch the latest list of pairs upon instantiation of each interface, so have a look at the _get_supported_pairs() method to get an idea.

As for the error you're seeing - that's a bug. The formatter appears to be outdated. I'll see if I can find time to supply a fix for this (or, if you like, you're welcome to submit a PR for that as well).

Regarding your last question - the general behavior is that a single pair is needed. Any deviations from this should be handled via a keyword argument (which is why all standardized methods support **kwargs), since it's exchange specific.

Since the number of exchanges supporting multiple tickers is relatively low, I wouldn't make changing the signature a priority - however, if you feel like this is something important, you could add *args to the signature, which would then support this behaviour. Just keep in mind to update the signature across all interfaces and the base interface class.

Cheers, Nils

bebosudo commented 6 years ago

Thanks for your response Nils.

I'm trying out the dev release, 2.0.0b4, and the first thing I noticed is that there are no more the Gemini and GDAX objects inside the main package (RockTradingLtd has been changed to TheRockTrading, while the REST interface is called RockTradingREST ... maybe adding 'The' in front of the REST interface could make things clearer?).

I tried the example in the README of the dev branch, and I submitted PR #124 which fixes the indentation in the example, and provides a more useful example, with a ticker on BitStamp.

While trying a similar pair on Poloniex, I found out something strange:

from bitex.pairs import PairFormatter

class BTCBCH(PairFormatter):
    def __init__(self):
        super(self.__class__, self).__init__(base='BTC', quote='BCH')

btcbch = BTCBCH()
>>> btcbch.format_for("Poloniex")
'BTC_BCH'

shouldn't it be 'BCH_BTC' ?

Moreover, Poloniex returns anyway the whole market tickers in its json response (don't think this can be considered a "bug" on bitex's side, anyway):

>>> from bitex import Poloniex
>>> resp = Poloniex().ticker(btcbch)
>>> pprint(resp.json())
{'BTC_AMP': {'baseVolume': '11.66534641',
             'high24hr': '0.00001906',
             'highestBid': '0.00001835',
             'id': 160,
             'isFrozen': '0',
             'last': '0.00001835',
             'low24hr': '0.00001700',
             'lowestAsk': '0.00001837',
             'percentChange': '-0.02028830',
             'quoteVolume': '639288.56424249'},
 'BTC_ARDR': {'baseVolume': '119.12828768',
              'high24hr': '0.00005422',
              'highestBid': '0.00004633',
              'id': 177,
              'isFrozen': '0',
              'last': '0.00004634',
              'low24hr': '0.00004546',
              'lowestAsk': '0.00004634',
              'percentChange': '-0.09509861',
              'quoteVolume': '2412654.39035808'},
...

I'm missing the "standard" and uniform formatted method, and I've seen issue #107. I would like to contribute to it (I'll write there).

Talking about the topic of this issue, when querying the supported pairs via bitex 2.0, I get strange results for Poloniex and Bitfinex.

>>> from bitex import Poloniex
>>> Poloniex().supported_pairs
~/.venvs/crypto-env/lib/python3.6/site-packages/bitex/api/base.py:54: IncompleteCredentialsWarning: Incomplete Credentials were given - authenti
cation may not work!
  IncompleteCredentialsWarning)
['BTC_ETH']

why only one BTC_ETH? may it be because I don't have authentication? but why only that specific pair?

And with Bitfinex:

>>> from bitex import Bitfinex
>>> Bitfinex().supported_pairs
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    Bitfinex().supported_pairs
  File "~/.venvs/crypto-env/lib/python3.6/site-packages/bitex/interface/bitfinex.py", line 45, in __init__
    super(Bitfinex, self).__init__('Bitfinex', BitfinexREST(**api_kwargs))
  File "~/.venvs/crypto-env/lib/python3.6/site-packages/bitex/interface/rest.py", line 18, in __init__
    super(RESTInterface, self).__init__(name=name, rest_api=rest_api)
  File "~/.venvs/crypto-env/lib/python3.6/site-packages/bitex/interface/base.py", line 26, in __init__
    self._supported_pairs = self._get_supported_pairs()
  File "~/.venvs/crypto-env/lib/python3.6/site-packages/bitex/interface/bitfinex.py", line 58, in _get_supported_pairs
    return self.symbols().json()
  File "~/.venvs/crypto-env/lib/python3.6/site-packages/requests/models.py", line 892, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Thanks again for your work, I really want to contribute a little: what do you suggest me to start working on?

deepbrook commented 6 years ago

While trying a similar pair on Poloniex, I found out something strange:


from bitex.pairs import PairFormatter

class BTCBCH(PairFormatter):
   def __init__(self):
       super(self.__class__, self).__init__(base='BTC', quote='BCH')

btcbch = BTCBCH()

>>> btcbch.format_for("Poloniex")
'BTC_BCH'

shouldn't it be 'BCH_BTC' ?

No, but I can tell why this confuses you. While every other exchange understood the principle of baseand quote currency, Poloniex elected to just not do that. So while the pairs in the BTC market are quoted in BTC and their base currency is the respective alt coin, Poloniex describes them the other way around- so instead of BaseCurrency_QuoteCurrency, they write it QuoteCurrency_BaseCurrency.

You can find this to be true while looking at the ticker endpoint (https://poloniex.com/public?command=returnTicker), where you'll find BitcoinCash quoted in BTC as BTC_BCH.

I tried the example in the README of the dev branch, and I submitted PR #124 which fixes the indentation in the example, and provides a more useful example, with a ticker on BitStamp.

The reason I used imaginary currencies was because the BTCUSD pair is already implemented with the library (you can import it with from bitex.pairs import BTCUSD). But I like your example, and I'll update the README with it.

Talking about the topic of this issue, when querying the supported pairs via bitex 2.0, I get strange results for Poloniex and Bitfinex.

If you take a look at the Poloniex Interface, you'll see I was simply lazy and hard-coded the return value - this needs fixing of course ;).

I'm trying out the dev release, 2.0.0b4, and the first thing I noticed is that there are no more the Gemini and GDAX objects inside the main package [...] You are correct. I haven't gotten around to re-implementing them, since 2.0 had a large portion of the of the old code base rewritten.

[...] (RockTradingLtd has been changed to TheRockTrading, while the REST interface is called RockTradingREST ... maybe adding 'The' in front of the REST interface could make things clearer?). I'd say it's not super confusing, but if you feel like this is something that needs fixing, submit a PR :)

Thanks again for your work, I really want to contribute a little: what do you suggest me to start working on? Cool! Glad to hear it. For starters, have a look at the code base - get a feeling for what each of the parts is doing. You're welcome to try and implement the changes I commented on in this post (Poloniex._get_supported_pairs(), for example) - just make sure to submit a single PR for each of them - that way it's easier to review for me and the merges are cleaner :)

If you're not sure how to do things exactly, feel free to open an issue, or submit your changes before completing them as PR - we can use the review function to discuss solutions and approaches.

Again, thanks for contributing!

bebosudo commented 6 years ago

... Poloniex describes them the other way around- so instead of BaseCurrency_QuoteCurrency, they write it QuoteCurrency_BaseCurrency.

I knew this, and I still have some doubts about the Poloniex pair formatter.

>>> from bitex.pairs import BCHBTC, BTCUSD
>>> BTCUSD.format_for("Poloniex")
'BTC_USD'
>>> BCHBTC.format_for("Poloniex")
'BTC_BCH'

As you said, shouldn't the formatter return the two values swapped? Or does it happen only when a pair is known to be traded on the platform? because e.g. on poloniex they don't trade USD, hence the pair doesn't get swapped?

The reason I used imaginary currencies was because the BTCUSD pair is already implemented with the library

Sorry, didn't find any reference to the fact that there were already some pairs. I created PR #127 in order to address that in the README to new users.

Talking about the topic of this issue, when querying the supported pairs via bitex 2.0, I get strange results for Poloniex and Bitfinex.

If you take a look at the Poloniex Interface, you'll see I was simply lazy and hard-coded the return value - this needs fixing of course ;).

Fixed in PR #127 .

You're welcome to try and implement the changes I commented on in this post (Poloniex._get_supported_pairs(), for example) - just make sure to submit a single PR for each of them - that way it's easier to review for me and the merges are cleaner :)

Ops, I've just seen this part, was hidden in the quote. Hope it's not a problem. Next time I'll try to keep things separate for each exchange object.

deepbrook commented 6 years ago

You don't have to submit a PR per exchange but per code change type - so for example, one PR if you change the signature for the ticker endpoint across all exchanges, one PR for updated docstrings, etc. Its important to keep commits and PRs as atomic as possible.

As for the pairs and poloniex formatter - that's why I said that you should familiarise yourself with the code base. As for the poloniex formatter specifically, have a look at its function in the pairs.py file.

You'll see it formats depending on input - if it's btc or xmr, for example, it formats the pairs differently.

bebosudo commented 6 years ago

You don't have to submit a PR per exchange but per code change type - so for example, one PR if you change the signature for the ticker endpoint across all exchanges, one PR for updated docstrings, etc. Its important to keep commits and PRs as atomic as possible.

Got it, thanks (I'm almost new to contributing to open source projects).

As for the pairs and poloniex formatter - that's why I said that you should familiarise yourself with the code base. As for the poloniex formatter specifically, have a look at its function in the pairs.py file.

Thanks, I thought there should have been some magic behind that behavior ;-)

I'm going to close this issue, because I think further comments should go in the discussion of PR #127 .