Implement one common set of currencies through all exchanges for easier merge data between them

[alexloiko]

There exists a gap between currency sets in different exchanges. Same coins have different symbols in different exchanges. Sometimes in code can be found parts of converters, but they are not full. It is impossible to identify is this coin same on not with some other from another exchange. E.g.

Bitfinex:

• IOT
• DSH
• YYW (coin YOYOW) etc.

Binance:

• IOTA
• DASH
• YOYO (coin YOYOW)

It will be valuable to have consistent API through all exchanges, converting data to common currency sets in one place. Or to have some util with mappings, (exchange definition) -> (common definition) and back. Also it must be optional I think. Thanks.

[timmolter]

This is definitely an issue that many people are dealing with, and a solution is needed. In general, we need to define an official XChange symbol set, and have an optional way to seamlessly do a conversion at the individual module level for all places where the exchange expects a different symbol than the official XChange symbol set.

[andre77]

actually a very good idea in my application i do have an optional configuration for every currency / exchange which must be "mapped", but it could be covered by XChange as well. the mapping is pretty static, so it could be in the json config or even hard coded, since there are not so many cases. one have to implement this aspect into all methods:

1. before every api call (convert currency parameter)
2. after the api call (convert the currencies in the result back)

[cymp]

There is some code like that in some modules, here for example with DASH and QTUM. I don't like this too much, I would prefer to have it configured into json config, and have a more general place in the code where these conversion are done given what is found in the local json.

[walec51]

this is a hard problem and simple aliases wont solve it because:

• exchanges add currencies all the time and if we can we read then from their API
• symbols have conflicts across exchanges - Bitcoin Cash has two symbols in use, BCH and BCC, but on some exchanges BCC was Bitconnect

[alexloiko]

@walec51 So that is because it have to be done :) To simplify the interactions with such "problem" coins like BCH. All new listed coins, of course, will not be at first time, so they can be used as is. With next releases they can be added to mappings if needed. I understand it is hard to maintain the list, but what the solution? Each app which is using xchange has own mappings, and refresh them in same mode I think. In our case refresh will be centralized and supported with community I hope.

[walec51]

some apps use native exchange codes other apps (like mine) have their mappings which are maintained in the database and often audited and corrected

static data in a library could give you some defaults but no bigger app will do library upgrades to add new mappings so whatevery you do this must be a not static solution

[walec51]

PS. I'm not saying it should not be done but be aware that this will be a complex feature and will probably require breaking backwards compatibility to be done right

[badgerwithagun]

See #2886 - with some discussion, an alternative is in the works, starting with Bitmex where this issue is felt most keenly due to the use of constantly-varying instrument names. There are two parts:

• Make XChange always use the exchange-native pair names. This avoids confusion. Generally, if you ask for LTC/H19 on Bitmex or DSH/BTC on Bitfinex, you expect this to work.
• Add a compatibility layer over the top of this that translates a common, understood set of symbols into what the exchange expects. If you need consistency of symbols between exchanges, use that.

So, for Bitmex:

// Returns XBT/H19 (at the moment)
CurrencyPair activeContract = ((BitmexExchange) exchange).determineActiveContract("BTC", "USD", QUARTERLY);
exchange.getMarketDataService().getTicker(activeContract);

The idea of all this is to give the user the choice - use native symbols or a "standard" set depending on your use case.

What would probably be a good idea is to elevate this to a generic API which can be supported by any exchange. JSON may work in many cases, as @cymp points out, but it certainly won't for Bitmex, so there needs to be a code route. It could probably fit quite neatly into the exchange metadata setup.

[gregoreficint]

// Returns XBT/H19 (at the moment) CurrencyPair activeContract = ((BitmexExchange) exchange).determineActiveContract("BTC", "USD", QUARTERLY); exchange.getMarketDataService().getTicker(activeContract);

The idea of all this is to give the user the choice - use native symbols or a "standard" set depending on your use case.

What would probably be a good idea is to elevate this to a generic API which can be supported by any exchange. JSON may work in many cases, as @cymp points out, but it certainly won't for Bitmex, so there needs to be a code route. It could probably fit quite neatly into the exchange metadata setup.

Could you @badgerwithagun elaborate a bit more on your concept of currency mapping? I don't understand why you need the contractTimeframe (e.g. QUARTERLY) in the determineActiveContract method? Do you assume that the symbol changes over time? If so, why would you not specify for which date and time you want to know how the symbol mapping looks like? So if you would like to know the currency pair for the standard symbol "BTC/USD" for the current time why would the call not look like:

CurrencyPair activeContract = ((BitmexExchange) exchange).determineActiveContract("BTC", "USD", new Date().getTime()); //This would get the Bitmex specific currency pair "BTC/USD" for the current time

Thank you

[badgerwithagun]

@gregoreficint - Bitmex has more than one contract for a currency pair at any given time. XBTH19 is only one of the BTC/USD tracking contracts available right now - the quarterly one. I can't remember which other BTC contracts there are. Possibly a monthly and an annual. The point is that it's not enough to say 'BTC/USD' and a date; you also need to know the timeframe of the contract.

That said, it might potentially be useful to know what, for example, the quarterly BTC contract was on a particular date, I guess.

[gregoreficint]

Thank you for your fast clarification @badgerwithagun

I also work on the issue of a common set and hopefully I will come up with an answer.

I agree with you that XChange would always use exchange native pair names and the compatibility layer is on top. This would make very clear where the “normalisation” is done.

However, I fear that this rule would break backward compatibility since sometimes exchange implementations do some “normalisation” and sometimes they don’t.

_{Sent with GitHawk}

[badgerwithagun]

@timmolter has seemed happy to break compatibility to bring things into consistency. We broke Bitmex compatibility to do this.

From what I understand, these inconsistencies are one of the biggest problems XChange has right now.

[gregoreficint]

@badgerwithagun @timmolter I believe the major problem is that all currencies have been defined in the Currency and CurrencyPairs classes as "static" and are used in a static way in nearly all individual exchange implementations. For example, "Currency.BTC" is statically used 172 times in all the different exchange projects. However, "Currency.BTC" and all other "Currency.Coin" static usages bind these implementation in a hard-coded sense to the static definitions in the Currency class. This causes two major issues: 1) You cannot define the reference symbols externally using something like a JSON file at runtime since you need to remove the static property. Especially for production system you don't want to upgrade a library (where symbols are defined in the code) if you want to update symbols only. 2) The existence of the current static currency symbols is convenient. However, by using directly these reference symbols (with sometimes a mapper) rather than the native exchange symbols fosters inconsistent implementations. The clear separation of using native symbols only in the exchange implementations and optionally a mapping between reference symbols and the native symbols is not enforced by the architecture.

To solve this I see the following solution:

1. All the static currencies (and currency pairs) are removed from the Class Currency and CurrencyPairs.
2. Now every exchange implementation is forced to use the native symbol since the former reference symbols don't exist any more. Only currency objects rather than static class references should be used. For example, one would write new Currency("BTC") at Binance but new Currency ("XBT") at Kraken. All mappers/adapters that changed the former reference to native symbols and vice versa need to be removed.
3. A new class is derived from the Currency class that provides the reference symbols for the coins (maybe called CurrencyRef). These reference symbols are loaded via a JSON file.
4. An abstract mapping class for currencies would be defined that can or cannot be implemented by the exchanges. This class would use an exchange specific JSON file that maps the native symbol from the exchange to the master symbols and vice versa.
5. Additionally one might add a mapping class for currency pairs from native symbols to reference symbols and vice versa. This class would use the definitions from 4 and would compute the currency pairs automatically.

With "breaking compatibility" in my previous post I was referring to step 1 and 2. This looks like a major change with a lot of work that has to be done at all exchanges at once. I am not sure if this is even feasible from a capacity standpoint.

Do I think to complex about this issue?

Currently, I use 2170 master symbols in a database with a mapping from these symbols to the native symbols for Binance, CoinBasePrime and Kraken. So curating the data of master symbols and mappings seem possible.