search

ccxt / ccxt

A JavaScript / TypeScript / Python / C# / PHP cryptocurrency trading API with support for more than 100 bitcoin/altcoin exchanges
https://docs.ccxt.com
MIT License
32.7k stars 7.5k forks source link

Upbit showing "None" for amount. #7235

Closed viti0sus1 closed 4 years ago

viti0sus1 commented 4 years ago

First and foremost, thank you for all that you've been doing. Please bear in mind I'm new to programming and github in general, so apologies for this mess of an "issue".

I'm currently trying to use freqtrade for Upbit, with the Korean Won Pairs. (KRW), during simulated runs everything was doing well so I tried live trading and got a fatal exception posted at the bottom of this issue. After asking one of the devs on the team he suggested I try the fetch_order so that we could see what happened in the process. After I posted the output, I was told that the "amount" shouldn't be None, and that I should post an issue here. Any help in this matter would be greatly appreciated.

import ccxt

exchange = ccxt.upbit({
    "apiKey": "hruieoahfdklhafuioehfuihdujhfdfdu",
    "secret": "hfdjfhdjfhdjfhdjfhdjfhdjfhdjfhdfd",
    "enableRateLimit": True,
})

exchange.verbose = True
if exchange.has['fetchOrder']:
    order = exchange.fetch_order('618ff809-f942-417f-8fa4-c4ea41410597')
    print(order)

Request: GET https://api.upbit.com/v1/order?uuid=618ff809-f942-417f-8fa4-c4ea41410597 {'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3Nfa2V5IjoiTVJmMjVSSVFTc1N0ZmtZeFg1eFp6VWdtdGRPZzhiTTgybnRscUY3YiIsIm5vbmNlIjoxNTk0MTM3NzMwOTQ2LCJxdWVyeV9oYXNoIjoiNjJhZTkwNmI3MjIxOTM1ZmM3ZDk3Y2ViODE3OTY4NThjMDNjZGFmMGQ5ZjYyZTAwMTM0ZGYzMmMzM2ViNjRhZWQ0ZjQ4OGJhYWE5NjNiODkzYjk4NTUwMDA2NjFlNmJjMDNmYWExYWVhNTNlNTNlNDgzMTNiMWU3NTNkNWFiMDgiLCJxdWVyeV9oYXNoX2FsZyI6IlNIQTUxMiJ9.2IQLsSjoJhGZXHoXnFI-4G1IdcKYs-2zC2d1IdXtgeA', 'User-Agent': 'python-requests/2.24.0', 'Accept-Encoding': 'gzip, deflate'} None

Response: GET https://api.upbit.com/v1/order?uuid=618ff809-f942-417f-8fa4-c4ea41410597 200 {'Date': 'Tue, 07 Jul 2020 16:02:10 GMT', 'Content-Type': 'application/json', 'Content-Length': '559', 'Connection': 'keep-alive', 'Remaining-Req': 'group=default; min=899; sec=29', 'Access-Control-Allow-Origin': '*', 'ETag': 'W/"df1af7e8c5932df7c2942b890bff9963"', 'Cache-Control': 'max-age=0, private, must-revalidate', 'X-Request-Id': '3e684802-7649-46a1-b809-44bc37a94799', 'X-Runtime': '0.026137', 'Vary': 'Origin', 'X-Kong-Upstream-Latency': '28', 'X-Kong-Proxy-Latency': '1', 'Via': 'kong/1.4.3'} {"uuid":"618ff809-f942-417f-8fa4-c4ea41410597","side":"bid","ord_type":"price","price":"1999.99","state":"cancel","market":"KRW-ETH","created_at":"2020-07-03T03:25:53+09:00","volume":null,"remaining_volume":null,"reserved_fee":"0.999995","remaining_fee":"0.000000671","paid_fee":"0.999994329","locked":"0.001342671","executed_volume":"0.00735831","trades_count":1,"trades":[{"market":"KRW-ETH","uuid":"e95ad0f2-2b03-4972-8e23-6610f04b397b","price":"271800.0","volume":"0.00735831","funds":"1999.988658","created_at":"2020-07-03T03:25:53+09:00","side":"bid"}]}
{'info': {'uuid': '618ff809-f942-417f-8fa4-c4ea41410597', 'side': 'bid', 'ord_type': 'price', 'price': '1999.99', 'state': 'cancel', 'market': 'KRW-ETH', 'created_at': '2020-07-03T03:25:53+09:00', 'volume': None, 'remaining_volume': None, 'reserved_fee': '0.999995', 'remaining_fee': '0.000000671', 'paid_fee': '0.999994329', 'locked': '0.001342671', 'executed_volume': '0.00735831', 'trades_count': 1, 'trades': [{'market': 'KRW-ETH', 'uuid': 'e95ad0f2-2b03-4972-8e23-6610f04b397b', 'price': '271800.0', 'volume': '0.00735831', 'funds': '1999.988658', 'created_at': '2020-07-03T03:25:53+09:00', 'side': 'bid'}]}, 'id': '618ff809-f942-417f-8fa4-c4ea41410597', 'clientOrderId': None, 'timestamp': 1593714353000, 'datetime': '2020-07-02T18:25:53.000Z', 'lastTradeTimestamp': 1593714353000, 'symbol': 'ETH/KRW', 'type': 'market', 'side': 'buy', 'price': None, 'cost': 1999.988658, 'average': 271800.0, 'amount': None, 'filled': 0.00735831, 'remaining': None, 'status': 'canceled', 'fee': {'currency': 'KRW', 'cost': 0.999994329}, 'trades': [{'id': 'e95ad0f2-2b03-4972-8e23-6610f04b397b', 'info': {'market': 'KRW-ETH', 'uuid': 'e95ad0f2-2b03-4972-8e23-6610f04b397b', 'price': '271800.0', 'volume': '0.00735831', 'funds': '1999.988658', 'created_at': '2020-07-03T03:25:53+09:00', 'side': 'bid'}, 'order': '618ff809-f942-417f-8fa4-c4ea41410597', 'timestamp': 1593714353000, 'datetime': '2020-07-02T18:25:53.000Z', 'symbol': 'ETH/KRW', 'type': 'limit', 'side': 'buy', 'takerOrMaker': None, 'price': 271800.0, 'amount': 0.00735831, 'cost': 1999.988658, 'fee': None}]}

2020-07-08 00:34:50,997 - freqtrade.resolvers.exchange_resolver - INFO - No Upbit specific subclass found. Using the generic class instead. 2020-07-08 00:34:50,997 - freqtrade.exchange.exchange - INFO - Using CCXT 1.30.48 2020-07-08 00:34:50,997 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'enableRateLimit': True} 2020-07-08 00:34:50,999 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'enableRateLimit': True, 'rateLimit': 1000} 2020-07-08 00:34:51,001 - freqtrade.exchange.exchange - INFO - Using Exchange "Upbit" 2020-07-08 00:34:52,125 - freqtrade.wallets - INFO - Wallets synced. 2020-07-08 00:34:52,129 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist VolumePairList from '/home/itiviti/freqtrade/freqtrade/pairlist/VolumePairList.py'... 2020-07-08 00:34:53,068 - VolumePairList - INFO - Searching 3 pairs: ['BSV/KRW', 'QKC/KRW', 'BTC/KRW'] 2020-07-08 00:34:53,068 - freqtrade.rpc.rpc_manager - INFO - Enabling rpc.telegram ... 2020-07-08 00:34:54,522 - freqtrade.rpc.telegram - INFO - rpc.telegram is listening for following commands: [['status'], ['profit'], ['balance'], ['start'], ['stop'], ['forcesell'], ['forcebuy'], ['performance'], ['daily'], ['count'], ['reload_config', 'reload_conf'], ['show_config', 'show_conf'], ['stopbuy'], ['whitelist'], ['blacklist'], ['edge'], ['help'], ['version']] 2020-07-08 00:34:54,587 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': status, 'status': 'running'} 2020-07-08 00:34:55,609 - freqtrade.worker - INFO - Changing state to: RUNNING 2020-07-08 00:34:55,609 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': custom, 'status': "Exchange: upbit\nStake per trade: 2000 KRW\nMinimum ROI: {'0': 0.12, '30': 0.1, '60': 0.05, '120': 0.025, '240': 0.0125, '480': 0.005, '960': 0.0025}\nStoploss: -0.25\nTimeframe: 15m\nStrategy: BR2"} 2020-07-08 00:34:56,077 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': status, 'status': "Searching for KRW pairs to buy and sell based on [{'VolumePairList': 'VolumePairList - top 3 volume pairs.'}]"} 2020-07-08 00:34:56,596 - freqtrade.persistence - INFO - Found open trade: Trade(id=1, pair=ETH/KRW, amount=0.00735835, open_rate=271800.00000000, open_since=2020-07-02 18:25:53) 2020-07-08 00:34:58,679 - freqtrade.data.converter - INFO - Missing data fillup for QKC/KRW: before: 199 - after: 213 2020-07-08 00:34:58,696 - freqtrade.data.converter - INFO - Missing data fillup for BSV/KRW: before: 199 - after: 200 2020-07-08 00:34:58,751 - freqtrade.freqtradebot - INFO - Found open order for Trade(id=1, pair=ETH/KRW, amount=0.00735835, open_rate=271800.00000000, open_since=2020-07-02 18:25:53) 2020-07-08 00:34:58,751 - freqtrade.commands.trade_commands - ERROR - must be real number, not NoneType 2020-07-08 00:34:58,751 - freqtrade.commands.trade_commands - ERROR - Fatal exception! Traceback (most recent call last): File "/home/itiviti/freqtrade/freqtrade/commands/trade_commands.py", line 20, in start_trading worker.run() File "/home/itiviti/freqtrade/freqtrade/worker.py", line 73, in run state = self._worker(old_state=state) File "/home/itiviti/freqtrade/freqtrade/worker.py", line 107, in _worker self._throttle(func=self._process_running, throttle_secs=self._throttle_secs) File "/home/itiviti/freqtrade/freqtrade/worker.py", line 128, in _throttle result = func(*args, **kwargs) File "/home/itiviti/freqtrade/freqtrade/worker.py", line 141, in _process_running self.freqtrade.process() File "/home/itiviti/freqtrade/freqtrade/freqtradebot.py", line 156, in process self.check_handle_timedout() File "/home/itiviti/freqtrade/freqtrade/freqtradebot.py", line 898, in check_handle_timedout fully_cancelled = self.update_trade_state(trade, order) File "/home/itiviti/freqtrade/freqtrade/freqtradebot.py", line 1212, in update_trade_state if not isclose(order['amount'], new_amount, abs_tol=constants.MATH_CLOSE_PREC): TypeError: must be real number, not NoneType 2020-07-08 00:34:58,754 - freqtrade.commands.trade_commands - INFO - worker found ... calling exit 2020-07-08 00:34:58,754 - freqtrade.rpc.rpc_manager - INFO - Sending rpc message: {'type': status, 'status': 'process died'} 2020-07-08 00:34:59,248 - freqtrade.freqtradebot - INFO - Cleaning up modules ... 2020-07-08 00:34:59,752 - freqtrade.freqtradebot - INFO - Buy order cancelled on exchange for Trade(id=1, pair=ETH/KRW, amount=0.00735835, open_rate=271800.00000000, open_since=2020-07-02 18:25:53). 2020-07-08 00:34:59,753 - freqtrade.freqtradebot - INFO - Found open order for Trade(id=1, pair=ETH/KRW, amount=0.00735831, open_rate=271800.00000000, open_since=2020-07-02 18:25:53) 2020-07-08 00:34:59,753 - freqtrade - ERROR - Fatal exception! Traceback (most recent call last): File "/home/itiviti/freqtrade/freqtrade/main.py", line 36, in main return_code = args'func' File "/home/itiviti/freqtrade/freqtrade/commands/trade_commands.py", line 29, in start_trading worker.exit() File "/home/itiviti/freqtrade/freqtrade/worker.py", line 179, in exit self.freqtrade.cleanup() File "/home/itiviti/freqtrade/freqtrade/freqtradebot.py", line 120, in cleanup self.cancel_all_open_orders() File "/home/itiviti/freqtrade/freqtrade/freqtradebot.py", line 932, in cancel_all_open_orders self.handle_cancel_buy(trade, order, constants.CANCEL_REASON['ALL_CANCELLED']) File "/home/itiviti/freqtrade/freqtrade/freqtradebot.py", line 973, in handle_cancel_buy self.update_trade_state(trade, corder, trade.amount) File "/home/itiviti/freqtrade/freqtrade/freqtradebot.py", line 1212, in update_trade_state if not isclose(order['amount'], new_amount, abs_tol=constants.MATH_CLOSE_PREC): TypeError: must be real number, not NoneType

kroitor commented 4 years ago

Hi!

Thank you for reporting it! It's not easy to explain, but I will try.

First, I've formatted your response output for readability, please take a look at this screenshot:

Screen-Shot-2020-07-07-at-19 41 18

↑ From this screenshot you can see how CCXT parses the Upbit's output.

CCXT takes the Upbit's output and reformats it according to the unified order structure described here:

After parsing, CCXT returns the unified order structure. It is highlighted with green color (the lower section of the screenshot). CCXT also keeps the original order response returned by Upbit in the info field of the unified order structure returned from CCXT – that is done for debugging and other technical purposes. You can see original order in the info field, that as the pink highlight on the above screenshot (the upper section of it).

When parsing the Upbit order, CCXT does the following conversions:

Those values have the following relationship: amount = filled + remaining ← this should be rather clear.

That usually works well for limit-type orders. However, in this case we're dealing with a market-type order.

Now, the problem is caused by the fact that for market orders, Upbit does not return neither executed_volume (filled) nor the remaining_volume (remaining). Both of these values are None. So, for market-type orders, Upbit will only return the filled amount of the order. Sometimes, we are also able to calculate the missing fields from other fields, but not in this case, apparently.

Unfortunately, we can't workaround this issue in CCXT, because we cannot invent those missing values from thin air. This is just how Upbit's market orders work. After placing a market order with Upbit, the original amount and the remaining amount for it become unknown, and Upbit will only return the actually-filled amount for it. This is actually a common issue across the exchanges with market orders – they just wouldn't return a complete set of info, that could be used to derive the missing fields.

However, this issue could be handled in freqtrade, at least, it could avoid crashing, when some of the market-order-info is missing from the exchange itself.

Let me know if that does not answer your question.

If you can paste a link to your freqtrade issue we could try to address it in collaboration with the authors of freqtrade. In this particular case, I would suggest freqtrade to add the code for the following:

  1. check if it's a market order
  2. check if the amount is not defined
  3. then use the filled value instead
viti0sus1 commented 4 years ago

Thank you for your thorough reply, I am grateful that you took the time to explain it in such detail. Unfortunately, I never filed an issue with the freqtrade team on their github, but rather raised the issue in their slack channel. So I can't paste a link to the issue, but in the future I'll do just that.

It's unfortunate that Upbit's market orders are setup in that fashion, as their maker and taker fees are the same whether you make limit buy/sell orders or market buy/sell orders using Korean Won which is dirt cheap compared to their BTC or USDT markets.

kroitor commented 4 years ago

@viti0sus1 ok, i'm closing this for now, feel free to reopen it or just ask further questions if any.