search

diadata-org / diadata

DIAdata.org platform
https://diadata.org
Other
233 stars 129 forks source link

Scraper for Houbi Future contracts #179

Closed kaythxbye closed 4 years ago

kaythxbye commented 5 years ago
  1. Define your data need specifications within this github issue template
  2. Fund the ticket on https://gitcoin.co
  3. Approve or reject work submitted

We are looking for a Go developer that can provide scrapers of the Websocket for future contracts.

We reliably want to get trades in realtime from Huobi exchange. This trade information is then written into our databases using a specific format. We have an existing framework for this which allows us to parametrize the scraper. We work with Go and expect that the scrapers will work concurrently and efficiently. The URL of the API can be found here: https://github.com/huobiapi/API_Docs_en/wiki/WS_api_reference_en

The submitted work has to include the API scraper as well as a pull request with the respective documentation of the added code.

The DIA documentation on how to contribute can be found here: https://github.com/diadata-org/diadata

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

This issue now has a funding of 1.0 ETH (190.91 USD @ $190.91/ETH) attached to it as part of the diadata-org fund.

gitcoinbot commented 5 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 2 weeks, 3 days from now. Please review their action plans below:

1) xiaods has applied to start work _(Funders only: approve worker | reject worker)_.

i am blockchain developer, i am familiar with huobi future api. 2) nazariyv has been approved to start work.

I was hoping I can actually hack this out this night. I have 3 years' coding experience as a quant and have my own research / systematic trading company called Traider, where I have built the scrapers for exchanges like BitMex, Coinbase, Binance, Huobi and more through a unified API and used these on AWS and saved to S3.

Learn more on the Gitcoin Issue Details page.

nazariyv commented 5 years ago

I have the code to run the scraper on any single Huobi futures market and restart the WS client for when it goes down.

It is not saving this data anywhere at the moment. You have mentioned that you have the capabilities to save this in the right format to your database, so I presume writing this data to a file is OK with you?

I will add the capability to subscribe to multiple markets at once, and then should be good to go.

nazariyv commented 5 years ago

I have the concurrent scraping of all the possible futures contacts on Huobi and writing to files. Need to handle a couple of edge cases now.

nazariyv commented 5 years ago

Will submit the work tomorrow. Actually, today...

gitcoinbot commented 4 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work for 1.0 ETH (186.24 USD @ $186.24/ETH) has been submitted by:

  1. @nazariyv

@kaythxbye please take a look at the submitted work:

gitcoinbot commented 4 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

The funding of 1.0 ETH (176.01 USD @ $176.01/ETH) attached to this issue has been approved & issued to @nazariyv.

nazariyv commented 4 years ago

the latest work (as of 12/02/2019) is on PR: #189

nazariyv commented 4 years ago

latest work on: #190