search

diadata-org / diadata

DIAdata.org platform
https://diadata.org
Other
232 stars 128 forks source link

CoinGecko data scraper #291

Closed tmlee closed 3 years ago

tmlee commented 4 years ago

CoinGecko provides market data via an API with the documentation here https://www.coingecko.com/api

I think it makes sense for these market data to be served on the DIA platform to be consumed by other smart contracts.

Looking at the readme here it seems like https://docs.diadata.org/documentation/tutorials/exchangescrapers might be useful to look at in order to get started for DIA.

kaythxbye commented 4 years ago

Please implement the scraping functionality in the foreign-scraper package. For the data types please refer to foreignTypes.go in the package internal/pkg/model. The scraper should be run by foreign.go in cmd/foreign-scrapers. Finally, the scraped data should be saved to influxdb. To get an idea how this process works, you can have a look at our already implemented exchange-scrapers.

gitcoinbot commented 4 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 266 years, 2 months from now. Please review their action plans below:

1) janus has been approved to start work.

I have go knowledge of string manipulation and I have done some work in Go. I would like to work on this challenge. 2) niktrix has applied to start work _(Funders only: approve worker | reject worker)_.

scrap data and dunp it to diadat types

Learn more on the Gitcoin Issue Details page.

gitcoinbot commented 4 years ago

@janus Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

janus commented 4 years ago

@gitcoinbot On it take it easy

janus commented 4 years ago

@kaythxbye Which of the APIs should I use here ? https://www.coingecko.com/en/api . However, I intend to experiment

kaythxbye commented 4 years ago

@janus You should be able to get most data points required by the struct here (https://github.com/diadata-org/diadata/blob/master/pkg/model/foreignTypes.go) from /coins/{id} in the Coingecko API.

janus commented 4 years ago

@kaythxbye 

type ForeignQuotation struct {
    Symbol             string
    Name               string
    Price              float64
    PriceYesterday     *float64
    VolumeYesterdayUSD *float64
    Source             string
    Time               time.Time
    ITIN               string
}

What is ITIN? And ForeignQuotation is it to be sent to the datastore or via the chanTrades(a replacement of Trade type)? I saw this internal/pkg/foreign-scrapers/CoingeckoScraper.go, how is it related to what I am coding?

janus commented 4 years ago

@kaythxbye Still waiting for your feedback

jppade commented 4 years ago

@janus Sorry for the delay. ITIN is the "International Token Identification Number" https://itsa.global/ It would be nice if you find a way to get it, but it's not mandatory. internal/pkg/foreign-scrapers/CoingeckoScraper.go is a suggestion on where to put the scraping logic. The ForeignQuotation struct can be sent directly to the datastore (influx) with appropriate tags. Also, it would be great if the main method could be called with a flag corresponding to the data source (in this case CoinGecko), as we plan to integrate more foreign quotation scrapers. You can make a similar construction as in our DeFi scrapers. In case of questions feel free to contact me on telegram. My user name over there is the same as here. I see that ETH lost value since we set up the bounty. You will be rewarded accordingly depending on the actual currency rate.

janus commented 4 years ago

@jppade I tried connecting with you via telegram but your setting would not allow me through. My handle is AkaMoFo.

gitcoinbot commented 4 years ago

@janus Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

gitcoinbot commented 3 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

@janus due to inactivity, we have escalated this issue to Gitcoin's moderation team. Let us know if you believe this has been done in error!

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

janus commented 3 years ago

@gitcoinbot I have pushed a WIP. This task would be submitted with 38 hours

janus commented 3 years ago

@jppade This is PR, https://github.com/diadata-org/diadata/pull/305

gitcoinbot commented 3 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work for 0.6 ETH (211.32 USD @ $352.2/ETH) has been submitted by:

gitcoinbot commented 2 years ago

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

The funding of 0.6 ETH (1190.6 USD @ $1984.34/ETH) attached to this issue has been approved & issued to @janus.