search

BlankerL / DXY-COVID-19-Crawler

2019新型冠状病毒疫情实时爬虫及API | COVID-19/2019-nCoV Realtime Infection Crawler and API
https://lab.isaaclin.cn/nCoV/
MIT License
1.99k stars 400 forks source link

[Feature Request] Paging to overal api call #69

Closed ReginFell closed 4 years ago

ReginFell commented 4 years ago

Hi, are you considering adding pagination to overall method? There are two options at the moment: recent or overal Pulling overal takes around 2 minutes in Europe, I believe that it will be even worse in the future.

Adding paging or puling result by some time frame will definitely help a lot

BlankerL commented 4 years ago

Hello, there are about 1800+ documents in overall API, and 6600+ documents in area API, and it is increasing from time to time, so it will take some time if you query latest=0.

If you are going to use this data frequently, I recommend you pull the data every hour (or any frequency you prefer) and save the file locally.

In fact, I also offered a Data Warehouse for you to download the static data. This data warehouse is hosted on GitHub so I suppose the opening speed will be quite decent.

If you have a demand to load all the data at once in JSON format, I can also push the overall data into the data warehouse as you wish. If you could tell me what you intend to do with this API, I might be able to help you better.