BlankerL / DXY-COVID-19-Crawler

2019新型冠状病毒疫情实时爬虫及API | COVID-19/2019-nCoV Realtime Infection Crawler and API
https://lab.isaaclin.cn/nCoV/
MIT License
1.99k stars 400 forks source link

服务器迁移计划 | Service Migration Plan #133

Closed BlankerL closed 2 years ago

BlankerL commented 3 years ago

各位小伙伴大家好,最近几个月由于本人时间有限没有及时维护本项目,导致API访问性降低,同时数据仓库由于服务器内存、带宽压力限制无法及时上传,在这里深表抱歉!

最近几天忙里偷闲,已经完成了数据库、数据仓库同步脚本的迁移,迁移后服务器性能翻倍、数据仓库的同步压力也得到了非常显著的缓解。

目前,数据仓库已经完成迁移,并且会从今天开始每天凌晨12点左右自动上传全量数据到数据仓库的Release当中。

近期,我会继续完成API部分的迁移,然而由于网站将从阿里云迁移到腾讯云服务器,过程中需要涉及:

  1. 域名从阿里云转出,并转入腾讯云(大约5-7个工作日);
  2. 在腾讯云重新完成域名及服务器备案(大约3-5个工作日);
  3. API在腾讯云中重新部署(很快)。

因此,可以预见的是API服务会有些许不稳定。

目前,API服务器仍部署在阿里云上,域名也仍然捆绑在阿里云上,然而API底层使用的数据库已经在腾讯云上,因此,API访问的延迟时间略有增加(需要把数据库的数据从腾讯云传输到阿里云,才能从API中返回数据,因此延迟时间主要在于跨机房的数据传输)。

同时,我已经开始申请域名迁移,域名迁移完成到备案的时间之内, 腾讯云和阿里云的服务器理论上来说均无法解析(因为没有备案),为了保证API的高可用性,我会暂时将API部署到海外服务器进行过渡(域名解析到境外服务器不需要备案),届时API的延迟会有非常明显的提升,望知悉。

域名和API正式迁移时,我会在本帖另做通知。


Hello buddies, due to my limited time in recent months, I did not maintain this project in time, resulting in reduced accessibility to the API, and the data warehouse could not be updated in time due to server memory and bandwidth pressure limitations. I am very sorry for the inconvenience.

In the last few days, I have completed the migration of the database and data warehouse synchronization scripts, which have doubled the server performance and relieved the synchronization pressure of the data warehouse.

The migration of the data warehouse has been completed and the full amount of data will be automatically uploaded to the Release of the data warehouse at around 12am every day starting today.

In the near future, I will continue to complete the API part of the migration, however, as the website will be migrated from Ali cloud to Tencent cloud servers, the process will involve the followings,

  1. Transfer of domain name from Ali cloud and to Tencent cloud (about 5-7 business days),
  2. Re-complete the domain name and server filing on Tencent Cloud's side (about 3-5 business days), and
  3. API redeployment in Tencent Cloud (easy!).

Therefore, it is predictable that the API service will be slightly unstable.

At present, the API server is still deployed on Ali cloud and the domain name is still bundled on Ali cloud, however, the database used in the underlying API is already on Tencent cloud, so the latency of API access has slightly increased (the data of the database needs to be transferred from Tencent cloud to Ali cloud to return data from API, so the latency mainly lies in the data transfer across service providers).

At the same time, I have started to apply for domain name migration, and within the time when the domain name migration is completed and filed, the servers in Tencent Cloud and Ali Cloud are theoretically unable to resolve (because there is no filing), so in order to ensure the high availability of API, I might temporarily deploy API to overseas servers for transition (domain name resolution to overseas servers does not require filing), and the latency of API will be very obviously improved then. Please be aware of this.

I will make another notice in this post when the domain name and API start officially migration.

BlankerL commented 3 years ago

备用腾讯云节点:http://81.68.90.103/