Chinese mirror of the package registry

chubbyginger commented 2 years ago

What kind of issue is this?

[x] Feature Request. Start by telling us what problem you’re trying to solve. Often a solution already exists! Don’t send pull requests to implement new features without first getting our support. Sometimes we leave features out on purpose to keep the project small.

Configuration

Operating system: Windows 10 Home 21H2 64-bit

PlatformIO Version (platformio --version): PlatformIO Core, version 6.0.2

Description of problem

I am a PlatformIO user from China. It is all known to devs in China that there is a You-Know-What system that will cause accessing github.com becoming very, very, very slow or even unreachable (ERR_CONNECTION_RESET).

When I create a PlatformIO project, it will download a lot of data, including toolchains, SDKs, etc from github or some places like that. Thus the process is extremely slow and it takes hours to install that. In many situations there is completely no network activity.

internet

I think it is appropriate for PlatformIO developers to add a import function which can import pre-downloaded toolchains, SDKs, etc (NOT the ones downloaded from the "Platform" page. Seems that it is only a project framework.). There should also be a place to manually download the files, so that I can call my friends to help me download them.

Steps to Reproduce

Create a new project.

Actual Results

The project wizard stuck there showing

Please wait...
The first initialization requires Internet connection and may take a while (need to install dependent toolchains, frameworks, SDKs).
Please be patient and let the initialization complete.

Expected Results

Project can be created successfully

ps. My board: Espressif ESP32 Devkit

424778940z commented 1 year ago

If I don't use vpn, I won't be able to download

This is totally normal then lol, you have to pay for "国际精品网" to get the privilege to access CN2 if you are using China Telecom "国际精品网" plan may not available in city other than Shanghai, my best suggestion is switch to China Unicom, or just access through VPN. The SSL error have nothing todo with pio, try socks5 proxy instead of http

PS: well, talk to something like Tsinghua University mirror team see if they welling to help would be the best solution as I mentioned earlier, but that depends on @ivankravets 's opinion

captainstdin commented 1 year ago

If I don't use vpn, I won't be able to download

This is totally normal then lol, you have to pay for "国际精品网" to get the privilege to access CN2 if you are using China Telecom

It's not important. I want to solve my problem.

Just download the link https://dl.registry.ns2.platformio.org/tools/34/0c/1bb3ce94604147dd87b7cb891a39637da5624ed991fe6fb9477b373fab2c/toolchain-gccarmnoneeabi-windows_x86-1.50201.0.tar.g

and unzip it to the directory C:\Users\admin\.platformio\packages\ ?

C:\Users\admin\.platformio\packages\toolchain-xtensa but ,it still doesn't seem to work

captainstdin commented 1 year ago

PS: well, talk to something like Tsinghua University mirror team see if they welling to help would be the best solution as I mentioned earlier, but that depends on @ivankravets 's opinion

and alibaba mirrors.aliyun.com ? I often use this。、

try socks5 proxy instead of http

The socket5 protocol will be blocked for seconds (GFW)

424778940z commented 1 year ago

It's not important. I want to solve my problem.

"try socks5 proxy instead of http"

424778940z commented 1 year ago

The socket5 protocol will be blocked for seconds (GFW)

that do not make any sense, the socks5 protocol is between your ide/wget/python and your local instance of v2ray, not crossing GFW

captainstdin commented 1 year ago

that do not make any sense, the socks5 protocol is between your ide/wget/python and your local instance of v2ray, not crossing GFW

thinks, I run the v2ray client, open the LAN proxy server: 10809, and then set HTTPS_PROXY and HTTP_PROXY environment variable in windows , which can be downloaded pio package through the agent

ivankravets commented 1 year ago

We have recently added a new mirror in Asia with a direct link to China. Could you confirm that this link does not work WITHOUT any proxies?

Chinese users in this thread, could you test on your side this link WITHOUT any proxies? Do we have a problem again?

424778940z commented 1 year ago

Chinese users in this thread, could you test on your side this link WITHOUT any proxies. Do we have problem again?

works for me without proxy, around 4M his issue is likely due to China Telecom lacking CN2 access, that's why access to all website outside of China sucks, not pio specific

captainstdin commented 1 year ago

We have recently added a new mirror in Asia with a direct link to China. Could you confirm that this link does not work WITHOUT any proxies?

Chinese users in this thread, could you test on your side this link WITHOUT any proxies. Do we have problem again?

cannot download, By the way ，I use China Telecom, the best operator in East China，If I can't download, others (especially China Mobile operator )can't event

Gianthard-cyh commented 1 year ago

We have recently added a new mirror in Asia with a direct link to China. Could you confirm that this link does not work WITHOUT any proxies?

Chinese users in this thread, could you test on your side this link WITHOUT any proxies. Do we have problem again?

China Mobile about 2MB/s without proxy.

424778940z commented 1 year ago

the best operator in East China

bruh, that is true if you only access CHINESE website

captainstdin commented 1 year ago

the best operator in East China

bruh, that is true if you only access CHINESE website

hhhh, Maybe we just play games 王者荣耀，

我不知道怎么购买CN2服务，现在是家庭299 ￥/MO 套餐，这都不行吗？> <....

ivankravets commented 1 year ago

The LayerStack provides services with the CN2 route. The feedback is very bad about this company. We still have not received access to try their services. It looks like a fraudulent Chinese company. No trial, no refund, and the minimum order is 12 months. People who bought their services claim that REAL speed is very bad, it is so far from the speed which you receive for their test link.

Anyway, @captainstdin, @424778940z, @Gianthard-cyh what is your download speed for https://hk03.layerstack.com/100MB.test?

424778940z commented 1 year ago

The LayerStack provides services with the CN2 route.

hummm, the problem is not the server do not have CN2 route, is the China Telecom have very limited bandwidth to CN2.

Anyway, @captainstdin, @424778940z, @Gianthard-cyh what is your download speed for https://hk03.layerstack.com/100MB.test?

I got about 1.5M without proxy

Gianthard-cyh commented 1 year ago

1.1MB/s

ivankravets commented 1 year ago

@424778940z, you have a better understanding of the issue. Could we resolve it from our side? We tried to buy servers in China but our requests were rejected because we "don't have business in China".

I still personally don't understand how people live in China and run international businesses.

424778940z commented 1 year ago

@424778940z, you have a better understanding of the issue. Could we resolve it from our side? We tried to buy servers in China but our requests were rejected because we "don't have business in China".

I would suggest talk to a mirror site that offers mirror for Linux distros, see if they are interested to help.

There is an international version of Aliyun, try Singapore server (edited: HK server are going around actually) Or Azure's Singapore server

I still personally don't understand how people live in China and run international businesses.

for website, English site usually hosted at server outside China for speed, or on Chinese server with CDN optimized for international access

ivankravets commented 1 year ago

We've just created a demo instance based on LayerStack. Sorry, that disturb you again, Chinese people please re-test WITHOUT proxy this new link named LSTEST. What is your download speed? If it is good, we will mirror packages here too.

@captainstdin does LSTEST link above work fast for you?

424778940z commented 1 year ago

We've just created a demo instance based on LayerStack. Sorry, that disturb you again, Chinese people please re-test WITHOUT proxy this new link named LSTEST. What is your download speed? If it is good, we will mirror packages here too.

I believe you mean http://122.128.106.144/toolchain-gccarmnoneeabi-windows_x86-1.50201.0.tar.gz I got 3M without proxy

424778940z commented 1 year ago

@captainstdin does LSTEST link above work fast for you?

my friend using China Telecom tested the speed jump between 700K and 2M

ivankravets commented 1 year ago

That's great! We will add an additional mirror in a few hours. Thanks again for reporting any issues. We built our own CDN network across continents and providing superior download speed for our users is No. 1 priority for us.

captainstdin commented 1 year ago

Anyway, @captainstdin, @424778940z, @Gianthard-cyh what is your download speed for https://hk03.layerstack.com/100MB.test?

About 4M/s without vpn , china tel

I still personally don't understand how people live in China and run international businesses.

Each country has its own characteristics. China's population base is too large. If strict control is not adopted, more fraud and other illegal acts will occur.

If you want to use a server in Chinese Mainland, you must be authenticated by real name.However, the service provider was unable to verify the identity of foreigners,

Cloud Service providers verify their identities through government cooperation channels, which are mandatory requirements of the government.

As a citizen living on the mainland, I recommend tencent cloud qcloud.com or alibaba cloud aliyun.com

424778940z commented 1 year ago

Each country has its own characteristics. China's population base is too large. If strict control is not adopted, more fraud and other illegal acts will occur.

I understand you may not mean to be rude, but personally your wording sounds a bit rude. Why GFW exists is a topic for another day, and the main reason with your issue have nothing todo with GFW As Ivan stated, he could not buy China based cloud service because he do meet the requirement: have business in China And again, the root of your issue is China Telecom do have very limited access to CN2, which is a way to reach from inside China to global network, it has nothing todo with GFW also.

ivankravets commented 1 year ago

@captainstdin could you try this link http://122.128.106.144/toolchain-gccarmnoneeabi-windows_x86-1.50201.0.tar.gz?

captainstdin commented 1 year ago

I understand you may not mean to be rude, but personally your wording sounds a bit rude

It may be a problem with translation tools.

I thought he was asking why he couldn't use servers in mainland China

i am sorry

@captainstdin could you try this link http://122.128.106.144/toolchain-gccarmnoneeabi-windows_x86-1.50201.0.tar.gz?

can not connect server A07B5400-6A88-48AF-A536-DBB0F4C9E5CD

ivankravets commented 1 year ago

Sorry, could you try it again?

captainstdin commented 1 year ago

Sorry, could you try it again?

maybe。

And again, the root of your issue is China Telecom do have very limited access to CN2, which is a way to reach from inside China to global network,

ivankravets commented 1 year ago

@captainstdin what is your download speed WITHOUT any proxies for http://122.128.106.144/toolchain-gccarmnoneeabi-windows_x86-1.50201.0.tar.gz ?

13406169210 commented 1 year ago

@captainstdin what is your download speed WITHOUT any proxies for http://122.128.106.144/toolchain-gccarmnoneeabi-windows_x86-1.50201.0.tar.gz ?

5.2 M/S

But it's been stuck in this place for a long time

424778940z commented 1 year ago

But it's been stuck in this place for a long time

assuming you are using V2rayN with default settings try set following ENV var I'm using powershell here

$global:http_proxy = "socks5://127.0.0.1:10808"
$global:https_proxy = "socks5://127.0.0.1:10808"
$global:HTTP_PROXY = "socks5://127.0.0.1:10808"
$global:HTTPS_PROXY = "socks5://127.0.0.1:10808"

Gianthard-cyh commented 1 year ago

But it's been stuck in this place for a long time

assuming you are using V2rayN with default settings try set following ENV var I'm using powershell here
$global:http_proxy = "socks5://127.0.0.1:10808"
$global:https_proxy = "socks5://127.0.0.1:10808"
$global:HTTP_PROXY = "socks5://127.0.0.1:10808"
$global:HTTPS_PROXY = "socks5://127.0.0.1:10808"

so you are using a proxy or something?

ivankravets commented 1 year ago

But it's been stuck in this place for a long time

We have not mirrored all packages yet to the new node. We will inform here.

424778940z commented 1 year ago

so you are using a proxy or something?

well when I'm testing here, no but if I'm in vscode, yes, may not needed, but I'm too lazy to swap around

13406169210 commented 1 year ago

But it's been stuck in this place for a long time

We have not mirrored all packages yet to the new node. We will inform here.

Is there any mirror address we can download manually?

13406169210 commented 1 year ago

I'm not using a proxy

captainstdin commented 1 year ago

@captainstdin在没有http://122.128.106.144/toolchain-gccarmnoneeabi-windows_x86-1.50201.0.tar.gz的任何代理的情况下，您的下载速度是多少？

about 2m/s

13406169210 commented 1 year ago

But it's been stuck in this place for a long time

assuming you are using V2rayN with default settings try set following ENV var I'm using powershell here
$global:http_proxy = "socks5://127.0.0.1:10808"
$global:https_proxy = "socks5://127.0.0.1:10808"
$global:HTTP_PROXY = "socks5://127.0.0.1:10808"
$global:HTTPS_PROXY = "socks5://127.0.0.1:10808"

使用代理后无线报错什么原因啊

424778940z commented 1 year ago

使用代理后无线报错什么原因啊

try use http proxy instead then see if there any difference

$global:http_proxy = "http://127.0.0.1:10809"
$global:https_proxy = "http://127.0.0.1:10809"
$global:HTTP_PROXY = "http://127.0.0.1:10809"
$global:HTTPS_PROXY = "http://127.0.0.1:10809"

13406169210 commented 1 year ago

Now do not use the agent download speed is very fast, hope to maintain this speed, thank you for your efforts

ivankravets commented 1 year ago

@13406169210 , thanks for the feedback. Yes, let us see how the new mirror will perform. Do you use a proxy or without it? what is your download speed? This is a new link from the Chinese mirror https://dl.registry.ns3.platformio.org/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz

13406169210 commented 1 year ago

@13406169210 , thanks for the feedback. Yes, let us see how the new mirror will perform. Do you use a proxy or without it? what is your download speed? This is a new link from the Chinese mirror https://dl.registry.ns3.platformio.org/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz

No agent is used, download as normal The download speed is between 150k/S and 2M/S, unstable

bubulgum commented 1 year ago

Hi, all. Where to setup mirrors for platformio? My speed in Guangdong awful with/without proxy... And which mirror better to use?

ivankravets commented 1 year ago

@bubulgum , what is your download speed for https://dl.registry.ns3.platformio.org/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz ?

bubulgum commented 1 year ago

@bubulgum , what is your download speed for https://dl.registry.ns3.platformio.org/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz ?

@ivankravets without proxy ~5.6Mb, with Singapore proxy ~2.5Mb

kojak-xyz commented 1 year ago

@bubulgum , what is your download speed for https://dl.registry.ns3.platformio.org/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz ?

@ivankravets without proxy ~1.8MB/s, I'm using China Telecom

bubulgum commented 1 year ago

today become 600kb~1.5Mb

ivankravets commented 1 year ago

We don't have any limitations on the speed of our Chinese mirrors. The server provider claims about 300MBps for Chinese users.

@bubulgum , do you still experience slow download speed WITHOUT a proxy?

cainiaoxiao commented 1 year ago

@ivankravets I'm in China without a proxy meet a problem.When i manual install packages use CLI, platformio's cdn server dl.registry.ns3.platformio.org not available at this time,and I try other cdn's download link at you mentioned here：

Could you provide a download speed for each mirror specified below according to their number?

https://eu2.contabostorage.com/28afe64ba7f44add9b788964b0b1907a:pioregmirror/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz

https://dl.registry.nm1.platformio.org/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz

https://github.com/ivankravets/pioreg-ci-mirror/releases/download/framework-espidf/08c7751-framework-espidf-3.40401.0.tar.gz

https://dl.registry.ns1.platformio.org/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz

This information will help us to improve our CDN. We had a plan to mirror our packages to the China region but we could not do this because most cloud providers require us to be a citizen of China. If you have ideas on how to help us here, please email me@ikravets.com.

I test link 4 and have a nice speed, about 4MB/s:

But like you say at #3835 :

We changed the priority for China on our side. So, please run pio system prune -f and try building project.

Does it download fast automatically now? and the terminal display:

Looking for another mirror...

I think maybe ther platformio shoule switch cdn form dl.registry.ns3.platformio.org to dl.registry.ns1.platformio.org but infact not:

So could I manual change platformio cdn server? Or why not allow user manual download packages? I very confusion why user cannot manual download packages.....

Thanks a lot!

ivankravets commented 1 year ago

@cainiaoxiao , could you try manually downloading this file dl.registry.ns3.platformio.org ?

cainiaoxiao commented 1 year ago

@ivankravets now not , I use you provide test link but only replace server name to dl.registry.ns3.platformio.org like this :

https://dl.registry.ns3.platformio.org/tools/08/c7/7511ca5c84f8e747e04d6d0021c98d0c8ffef7c588782b0aec5b92ff5691/framework-espidf-3.40401.0.tar.gz I can't download...... like this:

platformio / platformio-core