search

ianw1974 / zimbra-build-scripts

Zimbra Build Helper Script - makes it easier to build Zimbra
MIT License
64 stars 25 forks source link

wget binary from website #47

Closed magicevil83 closed 12 months ago

magicevil83 commented 12 months ago

Hello there @ianw1974 ,

thanks for your great help for us administrators all, very appreciated.

I'm trying to test compiled binaried directly from your main site, but download with wget produce a Forbidden error. Is that intended?

root@mail:~# wget https://cdn.techfiles.online/ubuntu/zcs-10.0.0_GA_0001.UBUNTU18_64.20231003091546.tgz --2023-10-20 18:26:03-- https://cdn.techfiles.online/ubuntu/zcs-10.0.0_GA_0001.UBUNTU18_64.20231003091546.tgz Resolving cdn.techfiles.online (cdn.techfiles.online)... 169.150.247.39 Connecting to cdn.techfiles.online (cdn.techfiles.online)|169.150.247.39|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2023-10-20 18:26:03 ERROR 403: Forbidden.

ianw1974 commented 12 months ago

It's explained on my website on the page in bold text that curl/wget is not possible from my site. I disabled this because of people abusing and scraping the website and overloading by downloading all builds and increasing my bandwidth usage.

If you wish to download on a server, the easiest is to install a console browser like elinks, links, w3m and visit the site with this. You then navigate the site to the page you wish to download from and it will download to the machine in a similar way than if you were to use curl/wget.

ianw1974 commented 12 months ago

Theoretically in the future, I could offer time-based download links to stop the abuse that I have been experiencing, but until then the best way is using the console browser as mentioned above and on the website just above the Zimbra download links.

magicevil83 commented 12 months ago

It's explained on my website on the page in bold text that curl/wget is not possible from my site. I disabled this because of people abusing and scraping the website and overloading by downloading all builds and increasing my bandwidth usage.

If you wish to download on a server, the easiest is to install a console browser like elinks, links, w3m and visit the site with this. You then navigate the site to the page you wish to download from and it will download to the machine in a similar way than if you were to use curl/wget.

My bad, didn't notice that. I see the cdn.* subfix right now , that is surely a DDoS service like Cloudflare or similar limiting abuse-clicks on it. In any case that is not really a big problem, you can even download anywhere else your build and SCP that to your server. You're already doin' a great job compiling binaries for those whom in troubles with that (every quarter even) so, can't ask nothing more than that. Thanks man.

ianw1974 commented 12 months ago

The cdn still charges me for storage and bandwidth used. Basically what was happening is people scrape the site, grab the urls and downloaded all the builds. A normal user would choose a single build and download, thus only incurring me 250MB approx of download bandwidth. Those scapers would be getting all builds at the same time, causing GB's of data and this was happening daily if not multiple times daily. Not enough to go over a limit to be counted as DOS/DDOS, but enough to cost me daily amounts that are unfair than compared to normal usage.

I'm fine with paying for fair usage by putting the builds up, I just hate people abusing things in such a way. It was either that, or not providing any builds whatsoever. I'm already paying for the VPS that hosts the site and the files, and the CDN protects that to stop overloading the origin server, as well as ensuring everyone downloads from their nearest servers for the content that is replicated across the world on the CDN network. So generally costs to me are:

  1. VPS.
  2. CDN bandwidth.
  3. CDN permanent storage across multiple locations worldwide to provide ultra-fast downloads.

Had I been a big corporate or company, then cost control wouldn't be too much of an issue I guess since it would come out of profits being made. I don't profit, I just provide a service for the community that unfortunately started to get abused by unfair people.