search

regolith-linux / regolith-desktop

Meta package for the Regolith Desktop Environment
1.5k stars 31 forks source link

Add CDN options for downloading Regolith ISO #687

Open diegocn opened 2 years ago

diegocn commented 2 years ago

Hello!

Downloading Regolith 2.1 ISO from the official link described on the website is taking a long time.

My suggestion is to add some sort of mirroring/CDN options to download Regolith's ISOs. Some options are:

I'd love helping with this matter. Thanks in advance.

ferdinandyb commented 2 years ago

Wouldn't also having a torrent be a good idea?

diegocn commented 2 years ago

@ferdinandyb it is a good idea too. I think we can have all these options and let the users choose :D But for torrents, I'd say I do not know how can we do that, and where can we publish the torrent files themselves.

ferdinandyb commented 2 years ago

I checked, it doesn't seem too complicated: https://techspree.net/how-to-create-torrent-files/

ferdinandyb commented 2 years ago

I'll maybe go ahead and try to do it later this week.

diegocn commented 2 years ago

Great! What about the CDNs? Could I reach them on behalf of the project or does it need to be some of the current members?

ferdinandyb commented 2 years ago

I'll leave answering that up to @kgilmer :)

kgilmer commented 2 years ago

Hi @diegocn , first, thank you for your suggestions on CDN hosts! This is not an area I know much about. As the project is volunteer and relies on donations to support itself, I've been hesitant to take on too much costs for things like hosting. This is why I choose to use GitHub to distribute ISOs for example: it's free.

If you are able to provide some help to get us able to distribute the ISO in such a way that it's faster for users, we'd love that. I have worked with a few hosts that offered free services to open source projects in the past but their infrastructure requirements didn't match up with our modest build system. I need the ability to scp or rsync from a GitHub workflow. If this sounds like something you'd be able to help out with, that would be great!

Regarding the torrenting idea, I spent an afternoon trying to figure that one out as well but was unable to get it working in the few hours I gave to the task so gave up. If there are known specific instructions that cover the entire process end to end I may take another stab at it.

diegocn commented 2 years ago

Sure! I can provide help to make it happen both by contacting providers and also with the technical setup :D

Can I use this Issue to contact CDN providers on behalf of the project?

diegocn commented 2 years ago

There is also SourceForge

TanZng commented 2 years ago

If you want to reach Digital Ocean for the Open Source Sponsorships here is the how-to guide https://do.co/oss-sponsorship

diegocn commented 2 years ago

It's a starting point -> https://sourceforge.net/projects/regolith-desktop/

diegocn commented 2 years ago

@kgilmer would you like me to add you as admin for this SourceForge project?

BTW, they support SCP/rsync. I uploaded the current 2.1 ISO there using rsync.

diegocn commented 2 years ago

Hey @TanZng!

Thanks for pointing out directions to talk to Digital Ocean. As per their rules, the project would fall on Staging Tier. I don't think $250/year would be enough for Object Storage costs related to space + bandwidth, but, it's just a guess.

@kgilmer do we have some statistics related to the amount of ISO downloads?

kgilmer commented 2 years ago

Hi @diegocn , currently we're averaging 50 - 80 ISO downloads a day. Regarding the SourceForge host, that's interesting. Very early on in the Regolith project I had used SF for all file hosting. However I found the ad treatment used on download pages to be sometimes misleading or "malwar-ish" and so found that direct links from GitHub were better. Personally, I would like to avoid any sort of service that uses obnoxious or misleading (fake "download" buttons that link to ad sponsor, etc) ad placement as part of the download workflow. Ideally all sources could be downloaded as direct links from regolith-desktop.com. (I realize bandwidth isn't free and it's entirely reasonable for companies to cover their costs via ads but as we can use GitHub without this restriction we have the luxury of a higher standard.)

I wonder if the Digital Ocean FOSS thing would also require adding advertisements?

diegocn commented 2 years ago

Hey @kgilmer!

I couldn't agree more with your points about SF ads. I'd say I don't like it too and we might not be able to provide direct links from regolith-desktop.com. Let's put SF as a 'backup plan'.

Digital Ocean Sponsorship would require some actions from the sponsored projects, like the ones below. image

BTW, I've been playing with Cloudflare's Object Storage (R2). It's very interesting in terms of a free tier. image Ref.: https://developers.cloudflare.com/r2/platform/pricing/

But, it's kinda tricky to make the objects publicly accessible. It needs a 'binding' between R2 and Workers (which is their 'Serverless' platform). R2 is S3-compatible so any S3 cli/sdk is expected to work for uploads, for example. I managed to upload the full regolith ISO using rclone and it worked very well managing upload chunks to not reach Workers limits.

Workers also have a reasonable free tier. image

Ref.: https://developers.cloudflare.com/workers/platform/pricing/

Another trick from this approach is: in order to 'enable' R2 usage within Cloudflare is required to set up a payment method, even for free tier usage. The same does not apply to Workers, one can use it without setting up a payment method.

I did it for testing purposes and the result is here -> https://test1.diegocn.workers.dev/regolith-ubuntu-jammy-2.1.zip (I'll not keep it up for a long time). The download speed reaches my connection limits, so it's a very good result so far in terms of CDN 'for free'.

Next Steps

I'll try to reach Fastly folks regarding their OSS Sponsorship in the coming days. Also, will try to reach some of my internal Cloudflare contacts and see if they could provide us R2 + Workers sponsorship.

diegocn commented 2 years ago

Would Google Drive be an alternative? WDYT @kgilmer? I saw it as an option for deepin ISO download.

kgilmer commented 2 years ago

Hi @diegocn , hmm personally I don't feel like I would want to deal w/ the Digital Ocean requirements, in particular the recurring overhead. I also tried the link you posted and was getting up to 27MB/s which is pretty fast! Out of curiosity, what speeds are you seeing with the github hosted link?

Is there some reason for the workers? Ideally I can just manage an SSH secret push, w/ scp/rsync and be done with it. I jumped through a bunch of hoops to work around Netlify's sync system but threw it all away because it became difficult to support.

IRT google drive, unsure. Are you saying that option has no dollar costs?

kgilmer commented 1 year ago

By all means, and thank you!

On Thu, Aug 4, 2022, 8:13 AM Diego Noguês @.***> wrote:

Sure! I can provide help to make it happen both by contacting providers and also with the technical setup :D

Can I use this Issue to contact CDN providers on behalf of the project?

— Reply to this email directly, view it on GitHub https://github.com/regolith-linux/regolith-desktop/issues/687#issuecomment-1205393515, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAMEE5DV6CDXUC3GNIXFLDVXPMZVANCNFSM55NA3PDQ . You are receiving this because you were mentioned.Message ID: @.***>