search

applait / meta

Policies and decisions and workflows for Applait
0 stars 0 forks source link

Start conversation about credits from cloud services #23

Open kaustavdm opened 7 years ago

kaustavdm commented 7 years ago

After #22, @debloper, you need to reach out to the cloud service providers and apply for credit. Rope me in the conversations. But you take charge. This may depend on #24 as you may be asked to answer related questions.

Cloud providers (in order of preference):

Azure: Gives up to 10K USD in monthly credits for 12 months. Hence top of the list. They have got tier 1 networks with most points of presence globally. FWIW, we are most likely to get their credits because Microsoft is pushing hard for increasing APAC sales this FY.

GCP: Gives up to 1K USD in monthly credits. Has maximum flexibility of servers configurations. Plus, is backed by the most resilient network tier any cloud service offers today. (Edit: Plus, good dashboard)

DO: Gives up to 1K USD in monthly credits, but has an additional clause stating the company needs to be less than 2 years old. They are the cheapest shot, hence worth talking. Poor network IO compared to the 2 others, but still better than AWS (which is why AWS does not feature on my list). Let them know our bandwidth usage and get network guarantees. FTR, I don't want to go with DO for the network performance issues, but we can always have our app servers running on their infra.

We are eligible for all three as part of 10K Startups programme, still, these companies would be eager to put in free credits if they see any progress on our side.

The crucial part here is to explain each of them our use case and resource requirements so that neither parties are in for surprise when the usage mounts on when the bills are generated.

Cloud network traffic is shit expensive

Cloud services are shit costly when it comes to bandwidth. As part of your outreach, also negotiate bandwidth cost with the providers. This will already start to hurt us for multi-streaming and will become Peter-North-up-your-ass for interactive streaming.

Some numbers

Just for multi-streaming, having only 500 users stream for 90 hours a month, 3 providers each, at 3.5 MiB/sec per stream, would cost us 1622TB of egress bandwidth.

GCP and Azure charge anywhere in between $0.08 to $0.2 per GiB egress depending on region. Even if we avoid sending data to the expensive zones, we would have a median cost of $0.10 per GiB egress on these cloud platforms, which amounts to about $166,113 for bandwidth bills per month for this workload. We need high throughput environments and doing media services on cloud platforms would cost us dearly.

DO doesn't charge for bandwidth yet, but gossip says they will have a price point of $0.08 to $0.09 per GiB egress, which they may flatline to $0.05 per GiB at announcement. But getting this resource usage out of DO is very unpredictable and they cannot guarantee you high grade network I/O because they share it with many VMs.

Solution

The solution can be to put our media deployments on bare metal for production scenarios. Bare metal offerings provide you with dedicated network ports and CPUs and they have earthly prices for network traffic. Some of them even offer unmetered connections. Details in #25.

Edit: If we do get excellent bandwidth price from cloud providers like GCP or Azure, we wouldn't have much reason to go bare-metal through other data centres.

dolftax commented 7 years ago

Consider Alibaba cloud? Not sure about numbers :p but they are being very generous. If 10k USD a month is an email away, why not.

screenshot_20170906_123849

kaustavdm commented 7 years ago

@debloper Can you please ask GCP to give us boatload of credits? Azure's portal gives me nervous pangs :flushed: :worried:

debloper commented 7 years ago

Some notes after preliminary researches:

All of them are 10,000StartUps partners. So only blocker here is to convince accelerator to promote us for the credits. Individually approaching the services would be less efficient and possibly less fruitful. Need to discover if they're exclusive of any sort, that accelerator program pitches for "either" one of them; but on the surface it doesn't look like it.

If we can get multiple platform credits, we can try to subdivide the service components in order to financially load balance them.

Marking this blocked by #22

debloper commented 7 years ago

...

Apparently, I just RE-searched the same shit that's the first half of the issue. BRB killing myself.

kaustavdm commented 7 years ago

That's funny, but useful as well. Validates my figures.

Additional info: Applait is part of BizSpark already :-) So, we are eligible for the basic goodies. We never used the BizSpark credits for Azure, I kept it for future purposes. 10K Startups makes us eligible for the BizSpark Plus, but it needs their recommendation. Note that these are still met on a case-by-case basis. MS may give extra or lesser.

The best multiplatform deal would be GCP + Azure. We are least likely to get DO, ironically.

kaustavdm commented 7 years ago

@debloper I think you added an extra zero for DO. They give 10K, last I checked.

kaustavdm commented 7 years ago

Also, in our current architecture, we are able to distribute components, including core services, media services and analytics across different providers.

dolftax commented 7 years ago

Not sure about performance/stability of azure machines but they admin UI almost never works. @kaustavdm Why were you behind azure? Got numbers?

kaustavdm commented 7 years ago

I tested with Pune and Singapore DCs.

Azure's F series VM, F4, (which is what we should use for edge servers) reached ~1600Mbps while pushing traffic to a GCP server (external network). I only kept it on for 5 minutes though. Didn't want to get poor on bandwidth costs.

Azure publishes expected network perf figures. Check https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-compute.

GCP also had similar figures somewhere.

kaustavdm commented 7 years ago

@debloper can you also reach out to OVH for their DLP programme (https://www.ovh.com/us/dlp/)? We can use their bare metal offering for media servers. So we need them to extend credits for bare metal which they do in phase 2 of their programme. We need to convince them that we need bare metal credits for the throughout and bandwidth.

100 users can be easily handled using just 2 of their bare metal servers, resulting in a cost of ~$300-400.

This is related to #25 though.