[Exploration] Infrastructure Audit and Cost Estimates for Daily User Usage

[viniciusdc]

This issue aims to track the work involved in conducting an infrastructure audit and generating cost estimates for daily user usage. The goal is to gain a comprehensive view of the costs associated with each resource, allowing for informed decisions about resource allocation and optimization. This work will be extended to other cloud providers to gain a comprehensive view of the costs associated with each resource on each provider.

Tasks

• Identify and audit the most costly resources Terraform is deploying on GCP.
• Generate a spreadsheet table for an initial overview of small/medium/large/large+gpu node groups and profiles on GCP.
• Extend the audit and table generation to other cloud providers to gain a comprehensive view of the costs associated with each resource on each provider.

Additional Information

• @dharhas requires specifications for small/med/large/single,gpu/multi-gpu instances on GCP and some storage and a way to specify usage patterns.
• @iameskild has been working on a script that can track all storage items and compute items on GCP, but it still needs some polishing.
• @dharhas also suggested manually putting the costs of 5 instance types into a spreadsheet for analysis, which can be used as a starting point for the CSV table.

Feel free to adjust the wording and formatting to fit your specific needs.

[viniciusdc]

Ok, so far, here are my findings based on the following:

• VM pricing (for user pods machine types)
• Storage pricing (for persistent volumes)
• GPU pricing

The assumptions so far:

• Prices are on-demand (without considering discounts after 1Year)
• All prices are based on GCP us-central-1 region
• NFS storage was supposed to use FileStore due to some uncertainty on how we are creating those volumes

User cost based on profile

Profile	CPU Lim	MEM Lim	Instance type	GPU type	#GPU	MEM Price (Gb)	vCPU Price	GPU Price	Total
Small	2	8	n1-standard-4		0	$0.004237	$0.031611		$0.097118
Medium	4	16	n1-standard-4		0	$0.004237	$0.031611		$0.194236
Large	90	375	n2-standard-96		0	$0.004237	$0.031611		$4.433865
Single GPU	8	30	n1-standard-8	nvidia-tesla-k80	1	$0.004237	$0.031611	$0.450000	$0.829998
Multi GPU (x4)	8	30	n1-standard-8	nvidia-tesla-k80	4	$0.004237	$0.031611	$0.450000	$2.179998
Single A100	16	60	a2-highgpu-1g		1	$0.004237	$0.031611	$2.934000	$3.693996

General Infrastructure cost

Node	Node cost	Storage shared (NFS)	Price (Gib)	Storage shared cost
General (8/32 - n1-standard-8)	$0.388472	500 (conda-store)	$0.000274	$0.137000
Worker (4/16 - n1-standard-4)	$0.194236	200 (jupyter)	$0.000274	$0.054800

*Those storage costs are based on FileStore, the resource commonly used for NFS storage, and I am using the Standard one for the expenses.

Other volumes	Allocation (GI)	Total Price (all standard) Monthly
data-keycloak	8	$0.32
data-nebari-conda-store-postgresql	8	$0.32
hub-db-dir	1	$0.04
nebari-conda-store-minio	500	$20.00
nebari-conda-store-storage	500	$20.00
nfs-server-nfs-storage	200	$8.00
redis-data	8	$0.32

[viniciusdc]

User costs for one month:

Let's start by adding up the costs for each user profile, assuming usage of (1, 1, 6, 4, 4,1) hours, respectively, over four days a week in 1 month:

• Small: (1 hour/day) x (4 days/week) x (1 month) = 16 hours CPU Lim: 2 MEM Lim: 8 GB Instance type: n1-standard-4 Total cost: (16 hours) x ($0.031611/vCPU/hour + $0.004237/GB/hour) = $1.553888

• Medium: (1 hour/day) x (4 days/week) x (1 month) = 16 hours
  CPU Lim: 4 MEM Lim: 16 GB Instance type: n1-standard-4 Total cost: (16 hours) x ($0.031611/vCPU/hour + $0.004237/GB/hour) = $3.107776

• Large: (6 hours/day) x (4 days/week) x (1 month) = 96 hours CPU Lim: 90 MEM Lim: 375 GB Instance type: n2-standard-96 Total cost: (96 hours) x ($0.031611/vCPU/hour + $0.004237/GB/hour) = $425.651040

• Single GPU: (4 hours/day) x (4 days/week) x (1 month) = 64 hours CPU Lim: 8 MEM Lim: 30 GB Instance type: n1-standard-8 GPU type: nvidia-tesla-k80 Total cost: (64 hours) x ($0.031611/vCPU/hour + $0.004237/GB/hour + $0.450000/GPU/hour) = $53.119872

• Single A100: (1 hours/day) x (4 days/week) x (1 month) = 16 hours CPU Lim: 16 MEM Lim: 60 GB Instance type: a2-highgpu-1g GPU type: nvidia-ampere-100 Total cost: (16 hours) x ($0.031611/vCPU/hour + $0.004237/GB/hour + $2.934000/GPU/hour) = $59.103936

• Multi GPU (x4): (4 hours/day) x (4 days/week) x (1 month) = 64 hours CPU Lim: 8 MEM Lim: 30 GB Instance type: n1-standard-8 GPU type: nvidia-tesla-k80 (x4) Total cost: (64 hours) x ($0.031611/vCPU/hour + $0.004237/GB/hour + $0.450000/GPU/hour) = $139.519872

Adding up all the user costs gives us $682.056384 for the month.

Infrastructure costs

The General and Worker nodes will be used constantly, so we can calculate their monthly cost as follows:

• General: 24 hours/day 30 days $0.388472/hour = $279.69984
• Worker: 24 hours/day 30 days $0.194236/hour = $139.84992

Both nodes use the shared NFS storage as well as the persistent volumes and will incur a monthly cost of:

• NFS storage: (500 GiB + 200GiB ) * $0.137/GiB/month = $95.90
• Persistent volumes: $0.32 + $0.32 + $0.04 + $20.00 + $20.00 + $8.00 + $0.32 = $49.00
• GCP monthly maintenance cost for Kubernets: $74.40

Finally, adding the user and infrastructure costs, we get $1.320,90 for the month.

[dharhas]

@viniciusdc please make this into a python function or a google spreadsheet

[iameskild]

This is a good source for a FAQ on the cost of running Nebari. Thanks @viniciusdc!