parse-community / parse-server

Parse Server for Node.js / Express
https://parseplatform.org
Apache License 2.0
20.84k stars 4.77k forks source link

How much is it costing $ you guys to host parse server? #1174

Closed jasan-s closed 8 years ago

jasan-s commented 8 years ago

This is more of a discussion question. I would love to hear from people who have deployed parse server. I have looked and mongolab and heroku pricing. it seems it would run in hundreds$$. SInce parse shutdown announcement i have considered firebase but don't like the the non-pointer and querying json structure. Although firebase pricing is quite a bit lower than using mongolab+heroku option. How much is it costing you guys per month to host parse server yourself, and how much were you paying on parse.com.

KudosGuy commented 8 years ago

I am using VPS at local provider in Europe, because AWS price in Europe is like three times higher than US prices and also now the data have to travel smaller distance... I am on Ubuntu 14.04 and setup everything by myself with the digitaloceans guide... I am paing around 120 USD per year for 30 GB SSD storage, 100 Mbps unlimited network and 1 thread Xeon 1.70 GHz... memory is good but I am not sure how much request this CPU can handle...

steven-supersolid commented 8 years ago

This is something I've looked in to as part of load testing, but only at the high end. My setup is Heroku + mLab (not using the mLab addon as that is more expensive).

I achieved around 600 requests/second split evenly between Cloud Code call, database read, database write, with: 1x Heroku Performance L dyno (has 8 cores, capitalise on this with Throng) = $500/month mLab Dedicated M1 Standard cluster = $180/month Total = $680/month, equivalent Parse.com cost = $5700, so 12% of Parse.com price This was pushing the limits of the hardware but even using 2 dynos and mLab M3 HP we are looking at $1000 + $1390 = $2390 per month, so 42% of Parse.com price.

If fewer requests/second are required then we could drop the dyno type down to 1 Performance M at $250/month but I found these to only have 2 cores, so not the same value as Performance L. The advantage of the Heroku Performance line is that they are dedicated instances, but this is not what we get on Parse.com so we could lower costs by dropping to Standard 1x and 2x at $25/$50 month per dyno respectively. Choosing the Hobby dyno type with mLab Shared Cluster will result in costs of paying tens of dollars per month and there is a free tier on each.

What I've not factored in is the behind the scenes service we get on Parse.com where the team of ops and backend engineers keep things running smoothly with the platform. Building your own setup on AWS is going to be the cheapest but then you have to maintain it. I chose Parse to avoid that. I've not used the support at Heroku but the mLab support is amazing.

jasan-s commented 8 years ago

wow , it does seem for a MVP of a product, firebase will be a lot more cost effective

otymartin commented 8 years ago

I deployed to app engine. So far as the only user of my app deploying my server 10 times a day, I racked up around 55USD in a month. Luckily I'm using 300dollars in free credits but its still not a good thing. Im hoping the problem is on my side because sometimes app engine/compute engine spins up like 8 instances while remember, Im the only one using my app (Messaging app).

flovilmart commented 8 years ago

Disable auto scaling, delete / stop unused versions

jasan-s commented 8 years ago

Actual $ amounts for comparison will be helpful On Mar 24, 2016 4:01 PM, "Amisalabs" notifications@github.com wrote:

There is huge cost savings from using Nodechef's managed parse server and mongodb. Nodechef actually uses mongorocks so compressed storage saves you money. Pricing is also cheap when compared with Heroku/mlab for small and big data considering Nodechef provisions containers on bare metal servers.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/ParsePlatform/parse-server/issues/1174#issuecomment-201065471

noder199 commented 8 years ago

@jasan-s firebase is gonna disappear at some point, Google does this with a lot of their acquisitions.

jasan-s commented 8 years ago

I am afraid of that as well but for a minimum viable product for an app I am hoping it lasts long enough and hope they offer migration similar to parse.com. I loved parse but with all the comments it seems for a product that still needs validation . firebase is more economical initially. On Mar 24, 2016 9:03 PM, "noder199" notifications@github.com wrote:

@jasan-s https://github.com/jasan-s firebase is gonna disappear at some point, Google does this with a lot of their acquisitions.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/ParsePlatform/parse-server/issues/1174#issuecomment-201127125

harrygt commented 8 years ago

T2 Micro Instances on Amazon AWS and auto scaling with Mlab.com free tier database. It's about $25 a month which is fine for a small usage app. If you're new to Amazon then you get a free years usage on those settings.

Knana commented 8 years ago

For many apps, the $9 cluster (1 GB database and 128 MB app containers) from Nodechef is adequate. The below example uses a news aggregation app which is doing ~100 req/sec The Parse Server boots at around 65 MB of the app memory so you have around 63 MB left. Most requests read around 30 rows maximum. When rows are batched as in 30 rows to show on a page, they barely exceed 1 KB per row. And when the article is actually displayed, that row does not exceed 32 KB. Since the row has to be in memory before it is transported, you can simply calculate (100req/sec x 1 KB) which is not even 1 megabyte of memory. Note that req/sec is mainly on the database size and Nodechef's SSD and mongorocks setup is fast.

I work at Nodechef and we provide a fully managed Parse Server and MongoDB with the RocksDB storage engine.

flovilmart commented 8 years ago

@Knana thanks for being awesome :trophy:

otymartin commented 8 years ago

@flovilmart how do i disable auto scaling on app engine?

flovilmart commented 8 years ago

Check the app engine documentation, Python docs on app.yml particularly

FridaySG commented 8 years ago

@Knana would like to hear more on the fully managed side of things. So if I provided an S3 setup to use with the FilesAdapter you guys could make that happen?

stephanelpaul commented 8 years ago

i use two instances of digitalocean, one for the mongodb and one for the app, for files management i used s3, its not costing me more then 40$ a month.

Although i'm working on a postgresql adapter will be using amazon rds.

Knana commented 8 years ago

@FridaySG as a fully managed service, you do not have to worry about setting up your Parse Server and database. With just one-click you can launch a Parse Server and MongoDB cluster. We handle the entire backend like Parse.com. we track every release of Parse Server and update shortly after it is released. You can certainly use your S3 setup at NodeChef. Please refer to the documentation on configuring external files storage (S3, GCP)

davimacedo commented 8 years ago

Another option is back4app. We are a Parse Server Hosting provider that allows the developers to build and host Parse Server apps with no worries about servers, deployments, etc. You only have to create your app (or migrate an existing one with our 5 min migration wizard) and start doing your requests. We automatically generate and manage your Parse Server and Parse Dashboard. We also have our own dashboard that provides some additional features (like Cloud Code Management) and will provide even more in the future.

You don't have to worry about how much memory or CPU or storage you need. We take care of all your infra-structure and scale your app up for you, like it was in parse.com. There is not req/sec limit. You pay for the total amount of requests per month. The advantage is not paying for the whole month of a complex infra-structure to cover occasional peaks of 600 req/s (for example). You pay for the requests you did, no matter if some time you needed do 600 req/s and other times no requests/s.

Because we are not providing a single server for you, but a hosting solution, we can optimize our servers usage and offer the most affordable plans in the market.

Our FREE plan includes up to 1GB of data storage and 100,000 requests / month forever. It is actually enough for development stage and also for production stage in the case of many apps.

Then you can migrate for one of our paid plans. You can find them here: https://www.back4app.com/pricing/

gateway commented 8 years ago

Please make sure that what ever mongo database service you are using provides backups and scaling depending on the traffic. Last thing you want is your database to die w/o having a backup and the ability to scale or fail over to another mongo instance to keep your app up and running.. Just a FYI

Copyrightsworld commented 8 years ago

@Knana can you provide some more details on your company? How long have you been around and what is your current status ? What i mean is. Why couldnt you just be another "Parse.com", shutting down in a year and we all have to move out again?

Knana commented 8 years ago

@globyworks I do understand your concerns. And they are valid concerns. We are building a platform for the Node.js and MongoDB community and the response has been very positive. We are very confident in this space. I can't tell what will happen a year or two from now but I know we are committed to delivering a solid product and being around for a very long time.

flovilmart commented 8 years ago

In the end, parse-sever and parse-dashboard being open sourced, you can migrate from one server implementation to another while having on strong hold on your data. That's what matters most in the end no?

hramos commented 8 years ago

Thanks to everyone for sharing their experiences hosting Parse Server. I am closing this out as we're trying to cut down on the number of non-issues. This will help us focus on the issues that are actively preventing people from migrating their production apps to Parse Server.

As an alternative, I suggest opening a new discussion at Server Fault, a Q&A site dedicated to "questions about managing information technology systems in a business environment".

andresdouglas commented 8 years ago

@davimacedo few questions I didn't find answers for on your website:

davimacedo commented 8 years ago

@andresdouglas you don't have to worry about upgrading the amount of storage. The exceed will be charged at the end of the month period. All infrastructure is running on AWS east and both Parse Server and MongoDB run on clusters.

andresdouglas commented 8 years ago

@davimacedo thanks for the reply. What do you mean by "clusters". How many replica sets does mongodb have across how many availability zones?

davimacedo commented 8 years ago

You're welcome @andresdouglas.

We have been working very hard to provide the reliability that Parse clients use to have. I have to confess it is not an easy task to do in two months, but I am sure we are doing a good job and we are in the correct way. :)

For instance, we have recently migrated all apps' databases to use the RocksDB engine on our MongoDB servers. That way, we can use the same tools Parse used to create constant database backups, while delivering a better performance for the user. It also reduced almost 10x the size of our customer's database. I am afraid that everybody now will use back4app for FREE :)

Despite not having a big user base as Parse has, we have been concerned with service reliability from day one. As the traffic grows, our database infrastructure is prepared to easily receive more nodes inside four availability zones in the US East region.

Due to the increase in apps' creation since the Parse migration process started, we have been constantly monitoring our servers. Automatic alarms are configured to warn whenever a node is reaching the limit of acceptable usage level. That allows us to create new replica sets before any noticeable impact on performance.

There is also in our product roadmap to create replica sets in different regions. So customers in different places will have shorter latency time.

If you are not comfortable in using our Database infrastructure, you can also use other MongoDB specialized provider (mLab, ObjectRocket, etc) and keep using back4app for Parse Server.

gateway commented 8 years ago

As a backend developer I'm looking at a few routes. AWS Elastic Beanstalk along with MongoDB cloud service https://www.mongodb.com/cloud, while running their databases in AWS East across separate zones for fail over and back up. If parse would use the new data engine from Mongo which is in the 3.2 branch you get your compression back like they have on their parse.com system. So you use AWS EB to scale your parse servers up and down based upon rules, while using the mongo cloud service to run your databases. Cost can very depending on the aws instances you use but as you scale you can up them to a bigger server or scale even more nodes horizontal.

I would caution anyone from maybe using another service provider to do this for you because well you could be faced at some point they go out of business for some reason and your back to square 1.

just my 2 cents right now and Ill be writing up an article on my experiences across IMB Blumix, Google Cloud and AWS.. but I think one things for sure Ill prob use the MongoDB cloud service to handle, monitor and run my db instances in AWS.

alyssoncm commented 8 years ago

Updating this topic, I wrote an article about the costs to host Parse Server on AWS. Chek it out and let me know your opinion. :) http://blog.back4app.com/2016/06/21/parse-aws/

blacha commented 8 years ago

Any reason as to why you don't use the t2.micro instances? I haven't had any bad experiences with them behind a autoscaling load balanced elastic beanstalk. 1-10 t2.micro instances depending on time of day/load.

Amazon offers a free tier (t1.micro instances), but this instance does not support Parse Server.

I am pretty sure parse server should work fine on a t1.micro, generally it only chews around 100MB of ram so should have plenty free.

flovilmart commented 8 years ago

Running on mLab + Google App Engine,

We have steady 100 requests per second on 4 n1-standard-1 (1 vCPU, 3.75 GB memory), all at around 40% CPU that would be about $100/month.

mLab is running on dedicated cluster on M2, in Google Cloud Platform for $480 a month.

Not to mention latency is very good In our case, it's about to be cheaper than running on parse.com.

otymartin commented 8 years ago

@flovilmart Good insight. Im not in production yet but using the same tools (Sandbox on mLab for now)

oallouch commented 8 years ago

I'll definitely try back4app . Their offer looks great, specially their sense of completeness (I'm thinking about backups).

steven-supersolid commented 8 years ago

@alyssoncm I think the database costs in your article could significantly lower for 10s to 100s requests per second.

Using mLab a few hundred reads/writes per second are easily possible on the standard M1 dedicated cluster for $180 per month and a few thousand reads/writes per second are possible on high performance M3 dedicated cluster (there is no M1 or M2 in HP) for $1,390 per month.

If you want to go slightly cheaper then you can build a similar setup on AWS using the same instances and storage types. Moving off mLab also enables you to switch to WiredTiger or use MongoRocks but I can't see the need unless hitting high thousands of reads/writes per second. Please note the mLab SSD cluster is only primary + secondary + arbiter (or hidden secondary on HP).

In the article i2.large is mentioned (I think this is a typo as the minimum is i2.xlarge at the same price) but this has 800GB storage and 35,000 read/write IOPS which is in the range of 100s-1000s reads/writes per second (depending on data size). I think mLab uses the c3 instance type to get the previously mentioned performance figures and these come in smaller SSD configurations. Again, SSDs should only be required in the high hundreds of request/second range.

zhouhao27 commented 8 years ago

I'm curious why you guys just run mongo db in mLab instead of in EC2 instance directly?

mitchellporter commented 8 years ago

@zhouhao27 many people don't have the skills or feel comfortable enough to manage an ec2 instance.

MrIceman commented 8 years ago

Firebase guys

chadpav commented 8 years ago

I've migrated multiple startup/mvp app's from parse.com to AWS/mLabs. For the most basic instances I'm using the $15/mo shared plan at mLab's and a t2.small instance behind a load balancer from AWS. All of AWS (EC2, S3, LB, etc) is running about <$25/mo.

The nice thing about this is that I know it will scale when ready (the infrastructure at least). We are using Elastic Beanstalk node.js instances. They are "managed" and can be configured to auto scale up/down based on utilization. I'm not worried about infrastructure, backups, or server maintenance.

I have a second client who is using the same setup but they are on (2) t2.medium Parse Server instances, a redis cache, (2) t2.medium Live Query servers, (1) t2.medium server that just handles cloud code. Images are stored in S3 and they have the dedicated cluster at mLab for $180/mo. Their AWS bill is coming in around $900/mo but that includes some other small services and additional instances for a DEV environment too. Additionally, we have auto scaling enabled up to 10 Parse Server instances so that happens when they are conducting a large user test with thousands of users.

Hope that helps!

Firebase - I have a personal project on Firebase because I wanted to see first hand what the pros/cons would be. First impression... it's apples/oranges. Firebase has some serious limitations that you should be aware of. There are some things that are better though. I'm working on a presentation to discuss the pros/cons right now. ;)

mnearents commented 7 years ago

I'm curious if anyone has experience with, or advice about Parse on Azure Managed Services which uses DocumentDB instead of Mongo:

https://azure.microsoft.com/en-us/blog/announcing-the-publication-of-parse-server-with-azure-managed-services/

flovilmart commented 7 years ago

@milostt advertising is not tolerate. Your account has been reported.

flovilmart commented 7 years ago

@jeacott1 that's out of the scope of the question also...

agwl-saurabh commented 6 years ago

Hi All,

I need your help in updating my parse server in terms of infrastructure. I have total 8 app and i am running on t2.small servers. one for DB and one for parse server and dashboard. so i have total 16 servers for all 8 apps. I started with 4 servers only and now my biggest issue is maintenance of all these server now i have almost 500000-700000 user in all app . and expected to grow 1 million currency i am facing lot of down time due to serves and not able to update my apps.

I want to make a single server which can hold atleast 1-2 Million users (i am not sure how to calculate request/sec of parse server or how much load server can pick) . My app is like instragram where user have login , photo,video upload profiling, comments like etc. so there is lot of mongo DB request.

I have already created mongoDB 3.4.7 replica set with mongoRock. (2 x t2.medium and one t2.micro) total DB size of all of my app is 42 GB (all videos and images are on mongoDB now recently i learned to store media on S3 ) so media will be stored to S3 but not sure about server configuration.

below I need info.

  1. how you are calculating the parse server request per sec or how to calculate if my current server are capable of handling x number of user.
  2. what exact server (please tell me the type of server also like t2,m3) i need to handle 1.5-2 million user app this app is like Instagram daily active user are approx half million
  3. Can i migrate RockDB data to default MongoDB if in future if i need to move to some dedicated service provided for mongo. right now its costly option.

@flovilmart @chadpav @steven-supersolid please help. !!!

flovilmart commented 6 years ago

You should first start to instrument monitor your code to get exactly how many RPS you’re doing right now, this will help you in the future. There are plenty of options, from grafana/prometheus to datadog, new relic etc...

The server size don’t really matter, you’ll realize it soon enough, in itself, parse is CPU bound and node is not multithreaded, to scale on a quad core machine, you’ll need 4 processes, if you’re using the cli startup script, you’ll be able to use the —cluster option, effectively spawning more processes; otherwise you can Look into throng.

For the sake of discussion, we’re running on kubernetes, it has it’s issues, and is not trivial to deploy but we’re sustaining 1000rps with a response time of 30ms.

agwl-saurabh commented 6 years ago

After today full day work some how i am able to calculate my app request per second. I have 50-70 request per second for one app i have total 8 main app n 3 very small app so i think total request for all app is 400 - 700 request per second. on the peak i think i will have approx 500+ request.

@flovilmart . you scared me by saying server size does not matter. i am already straggling to find the salable solution for my failing app and still not sure if my app is failing due to DB or parse server it self.

kubernetes are some google docker system i am not sure how to use them . i am on AWS

I am running my parse server on t3. medium AWS and my CPU uses is hardly 10% as per DataDog. and parse server (Node Js ) request is 50-70 request per second. does it means this server can handle more load. or I should apply rule in loadbalncer if one instance have 50+ request then create new instance. this i have seen in some issue "flovilmart" mentioned they have setup loadbalancer for 50 request per second so that respond time of parse will remain constant

As per the parse document I was assuming if i run parse server using "Aws beanstalk" by the button provided on git my app can handle any number or request just i need to take care on DB. but now its looks like there is more needs to be done to make app scalable as i am already on default parse Aws beanstalk but my app is falling . can any one please tell me what more needs to be done. i am new in backend and still learning . AWS S3 loadbalncing node JS mongo rockBD lot more from last 6 months .. :(

chadpav commented 6 years ago

@agwl-saurabh you need to open a different discussion topic with your questions about how to use Parse at scale. I think that would be an interesting thread on it's own.

agwl-saurabh commented 6 years ago

Thanks @chadpav for great Idea I just open an new discussion. if you like to share your experience please share on below

https://github.com/parse-community/parse-server/issues/4278