True beginner's migration guide

bmueller commented 8 years ago

A lot of people chose Parse because they had no experience with developing backends. The current migration guide is a great start, but much of it is just an executive summary for people who already know what they're doing (particularly the section "Deploying to Heroku/MongoLab").

What's really needed is a true beginner's guide and best practices that takes you through all the steps of setting up a Heroku account correctly, installing node, setting up mongoDB, migrating everything over, making sure everything's working, etc. Ideally, you'd be able to follow the steps to do a complete migration of a standard Parse app with little to no experience in the platform/languages. (Sort of like the original Parse tutorials for developing your first iOS app with Parse integration.)

Otherwise, a lot of developers will be left out in the cold here, either abandoning their apps completely, spending weeks/months learning new platforms/languages, or spending thousands hiring someone else to do it for them.

kingmatusevich commented 8 years ago

Agreed

sinosoidal commented 8 years ago

+1

jdmcd commented 8 years ago

+1

nitrag commented 8 years ago

+1, has anyone done pricing of moving to Heroku though?? Their DB's aren't cheap if you want to keep your data safe and replicated.

I'm think we (developers) need to start budgeting for minimum $100/month in hosting. Not cheap for us Indie guys that haven't even launched a product yet. Experts chime in but I just need to get back to the 30 req/sec benchmark. Then be able to scale from there.

Let's also not restrict our mindset to Heroku-only and help beginners like me make the transition to the most appropriate solution.

phyominthu commented 8 years ago

+1

bmueller commented 8 years ago

Agreed, @nitrag. Looking over their pricing page, I don't really have any clue what I'd need for a simple app that has ~5k users and is doing less than 1 request per second. Would the hobby tier work for that?

michael-mansour commented 8 years ago

+1 yes a detailed guide explaining how to deploy Parse Server on Windows Azure/AWS/Google App Engine would be great !

nitrag commented 8 years ago

Looks like we'll also need to crowd-open-source a web-based dashboard. That's is handy for me while I am still developing.

On Fri, Jan 29, 2016 at 1:04 PM, Michaël notifications@github.com wrote:

+1 yes a detailed guide explaining how to deploy Parse Server on Windows Azure/AWS/Google App Engine would be great !

— Reply to this email directly or view it on GitHub https://github.com/ParsePlatform/parse-server/issues/38#issuecomment-176889908 .

phyominthu commented 8 years ago

@bmueller According to this comment, a 5$/month Digital Ocean VPS could work for you. I have a few small apps and I'm interested in Digital Ocean right now. You may want to checkout its pricing plans too.

jamesyu commented 8 years ago

The current guide is definitely our first stab at explaining the core principles, and I agree we can make better guides for each vendor.

We're thinking of these vendors: AWS, Digital Ocean, Heroku, Azure. Any others you guys would be interested in?

kingmatusevich commented 8 years ago

Linode is interesting as well, and the Digital Ocean guide could easily be worked out to also match linode's similar offerings.

bmueller commented 8 years ago

@jamesyu great to hear, I think a lot of us would be totally lost without something a lot more in-depth for all the newbies who have no experience doing any of this stuff. Really appreciate the effort you're putting into this.

arxidon commented 8 years ago

Perhaps Google App Engine as well

mattcreager commented 8 years ago

Hey Folks, Creager here from Heroku, happy to help prepare a beginner-friendly guide to getting the parse-server up and running on Heroku! In the meantime, we're working on adding a Heroku Button to the parse-server-example!

meilers commented 8 years ago

The cheapest option I see right now is to use Google App Engine + MongoLab

Any one have better ideas?

On Jan 29, 2016, at 1:04 PM, Michaël notifications@github.com wrote:

+1 yes a detailed guide explaining how to deploy Parse Server on Windows Azure/AWS/Google App Engine would be great !

— Reply to this email directly or view it on GitHub https://github.com/ParsePlatform/parse-server/issues/38#issuecomment-176889908.

jamesyu commented 8 years ago

Here's one for GAE that someone wrote: https://medium.com/google-cloud/deploying-parse-server-to-google-app-engine-6bc0b7451d50

nitrag commented 8 years ago

@mattcreager @jamesyu and Parse/Facebook staff.

It means a lot to us that you guys are still dedicated to supporting the developers and helping us find the best path forward. Getting us on the right path early (before you move on at FB) will give us the best shot at continuing to improve and grow Parse and the community.

Waht we'd like to see in your guides

Deploying to the different platforms mentioned above BUT also pro's and con's of each. What struggles/limitations can you help us foresee so we can be prepared for that
Reliability: How do we achieve the similar level of stability and data reliability we've had with Parse. With whatever platform we choose, what features/services will we need to ensure no downtime for our users.
Performance benchmarks: What level of server processing power should be considered minimum for supporting X users or X req/s? Can you baseline this since this is now something we will have to manage ourselves?
Operationally - What to look out for, such as the indexing information already provided. Tips to optimize and keep the platform running healthy/stable.

Again, thank you for you time and support.

hslightnin commented 8 years ago

@meilers https://www.digitalocean.com/pricing/ $5 here gets us close to what Parse was for free.

+1 for a ELI5 guide for multiple providers.

nitrag commented 8 years ago

@hslightnin Yes but, correct me if I'm wrong, you can't scale that if you dump the DB, parse-server, etc on the same server. With digitalocean or AWS wouldn't you need a load balancer, multiple nodejs application servers and a mongoDB replicated to be completely safe? What your talking about is potential major downtime. I was hinting at this in my previous post and waiting for some professionals to chime in.

hslightnin commented 8 years ago

True , but those of us who would never outgrow the free Parse subscription need something that isn’t going to be $400/mo.

francocorreasosa commented 8 years ago

I use DigitalOcean to run my Parse Server and it's great. Not too different to run any other NodeJS and MongoDB app.

jpv123 commented 8 years ago

This is a step by step guide I made for the transition. I hope this helps someone. I just want to say that I don't know anything about node.js and yesterday was my first day using it.

Please try this guide with a not production parse app first. It’s recommended to clone your repo and try with this copy so if something goes wrong, you don't loose any data. I’m not responsible for anything that can happen with your data, app, computer or anything.

First Time Only: Install source tree (you can use git from console if you want to) Install node.js. This also installs npm. Install Heroku toolbelt Open Terminal npm install express-generator -g //Installs a generator for express (where Parse-server works) Create a Mongolab account Create a Heroku account Create a Bitbucket account and setup source tree with this account (for git with source tree)

Every Time a new parse server is needed Open Terminal cd my/directory/where/folder/will/be/with/node.js/ express nameOfMyApp //Creates a template of node.js. MAKE SURE IS ALL LOWERCASE! cd nameOfMyApp npm install //Install dependencies npm install express —save //Install dependencies npm init Open Source tree Select create local repository Select your folder of nameOfMyApp and create repo heroku login enter your credentials of heroku heroku create echo ‘web: node app.js' >procfile //Create procfile for Heroku In source tree, commit and push to Heroku master. Wait for everything to be successful heroku ps:scale web=1 //Set a node for Heroku heroku open Verify it’s working and no error message appears. You should se a Welcome to Express screen npm install parse-server —save //Installs Parse Server. If you see a warning of kerberos ignore it

For new parse databases Create in mongolab new deployment For the new deployment create a user and a password

For migrations from Parse to mongodb Enter to parse new dashboard Go to app settings -> General -> Migrate to external database From mongolab, search for the MongoDB URI replace dbuser and dbpassword with your user and password you created Copy the whole url into parse Wait for the transition to finish. This can take a few minutes to hours. It depends on the amount of data. Once it finishes, go to mongolab and check that all your info is there.

Final steps: Open app.js. You can do it with the program of your preference Add var ParseServer = require(‘parse-server').ParseServer; at the top of the file Add this before 404 call

// Specify the connection string for your mongodb database // and the location to your Parse cloud code var api = new ParseServer({ databaseURI: ‘mongodb://yourMongoDBURL', cloud: 'parse-server/cloud/main.js', appId: ‘Your App Id', masterKey: ‘Your Master Key' });

// Serve the Parse API on the /parse URL prefix app.use('/parse', api);

var port = process.env.PORT || 1337; app.listen(port, function() { console.log('parse-server-example running on port ' + port + '.'); });

Commit and push to heroku

You’re finished!! Now, to use Parse just point your apps to the new Server. You will need at least version 1.12 for iOS, 1.13.0 for Android, 1.6.14 for JS, 1.7.0 for .NET

Rhadammanthis commented 8 years ago

@francocorreasosa Could you provide some sort of guide about how you achieve that? I had never used Node JS nor Mongo before and I'm having lots of troubles with it. I'd be forever in your debt.

francocorreasosa commented 8 years ago

@Rhadammanthis You'll need to install nginx and do a reverse proxy to the port 5000. Then install nodejs and npm. Next, create a project folder and cd it, do a npm init and npm install pm2 express parse-server --save. Then create a server.js file and paste this:

var express = require('express');
var ParseServer = require('parse-server').ParseServer;

var app = express();

// Specify the connection string for your mongodb database
// and the location to your Parse cloud code
var api = new ParseServer({
  databaseURI: 'mongodb://localhost:27017/dev',
  cloud: '/home/myApp/cloud/main.js', // Provide an absolute path
  appId: 'myAppId',
  masterKey: 'mySecretMasterKey',
  fileKey: 'optionalFileKey'
});

// Serve the Parse API on the /parse URL prefix
app.use('/parse', api);

// Hello world
app.get('/', function(req, res) {
  res.status(200).send('Express is running here.');
});

var port = process.env.PORT || 1337;
app.listen(port, function() {
  console.log('parse-server-example running on port ' + port + '.');
});

Then do pm2 start server.js -i 2

You're done

meilers commented 8 years ago

@francocorreasosa you did all that on digitalocean? via terminal? And are you hosting your mongo DB on digitalocean as well?

francocorreasosa commented 8 years ago

@meilers Yes, in my case I am hosting the DB on the same droplet but if you want you can host the mongo instance in another DB and connect it within the DigitalOcean's Private Network.

docherty commented 8 years ago

My 2¢ : There is a group of users who were drawn to Parse because it was so accessible/easy to use. Everything was in one place and it solved a lot of potential headaches. I am one of those users. Since the news of the shut down broke, I've been looking at the opportunity to switch to something like a Docker container on AWS. However, this just feels like an opportunity to fail in a production environment (I know nothing about professional security, adequate redundancy planning etc.). I've hacked together a working server but I'm sure professionals would laugh/shake their head at my incompetence.

So, I think it would be really helpful to people like me if we could put together a suggested "architecture" for purely managed services where all the security/uptime issues are taken care of by 3rd parties. I haven't found a direct replacement for the breadth of services Parse offered but a combination of 3-4 (?) might work well for a lot of people. I'm happy to contribute my findings/approach but thought the experts on here might be able to provide objective guidance. Off the top of my head, I'm thinking something like this (not done the research yet): Firebase (replaces Core Data)+Heroku??? (Cloud code)+Flurry (analytics)+UrbanAirship (push notifications)...you get the idea.

@gfosco have you got any suggestions where this kind of content could be put/discussed?

BTW something I noticed which isn't obvious in a comment above: the "Migrate to external database" button is only available in the "beta" admin web interface.

poseidonsw commented 8 years ago

I got mine up and running on digital ocean in a couple of hours as well, minus the login. Had an issue with my SB when it migrated I think, but all up and running now.

dcdspace commented 8 years ago

@docherty I agree it would be helpful to have a list of suggestions on how to rebuild our stack. Would we need to use an external service like Firebase for Core or could we survive simply on Parse Server running on Heroku?

Does anyone have any thoughts on OneSignal for Push? It is completely free. Also I wouldn't use Flurry as they are part of Yahoo, and I've learned my lesson from Facebook with Parse.

abremner commented 8 years ago

I've contacted Digital Ocean to request they add Parse Server to their list of One-Click Applications. That, with MongoDB, would be a big help for those of us without that in-depth backend experience.

Edit: Please vote for this request https://digitalocean.uservoice.com/forums/136585-digitalocean/suggestions/11629764-parse-com-as-a-1-click-app

mkll commented 8 years ago

Just rewrote a @jpv123's comment above for better readability:

This is a step by step guide I made for the transition. I hope this helps someone. I just want to say that I don't know anything about node.js and yesterday was my first day using it.

Please try this guide with a not production parse app first. It’s recommended to clone your repo and try with this copy so if something goes wrong, you don't loose any data. I’m not responsible for anything that can happen with your data, app, computer or anything.

First Time Only

Install source tree (you can use git from console if you want to)
Install node.js. This also installs npm.
Install Heroku toolbelt
Open Terminal
npm install express-generator -g //Installs a generator for express (where Parse-server works)
Create a Mongolab account
Create a Heroku account
Create a Bitbucket account and setup source tree with this account (for git with source tree)

Every Time a new parse server is needed

Open Terminal
cd my/directory/where/folder/will/be/with/node.js/
express nameOfMyApp //Creates a template of node.js. MAKE SURE IS ALL LOWERCASE!
cd nameOfMyApp
npm install //Install dependencies
npm install express —save //Install dependencies
npm init
Open Source tree
Select create local repository
Select your folder of nameOfMyApp and create repo
heroku login
enter your credentials of heroku
heroku create
echo ‘web: node app.js' >procfile //Create procfile for Heroku
In source tree, commit and push to Heroku master. Wait for everything to be successful
heroku ps:scale web=1 //Set a node for Heroku
heroku open
Verify it’s working and no error message appears. You should se a Welcome to Express screen
npm install parse-server —save //Installs Parse Server. If you see a warning of kerberos ignore it.

For new parse databases

Create in mongolab new deployment
For the new deployment create a user and a password

For migrations from Parse to mongodb

Enter to parse new dashboard
Go to app settings -> General -> Migrate to external database
From mongolab, search for the MongoDB URI
replace dbuser and dbpassword with your user and password you created
Copy the whole url into parse
Wait for the transition to finish. This can take a few minutes to hours. It depends on the amount of data. Once it finishes, go to mongolab and check that all your info is there.

Final steps

Open app.js. You can do it with the program of your preference Add var ParseServer = require(‘parse-server').ParseServer; at the top of the file Add this before 404 call

// Specify the connection string for your mongodb database
// and the location to your Parse cloud code
var api = new ParseServer({
databaseURI: ‘mongodb://yourMongoDBURL',
cloud: 'parse-server/cloud/main.js',
appId: ‘Your App Id',
masterKey: ‘Your Master Key'
});

// Serve the Parse API on the /parse URL prefix
app.use('/parse', api);

var port = process.env.PORT || 1337;
app.listen(port, function() {
console.log('parse-server-example running on port ' + port + '.');
});

Commit and push to heroku

You’re finished!! Now, to use Parse just point your apps to the new Server. You will need at least version 1.12 for iOS, 1.13.0 for Android, 1.6.14 for JS, 1.7.0 for .NET

docherty commented 8 years ago

@dcdspace Re Firebase: The reason I'm looking at it (rather than mongoDB) is that I want to replicate the functionality of Parse.User.current() in a hybrid (JS SDK) app. I use this feature to sync data between the device and the server really easily. The Firebase SDK has a similar feature (I need to do more testing but that's what I remember from a while ago). If I just wanted a datastore I'd probably use a managed mongoDB service like mongoLab.

Re. Flurry: I know what you mean but ultimately businesses are typically owned by someone else - Heroku is Salesforce. So while I agree it's a bummer, any service could disappear/change. In the case of Parse/Facebook you have to agree that they've done a pretty good job of trying to help users transition (we have a year, they're paying guys to support the community, they've open sourced their code etc.) I doubt that would have happened if it was a small independent company and the money ran out. Flurry is owned by Yahoo but it's an integral part of their ad business.

I've created a repo here: https://github.com/docherty/parse-migration-to-managed-services with my notes which I'll keep updating and eventually turn into a tutorial for the few people that want to piece together a "managed" replacement for Parse.com .

niraj-shah commented 8 years ago

If it helps, I've created a tutorial on installing Parse API Server on AWS. I have also released an Amazon Machine Image (AMI) with everything already installed, just needs configuration. See: http://bit.ly/parse-server

dcdspace commented 8 years ago

Thanks @docherty , that is an awesome repo and will be very useful to me and others I'm sure.

hslightnin commented 8 years ago

know this would be a long shot but any chance you could package this up like backendless? Standalone Backendless

JcMinarro commented 8 years ago

Here is a HowTo Step by Step to create a Parse Server on your own Server. https://medium.com/@jcminarro/run-parse-server-on-your-own-server-using-digitalocean-b2a7d66e1205#.e33bkf4m3

natario1 commented 8 years ago

@jamesyu also GAE is missing. The tutorial from medium is not enough in my opinion, totally ignores migration. A comprehensive guide should also consider holding MongoDB in a dedicated GCE cluster, see https://cloud.google.com/launcher/solution/click-to-deploy-images/mongodb?q=mongodb . That can cost less that the 180$ for MongoLab and is just as easy.

For example - just to name one of the things that a typical parse user is not understanding - when configuring the above mentioned cluster, I don't understand if I should have disk space for both data (Parse's Data Storage) and images pointed from that data (Parse's File Storage), or if the latter should go in my GAE Storage space. The migration tool is going to point to the mongo in GCE.

Another thing that is very very unclear is how ParseFiles are managed by the migration tool. That's missing in your guide.

So my suggestions are:

GAE + GCE
Better ParseFile understanding. How are these managed by the migration tool?
Better storage understanding, mostly related to files pointed from ParseFile columns. Where will they be hosted? When assigning storage, should we 10x the data storage or both data and file storage for those files?
Deeper info on "You need to manage your indexes yourself. Hosted Parse automatically adds indexes based on the incoming query stream."
More generally, please take into account that most of your users do not know (but now are forced to) anything about backends. Some phrases could be re-worded in my opinion.
Consider writing in your guide that push and dashboard are probably coming.

Kinzi commented 8 years ago

Another great guide for digital ocean: http://julienrenaux.fr/2016/01/29/complete-parse-server-migration-guide/

arcopo commented 8 years ago

I wrote up a more comprehensive tutorial than the one parse give, and actually realised it's really easy to deploy your own parse server to heroku and mongo lab [https://medium.com/@timothy_whiting/setting-up-your-own-parse-server-568ee921333a#.s09l7a5vh]

meilers commented 8 years ago

@mkll thank you very much for that. Do you think it's cheaper to setup a mongoDB directly in mongoLab and then just add its URI in app.js instead of using Heroku's mongoLab add-on?

dcdspace commented 8 years ago

Super helpful tutorial @arcopo thanks!

mkll commented 8 years ago

@meilers, your thanks should be addressed to @jpv123, I just rewrote his comment. :) As for me, I still collect the info and didn't do anything to migrate from Parse.

DrJid commented 8 years ago

This is a very nice beginner friendly write up by Timothy Whiting. Even if you've never used Heroku, Mongodb or node.js - https://medium.com/@timothy_whiting/setting-up-your-own-parse-server-568ee921333a#.dfndszn7w

brennen commented 8 years ago

I posted a guide on running Parse Server on a DigitalOcean Ubuntu Droplet (or really any Ubuntu system) earlier this morning. It's definitely not a full migration guide, but does cover installing Node and running the example code in parse-server-example.

Parse is a new topic to me personally, so I've got some learning to do, but we'd love to see a beginner-friendly migration guide supported within the scope of the open source project that includes our platform (well, and by logical extension generic Linux VPS provider installations). I can likely devote some energy to helping with that.

markuswinkler commented 8 years ago

First kudos to the parse.com developer team to open source their server. That's an incredible move on their end! Second, it would be really helpful as mentioned above to get some real world numbers for what service can handle what load and how to set it up for scaling (possibly in the millions of users). What's the advantage of AWS vs. Heroku vs. GAE vs. Digital Ocean? I am leaning towards AWS because they just have the biggest server farm there is (imho), but all the guides above are for Heroku or Digital Ocean, is AWS too hardcore to setup?

niraj-shah commented 8 years ago

@markuswinkler actually, the guide I posted was for AWS setup on Ubuntu 14 (http://bit.ly/parse-server), with a pre-built AMI you can easily launch without having to setup the software yourself.

markuswinkler commented 8 years ago

fantastic! Very much appreciated. I am not too familiar with AWS. How would I scale from there if one server is not enough? I don't have any frame of reference how much a single server can handle or process.

thewizster commented 8 years ago

Great discussion and info here... Much appreciated everyone!

auggernaut commented 8 years ago

Any word on whether Parse is going to open source the dashboard? Browsing/editing data in a GUI was a huge benefit to us when prototyping. It's gonna be a drag to build something like it from scratch...

hslightnin commented 8 years ago

Looks like the first company has started.

parse-community / parse-server