Make DB Persistent

[meneal]

Looking at a few options for this.

• Mounting or using intermittent upload/download to a S3 bucket to keep the sqlite db between shutdowns so as to avoid rebuilds.
• Using one of the addons for Heroku to add in a db that's separate. If the API can talk to one of these options let me know and I'll implement it. I think it shouldn't be too hard to do this way, but I guess I'd want to know what the path of least resistance is.

Either way should work. I think long term the second option is preferable, since we could have DATA LOSS (noooooooo!!!!!) otherwise if we somehow have the db go down in between image saves. We can probably short term make it work with the S3 option.

[duh102]

I can use either the PostGres or MySQL databases, Node supports both (the others are either document DBs or I don't know about them), MySQL is probably the easier of the two. I can work on porting the DB over today or tomorrow, depending on how busy I get.

[meneal]

Sounds good. Just let me know when you want to do it and which one you want to use for sure and I'll set up the addon.

[meneal]

Actually make sure you sign up for a Heroku login and i'll just give you access to the dyno.

[duh102]

I think you invited me to it last night? I got an invitation at least, signed up with wemorrow@ncsu

[meneal]

Sweet. I wasn't sure if you got that. Some of them looked like they errored out or something.

On Mon, Mar 16, 2015 at 2:17 PM, Will Morrow notifications@github.com wrote:

I think you invited me to it last night? I got an invitation at least, signed up with wemorrow@ncsu

— Reply to this email directly or view it on GitHub https://github.com/CSC510-2015-Axitron/maze/issues/49#issuecomment-81850856 .

[duh102]

I think I've successfully changed the DB over from SQLite to MySQL (finished feature in 8145860e8f024f8e1fe96ffb419a16b6d6db627a), which requires a config.js file. Standard practice is to have a config_example (or in this case, config.js_ex) and then make a real config.js of your own. Would heroku destroy that every time we pushed? Should I include config.js in the repo?

[meneal]

Config.js should be in the repo. Heroku will destroy it. Are you going to use that jawsdb addon?

On Mon, Mar 16, 2015 at 3:08 PM, Will Morrow notifications@github.com wrote:

I think I've successfully changed the DB over from SQLite to MySQL (finished feature in 8145860 https://github.com/CSC510-2015-Axitron/maze/commit/8145860e8f024f8e1fe96ffb419a16b6d6db627a), which requires a config.js file. Standard practice is to have a config_example (or in this case, config.js_ex) and then make a real config.js of your own. Would heroku destroy that every time we pushed? Should I include config.js in the repo?

— Reply to this email directly or view it on GitHub https://github.com/CSC510-2015-Axitron/maze/issues/49#issuecomment-81877573 .

[duh102]

I only worry about it being a security risk, since that information will be public. I was thinking of using the ClearDB, but as long as whichever one we choose supports MySQL, that's all that matters. The shark one seems to have more storage allocated in the free (only?) plan, so I'll start out with that and see what I can get accomplished.

[duh102]

I looked into it, connection parameters can actually be grabbed from an environment variable, so config.js isn't even needed for heroku.

[meneal]

I was thinking that might be the case. Heroku is pretty sweet man.

On Mon, Mar 16, 2015 at 3:47 PM, Will Morrow notifications@github.com wrote:

I looked into it, connection parameters can actually be grabbed from an environment variable, so config.js isn't even needed for heroku.

— Reply to this email directly or view it on GitHub https://github.com/CSC510-2015-Axitron/maze/issues/49#issuecomment-81896397 .

[duh102]

Had to fiddle with many small things, but it's now working (or seemingly, all my manual tests are working). To try it out, try doing the following in sequence:

• curl --data '{"email":"(some email)","password":"(some password)"}' http://axemaze-db.herokuapp.com/register -H "content-type:application/json"
• curl --data '{"email":"(some email)","password":"(some password)"}' http://axemaze-db.herokuapp.com/login -H "content-type:application/json"
• curl http://axemaze-db.herokuapp.com/user/(id) -H "content-type:application/json" -H "authorization:(token)"
• curl http://axemaze-db.herokuapp.com/logout -H "content-type:application/json" -H "authorization:(token)"

The DB itself works fine, API just needs testing, so I'll put a QA on this and close it tomorrow.

[duh102]

Oh, as an added bonus, I took out a lot of unnecessary things in the dbdev branch. There are some unnecessary libs in package.json and the levels folder is still there, but everything else that's not necessary has been cleared out.

[meneal]

The third command comes back with invalid syntax missing parameters. But that makes sense right?

[duh102]

Did you fill in your user id there? It should just repeat back to you your email and user id if you're the right user and with a valid token.

[meneal]

You're talking user id number right?

[duh102]

Yes, if you were user 3 with token "abc" it would be curl http://axemaze-db.herokuapp.com/user/3 -H "content-type:application/json" -H "authorization:abc"

[meneal]

On line 271 it looks like you're requiring a username and password sent in the post right?

[duh102]

That's if it's a POST. You can also GET the same address, and as long as you're the correct user and have a valid token for that user it'll give you back your email and user id.

[meneal]

Oh I gotcha. Nice thing about that is I already got the username changing function working!

[duh102]

The maze stuff is a bit complicated but not too bad, really it's just a maze JSON like we have for the level files now, but with a few things added. They look like the following (I've expanded the data portion for readability): curl --data '{ name:(name) (,category:(category)), maze: { height:(height), width:(width), start:[0,0], end:[0,0], board:[[0,0,...],[],...[]] } }' (addr)/maze -H "content-type:application/json" -H "authorization:(token)"

category is optional, if it's not there the maze will not be put into a category and won't be visible if you search for mazes by category, you can only load it by knowing the maze number (which is returned to you upon submission)

[duh102]

Which reminds me, I should make some sort of function to get a user's mazes so you can get them even if they're not in a category.

[duh102]

Found I had a bug in the maze submission/editing code, seems the mysql bridge doesn't understand named placeholders while the sqlite one did and I didn't replace them. Also changed the old /mazes/:category to /mazes/category/:category and added /mazes/user/:user in (3c733c0899c761709013935f9d8d21603b746233)

[meneal]

So here's one thing I was thinking about. Should the function on the client side for adding mazes or editing them take a string or a json as input? Either way is workable for me. I just wanted to check to see if there was a difference for everyone else. I guess alternatively I could do both.

[duh102]

When you get the reply from the server, it's a string, or at least it would be unless your REST client lib fixes it for you. On the designer side, I'm pretty sure we could modify the designer to output in either form (currently a string), so I dunno. If the REST client doesn't auto-convert to JSON I think it might be better to use strings, since that's the stuff you'll be getting on both ends, but if it does, then it'd be nicer to change the designer's output to JSON.

[meneal]

Actually for coding the client they're equivalent in difficulty. I'll have it accept strings and then convert it to JSON.

On Tue, Mar 17, 2015 at 2:24 PM, Will Morrow notifications@github.com wrote:

When you get the reply from the server, it's a string, or at least it would be unless your REST client lib fixes it for you. On the designer side, I'm pretty sure we could modify the designer to output in either form (currently a string), so I dunno. If the REST client doesn't auto-convert to JSON I think it might be better to use strings, since that's the stuff you'll be getting on both ends, but if it does, then it'd be nicer to change the designer's output to JSON.

— Reply to this email directly or view it on GitHub https://github.com/CSC510-2015-Axitron/maze/issues/49#issuecomment-82510305 .

[duh102]

Gonna close this, since it's pretty much done. Open new issues for bugs or new features.