Database stats report is returning 0 for data

[louh]

Background: With the recent back-in repository merge back into the main repository, we're having to recreate some of the functionality of the back-end Heroku app on the main Heroku app. This includes the shell script that is run daily by Heroku Scheduler to report database usage statistics.

Currently both apps are running. The streetmix-api-v2 app has been reporting the same data since September 5, as expected. I have set up the Scheduler on streetmix-v2 (combined) app yesterday to execute the same script, but today the observed result is that an e-mail was sent to the Streetmix e-mail list containing:

# of registered users = 0
# of streets = 0
DB storage size = 0 MB
DB storage utilization = 0%

None of the data should be actually 0, obviously.

Here are the logs from Papertrail (only the lines related to scheduler):

Sep 15 16:32:42 streetmix-v2 heroku/api:  Starting process with command `./bin/email_heroku_db_stats.sh` by scheduler@addons.heroku.com 
Sep 15 16:32:45 streetmix-v2 heroku/scheduler.8333:  Starting process with command `./bin/email_heroku_db_stats.sh` 
Sep 15 16:32:46 streetmix-v2 heroku/scheduler.8333:  State changed from starting to up 
Sep 15 16:33:24 streetmix-v2 app/scheduler.8333:  ./bin/email_heroku_db_stats.sh: line 66: let: 23:33:22.326: syntax error in expression (error token is ":33:22.326") 
Sep 15 16:33:24 streetmix-v2 app/scheduler.8333:  exception: connect failed 
Sep 15 16:33:26 streetmix-v2 app/scheduler.8333:  exception: connect failed 
Sep 15 16:33:26 streetmix-v2 app/scheduler.8333:  ./bin/email_heroku_db_stats.sh: line 67: let: 23:33:24.399: syntax error in expression (error token is ":33:24.399") 
Sep 15 16:33:28 streetmix-v2 app/scheduler.8333:  exception: connect failed 
Sep 15 16:33:28 streetmix-v2 app/scheduler.8333:  ./bin/email_heroku_db_stats.sh: line 69: let: 23:33:26.507: syntax error in expression (error token is ":33:26.507") 
Sep 15 16:33:30 streetmix-v2 heroku/scheduler.8333:  State changed from up to complete 
Sep 15 16:33:30 streetmix-v2 heroku/scheduler.8333:  Process exited with status 0 

It seems the Mongo connection is dropping. I'm not a shell script expert. How to fix this?

cc @drewda @ycombinator

[ycombinator]

I'm looking into this. I might have to run the script a couple of times manually which might generate reports. I'll report back with what I find.

[ycombinator]

Here's the issue:

On the old streetmix backend app (streetmix-api-v2), the value of the MONGOHQ_URL is something like this:

$ env | grep MONGO
MONGOHQ_URL=mongodb://XXXXXX:XXXXXXX@XXXX.mongoZZ.com:XXX/XXXXXXXX

In particular note the hostname and port combination. There is only one: XXXX.mongoZZ.com:XXX.

On the new streetmix combined app (streetmix-v2), the value of the MONGOHQ_URL is something like this:

$ env | grep MONGO
MONGOHQ_URL=mongodb://XXXXXX:XXXXXXX@XXXX.mongoZZ.com:XXX,YYYY.mongoZZ.com:YYY/XXXXXXXX

In particular note the hostname and port combinations. There are two: XXXX.mongoZZ.com:XXX and YYYY.mongoZZ.com:YYY.

The latter is valid syntax for MongoDB Connection String URIs. Consequently what needs to be fixed here is the code on this line to parse out a host and port combination correctly. Fix incoming.

[ycombinator]

@louh The fix is in but I need someone with sufficient Heroku privileges to redeploy the Streetmix app.

[ycombinator]

I take that back. I have privileges to push to Heroku; my SSH key just needed to be updated.

I've pushed the app now and manually run the bin/email_heroku_db_stats.sh script. As you'll see in your emails, non-zero stats are back!

[louh]

Awesome. Thanks much @ycombinator :smile_cat:

[louh]

Further note: I've now removed the scheduled job from streetmix-api-v2.