search

webcompat / webcompat-metrics-server

Server in charge of delivering different data to the webcompat-metrics-client
Mozilla Public License 2.0
6 stars 5 forks source link

Fixes #90 - Insert context and fix model types so scheduled scripts can write to DB #91

Closed laghee closed 5 years ago

laghee commented 5 years ago

I enclosed the database interactions for daily_total in an app context as a test. Will test by deploying this branch to heroku for a few hours tomorrow. If that looks successful, I'll make the changes to the get_count script as well... 🤞

laghee commented 5 years ago

@karlcow OK! Exciting!

By deploying this branch on heroku -- and then running the script manually in the heroku console so I could see errors and real-time logging -- I learned that using with app.app_context(): to wrap the DB actions worked. No more errors were shown in response to the script running. But when I inspected the Postgres add-on... there was still no DB action happening.

Also, there was no logging showing up, either positive or negative, which made figuring out what was wrong... not straightforward. When I looked at the logging configuration, I realized it was set up to write to a temp log file rather than stdout. Fixing that meant that I was able to see my logging failure message from the script. Yay! 😁

Only all my failure message was configured to say was that the DB write failed... which wasn't very useful 😞 . I changed the message to include the specific SQLAlchemy error thrown. From there, I was able to see that the write failed because... the tables didn't yet exist. 💡

Since it didn't make sense to me to have a line in the script create the tables for the first time and then keep running daily, I ran db.create_all() from the heroku console. This failed with an error saying that DATETIME was not a valid type. I went back and changed the types for the tables from all caps. For some reason, my autocomplete formatting went wacky in VScode, and this had been changed from DateTime. After I made that change, I tried to create the tables again on the console, and this now worked fine -- I could see that 2 tables existed in the DB. 🎆

When I re-ran the daily_total script in the console, I got a log message telling me that the data was successfully written to the table, and querying the table with pg:psql, I was able to see our lovely first row of input: screenshot 2019-02-24 00 14 13 🎉 😁

I'll make the app context change now to the get_count script as well.

karlcow commented 5 years ago

What an amazing fest of finds. Thanks a lot @laghee You rock!