Walkmapper input running but not importing

[danrademacher]

I just noticed that our daily task on this project is running and seems not to be reporting errors. Here's Heroku Logs:

2021-07-30T22:31:15.945781+00:00 app[api]: Starting process with command `python walkmapper/mysql2carto.py` by user scheduler@addons.heroku.com
2021-07-30T22:31:19.739117+00:00 heroku[scheduler.6975]: Starting process with command `python walkmapper/mysql2carto.py`
2021-07-30T22:31:20.346862+00:00 heroku[scheduler.6975]: State changed from starting to up
2021-07-30T22:31:22.039966+00:00 app[scheduler.6975]: 10:31:22 PM - INFO - Found 20 obstructions in CARTO
2021-07-30T22:31:22.233539+00:00 app[scheduler.6975]: 10:31:22 PM - INFO - Found 137 obstructions in MySQL
2021-07-30T22:31:30.861763+00:00 app[scheduler.6975]: 10:31:30 PM - INFO - Sorted: 15 to skip
2021-07-30T22:31:30.861787+00:00 app[scheduler.6975]: 10:31:30 PM - INFO - Sorted: 118 to insert
2021-07-30T22:31:30.861830+00:00 app[scheduler.6975]: 10:31:30 PM - INFO - Sorted: 4 to update
2021-07-30T22:31:30.861880+00:00 app[scheduler.6975]: 10:31:30 PM - INFO - Sorted: 0 to delete
2021-07-30T22:31:30.862203+00:00 app[scheduler.6975]: 10:31:30 PM - INFO - ALL DONE
2021-07-30T22:31:30.925363+00:00 heroku[scheduler.6975]: Process exited with status 0
2021-07-30T22:31:30.991896+00:00 heroku[scheduler.6975]: State changed from up to complete

With 20 in the system and 118 to insert, we'd expect to see 138 in the map. But we have only 20 still: https://chekpeds.carto.com/viz/a71e66b8-cb4b-4c0a-aee7-3ef79a3f4b1a

I wondered if it might be a permissions issue, but I'd expect that to throw errors and I also verified that the key we have has insert and update permissions.

I wonder if parts of the script simply are not running?

[danrademacher]

This can wait till you get back -- client is on vacation till 8/18 anyway

Looking at the database directly, I confirmed we have 152 total records, of which 15 where isDelete = True:

And 137 where isDelete = 0:

[gregallensworth]

I did indeed have a large chunk of code commented out for some debugging, and did not reinstate this before I took off.

• [x] uncomment
• [x] push to heroku
• [x] set next run for 11:30 pm UTC (about 16 minutes from now)
• [x] watch the data come into CARTO

[gregallensworth]

There are now 138 entries.

https://chekpeds.carto.com/viz/a71e66b8-cb4b-4c0a-aee7-3ef79a3f4b1a/map https://chekpeds.carto.com/tables/walkmapper_obstructions