search

alan-turing-institute / CROP

CROP is a Research Observation Platform
MIT License
25 stars 5 forks source link

Add sensor data to harvest list #362

Closed mhauru closed 2 years ago

mhauru commented 2 years ago

There's still plenty I want to do here, but probably won't get to it tomorrow or Friday, and since this is already an improvement, may as well merge.

This adds averages of T&RH data to the harvest_list table from the relevant sensors in the relevant time periods. This is done through a massive SQL query that is currently just written as a string. That's because the old way of doing things using pandas would have been very slow, and I needed to learn a bunch of SQL to do the heavy lifting on the postgres server. That massive query needs to be converted into SQLAlchemy commands. Much of those can then be reused to implement batch_list and batch_details more efficiently and elegantly. Some performance optimisation could also be done for the query. It's currently bearable, but might get quite slow once more crop data comes in.

Also fix a bug introduced in batch_list by the previous PR on this topic.

mhauru commented 2 years ago

I've now converted the SQL query into SQLAlchemy and used it to also implement a first version of Zack's parallel axes plot. Ready for review when you have a moment @nbarlowATI. I'll open issues for some tweaks I have in mind that I didn't yet get to.

mhauru commented 2 years ago

I fixed the crash and added checkboxes for choosing which columns to display. By default all are displayed which I agree is too much, can adjust that based on e.g. feedback from Jakob.