Integrate processing data for display

[matiasandina]

This is the code I worked on. I hesitated whether to include this as separate or not...Let's discuss where we want to put something like this or whether we want to cannibalize it for parts.

Because we have not fully decided on #7 , I'm not sure where we are calling this from. I wanted to have something that could read data saved to file and get the info we want to display.

I find a few issues we might want to discuss:

1. If reading from file (which is convenient, and it's the way it's done here) we will have a significant lag (we are writing to csv every 100 rows, we could write more often too...let's discuss options)
2. What is the script that runs all other scripts? Should this go into that one or be ran as separate? There's a while loop but it

import pandas as pd
import numpy as np
import os
import datetime

def absolute_file_paths(directory):
    path = os.path.abspath(directory)
    return [entry.path for entry in os.scandir(path) if entry.is_file()]

def read_data(file_list):
    df_list = list()
    for filename in file_list:
        df_list.append(pd.read_csv(filename))
    return df_list

def process_data(df_list):
    # this is fake now for debugging
    now = datetime.datetime.strptime("2021-03-21 15:03:10", "%Y-%m-%d %H:%M:%S")
    # now = datetime.datetime.now()
    last_hour = now - datetime.timedelta(hours=1)
    #print(df_list)
    for df in df_list:
        df["datetime"] = pd.to_datetime(df['MM:DD:YYYY hh:mm:ss'])
        subset_df = df[(df.datetime >= last_hour) & (df.datetime <= now)]
        status = "active" # this should change when we get inputs from jammed devices
        # get all the things we want
        n_events = len(subset_df.index) 
        last_pellet_time = subset_df[(df.Event == "Pellet")].datetime.iloc[-1] 
        pellets_last_hour = len(subset_df[(df.Event == "Pellet")])
        session_pellets = subset_df[(df.Event == "Pellet")].tail(1).Pellet_Count.iloc[-1]
        out = [n_events, last_pellet_time, pellets_last_hour, session_pellets]
        print(out)
        #last_pellet = 

if __name__ == '__main__':
    file_list = absolute_file_paths("data")
    all_data = read_data(file_list)
    # something like this?
    #while True:
    #   process_data(all_data)
    #   time.sleep(5)
    # some sort of escape key?
    process_data(all_data)

With the sample data, this produces

[5, Timestamp('2021-03-21 14:24:48'), 5, 805]
[3, Timestamp('2021-03-21 14:54:40'), 3, 807]
[8, Timestamp('2021-03-21 15:02:42'), 8, 808]
[2, Timestamp('2021-03-21 15:02:55'), 2, 722]

This is in line with the minimal info that we would like to display (at least for now). Once we show this we have to add the FED device number and the status (whether it's active or jammed)

[matiasandina]

5 was related and closed due to 30091a6