orcasound / aifororcas-livesystem

Real-time AI-assisted killer whale notification system (model and moderator portal) :star:
http://orcahello.ai4orcas.net/
MIT License
35 stars 24 forks source link

incomplete data problem #21

Open dbaing17 opened 3 years ago

dbaing17 commented 3 years ago

The system normally processes one minute at a time. However, sometimes data are missing, so that only 40-50 seconds are received. When this happens, the bars indicating when detections occurred are misaligned. This is a rare problem and has little impact on the function of the system, so is a low priority from my perspective, but could be important for retraining runs.

scottveirs commented 3 years ago

It may be helpful to not the time of day when these shorter-than-expected candidates occur. One possible cause may be the re-starting of the containers on the Raspberry Pis. This currently happens every 6 hours on the 3 orcasound nodes (e.g. 00:30, 06:30, 12:30, 18:30). However, the orcanode roadmap includes (a) moving to 24 hour re-starts or (b) avoiding restarts by incorporating a time stamp in the filename for each HLS/*.ts data segment. These anticipated changes (likely during fall 2021) might also suggest this may be a relatively low priority issue.