Introducing a Long-Term Performance Monitoring Tool

With the increasing need to monitor long-term performance trends and assess their impact on the reliability and efficiency of the camera systems, I propose the introduction of a dedicated monitoring tool. This tool is designed to log key performance metrics and settings directly into a CSV file, with a new entry added after each run. This approach would enable us to identify potential issues related to settings, storage, or software that may affect machine performance. The CSV file is stored in the RMS root directory and included in the archive folder for easy access.

To obtain information not automatically collected from the code, two new fields have been added to the config file, allowing users to provide detailed descriptions of their storage drives.

The tool currently logs the following information:

data_dir_name: Name of the night data directory.
model: The Raspberry Pi model, if available.
os_version: Operating system version.
architecture: CPU architecture.
system_drive_desc: User-specified system drive description (e.g., 'SanDisk SD Extreme Pro A2').
system_drive_speed: Write speed of the system drive, tested with 100 MB after a 5-second delay.
data_drive_desc: User-specified data drive description, if different.
data_drive_speed: Write speed of the data drive.
res: Camera vertical resolution (e.g., 720).
calc_fps: GST-calculated FPS at the end of a run, if available.
media_backend: Configured media_backend setting.
media_backend_ovr: Indicates if the media_backend setting was overridden by the code.
live_maxpixel: Config live_maxpixel setting, as it may exert additional load on the system.
live_jpg: Config live_jpg setting, as it may exert additional load on the system.
slideshow: Config slideshow setting, as it may exert additional load on the system.
hdu_compress: Config hdu_compress setting, to assess the impact of compressing FF files.
fireball_detection: Config fireball_detection setting, as it may exert additional load on the system.
jitter_quality: Reported jitter quality.
dropped_frame_rate: Reported dropped frame rate.
total_gb: Total capacity of the data drive.
used_gb: Used capacity of the data drive.
free_gb: Free capacity of the data drive.

I welcome your thoughts on this initiative. Is this something we're interested in pursuing? Are there any additional metrics or data points you believe should be included?

I've attached a sample log file. In the future, it could be used to generate graphs showing long term trends if desired. XX0001_perfMonitorLog.csv

Cheers, Luc

CroatianMeteorNetwork / RMS

Introducing a Long-Term Performance Monitoring Tool #283