2.0.06

[lux4rd0]

New Features and Enhancements

1. Device Data Caching and Concurrency Control:

   • Added Caching for Device Data: We introduced caching for device data with a configurable expiry time to reduce redundant API calls and improve performance.
     • Environment Variable: SENSE_COLLECTOR_DEVICE_CACHE_EXPIRY_SECONDS
     • Default Value: 120 seconds
   • Concurrent Device Lookups: This enhancement allows users to perform multiple device data lookups concurrently, up to a specified number of concurrent operations.
     • Environment Variable: SENSE_COLLECTOR_DEVICE_MAX_CONCURRENT_LOOKUPS
     • Default Value: 4
   • Lookup Delay Between Requests: To manage API rate limits effectively, we introduced a delay between consecutive device lookup requests.
     • Environment Variable: SENSE_COLLECTOR_DEVICE_LOOKUP_DELAY_SECONDS
     • Default Value: 0.5 seconds

2. Session Management:

   • Session Lifecycle Management: We have added methods to manage the lifecycle of aiohttp sessions, ensuring that sessions are properly opened and closed, which improves resource management and stability.

3. Always-On Functionality:

   • WebSocket Reconnection Logic: Implemented robust reconnection logic to maintain an "always-on" connection with the Sense API WebSocket feed. The application will now attempt to reconnect indefinitely with configurable intervals and backoff strategies.
     • Heartbeat Interval: A configurable interval sends heartbeat messages to keep the WebSocket connection alive.
     • Environment Variable: SENSE_COLLECTOR_WS_HEARTBEAT_INTERVAL
     • Default Value: 10 seconds
     • Heartbeat Timeout: Configurable timeout to wait for a response before considering the connection lost and attempting to reconnect.
     • Environment Variable: SENSE_COLLECTOR_WS_HEARTBEAT_TIMEOUT
     • Default Value: 30 seconds
     • Reconnection Strategy: Configurable initial delay, cap, and backoff factor for reconnection attempts.
     • Environment Variables:
       • SENSE_COLLECTOR_WS_RECONNECT_DELAY_INITIAL (Default: 5 seconds)
       • SENSE_COLLECTOR_WS_RECONNECT_DELAY_CAP (Default: 60 seconds)
       • SENSE_COLLECTOR_WS_BACKOFF_FACTOR (Default: 1)
       • SENSE_COLLECTOR_WS_RECONNECT_INTERVAL (Default: 840 seconds)

4. Improved Logging:

   • Detailed Logging: Enhanced logging throughout the application to provide more granular insights into the internal processes, including API request timings, cache hits/misses, and error handling.
   • Environment Variable Renamed: Changed SENSE_COLLECTOR_SENSE_API_RECEIVE_DATA_OUTPUT to SENSE_COLLECTOR_OUTPUT_RECEIVED_DATA for clarity.

5. API Worker Enhancements:

   • Improved API Worker Functionality: The API worker now includes more detailed logging, better error handling, and clearer tracking of the processing status for each device.

Functional Changes

1. Configuration Updates:

   • Updated Docker Configuration: Streamlined the Dockerfile to remove unnecessary packages and dependencies, resulting in a smaller and more efficient container image.
   • Removed Packages: python3-dev and gcc are no longer installed, as they are not needed for the current dependencies.

2. Environment Variables Management:

   • New Environment Variables: Introduced new environment variables for managing device data cache expiry, maximum concurrent lookups, and lookup delay.
   • Updated Environment Variables:
     • SENSE_COLLECTOR_OUTPUT_RECEIVED_DATA replaces the old SENSE_COLLECTOR_SENSE_API_RECEIVE_DATA_OUTPUT.

Breaking Changes

1. Environment Variable Changes:

   • Renamed Environment Variable: If you were using SENSE_COLLECTOR_SENSE_API_RECEIVE_DATA_OUTPUT, you must update your configuration to use SENSE_COLLECTOR_OUTPUT_RECEIVED_DATA.

2. Configuration and Deployment:

   • Docker Image: Ensure your deployment scripts are updated to use the new Docker image structure, as the installation process has changed.

Bug Fixes

1. Rate Limiting Handling: We improved the handling of rate-limiting responses from the Sense API to ensure more robust and resilient operation under heavy load.
2. Session Management: Issues with aiohttp session management were fixed to prevent potential resource leaks and ensure reliable connections.