[Feature]: Extend REST Server with Poller Control, Env Based Config, Mulitple-API Keys and Key Permissions

[challey74]

Suzieq version

0.22.0

Install Type

container

Feature type

sq-rest-server

Use case

For Me

• Connect a Netbox Plugin to SuzieQ to gather data for creating a device, with plans to extend to more SuzieQ functionality.
• Integrate config verification with existing scripts without having SuzieQ as a dependency

In General

• Adding functionality to create new pollers and stop running ones allows for ad-hoc polling, requiring less manual control of SuzieQ
• Allows all rest config variables to be configured as environment variables and/or in the yaml config.
  • This would allow for api keys to be injected at runtime and not stored in plain text.
• Multiple api keys and permissions for better security management especially when connecting several applications to SuzieQ

Proposed functionality/solution

1. Restructuring the Codebase

• Introduce new directories in restServer: routes, services, models and utils to organize the codebase better.
  • routes: Contains the endpoint definitions and request handling logic.
  • services: Implements the business logic and data access layer.
  • models: Contains the Pydantic models used for the endpoints
  • utils: Provides utility functions and helper modules.

2. Separating Business Logic from Endpoints

• Move the business logic and data access functionality from the endpoint handlers to separate service modules.
• Endpoint handlers in the routes directory should focus on request validation, authentication, and response formatting.
• Service modules in the services directory should encapsulate the core business logic, data retrieval, and manipulation.
• This separation of concerns improves code readability, reusability, and testability.

3. Configuration through Environment Variables and YAML File

• Introduce a configuration mechanism that allows settings to be defined either through environment variables or the standard YAML configuration file.
• Implement a configuration module in the utils directory to handle the loading of environment variables and parsing and validation of configuration values from YAML or ENVs.
• The configuration module will handle the precedence of environment variables over the YAML configuration file.

4. Multiple API Keys and Permission-Based Access Control

• Implement support for multiple API keys to enable fine-grained access control and authentication.
• Introduce a permission system that associates each API key with a set of permissions.
• Define the following permission levels for API keys:
  • read: Allows read access to endpoints.
  • write: Allows write access to endpoints.
  • delete: Allows deletion of resources through endpoints.
  • execute: Allows execution of specific actions or operations.
  • admin: Grants administrative privileges and unrestricted access to all endpoints.
• Modify the endpoint handlers to check the permissions associated with the provided API key before processing the request using a local authentication function.
• Return appropriate HTTP status codes and error messages for unauthorized or forbidden access attempts.
• Allow header Authorization: Bearer <api_key> for a more standard auth pattern and easier transition to OAuth/JWT if desired in the future.

5. New Pollers Route

• Create a new Pollers Route with the following functionality:
  • GET /pollers: Retrieve config and state of all pollers.
  • POST /pollers: Create a new poller with the specified configuration.
  • POST /poller: Create a new poller with the specified configuration.
  • GET /pollers/{id}: Retrieve config and state of the specified poller.
  • PATCH /pollers/{id}: Update a poller with the specified configuration.
  • DELETE /pollers/{id}: Stop and Delete a poller with the specified identifier.
  • POST /pollers/{id}/start: Start a poller with the specified identifier.
  • POST /pollers/{id}/stop: Stop a poller with the specified identifier.
• Implement the necessary authentication and authorization checks for each endpoint based on the API key permissions.
• Update the service layer to handle the business logic for managing pollers, including creation, deletion, starting, and stopping of pollers.
• Provide appropriate response codes and error handling for each endpoint.

External dependencies

None

Additional Context

• I have begun work on this already, but happy to make changes anywhere.
  • Currently, I have changes 1-4 implemented and working on the poller functionality.
• I'm not committed to the new sqPoller endpoint schema and am looking into the changes needed to pass an inventory directly to a new poller. This would remove the need to generate and remove a inventory file. Though, that functionality might be useful to some.
• I have no breaking changes and default behavior is the same. However, there are several areas where new rest configuration variables were needed for the desired behavior and code to migrate existing variables to the new (e.g. API_KEY to api_keys). These would probably want a deprecation strategy at some point.