pblvrt
commented
9 months ago Project Task: Creation of Importer Service
Objective:
Develop a standalone Importer Service that abstracts the process of importing data into our database through an API. This service is required to efficiently handle import errors, execute on a scheduled basis, and support asynchronous operations for extended tasks.
Dependencies:
-
253
Key Features and Requirements:
-
Error Handling Mechanism:
- Design the service to gracefully handle and log various import errors.
- Implement mechanisms to retry failed imports a configurable number of times before flagging them as errors.
- Provide clear error messages and notifications for issues that require manual intervention or immediate attention.
-
Cron Job Configuration:
- Set up the service to run as a cron job, ensuring automated, regular data imports.
- Allow flexibility in configuring the cron job's frequency and timing to meet different operational requirements.
-
Asynchronous Processing:
- Enable asynchronous execution to handle long-running import processes without blocking other operations.
- Implement a queueing system or similar mechanism to manage and process data imports efficiently.
-
API Integration:
- Ensure seamless integration with our existing API for pushing data into the database.
- The service should be capable of handling different data formats and structures as required by the API.
Eric-Vondee
Is your feature request related to a problem? Please describe. The schedule import is currently triggered from code. I would propose we add an action on the Admin UI so anyone with event/admin access can run an import.