Enhance Error Handling with Custom Error Codes, Defined Failure Points and Logging

[nicolemah99]

Summary: Refactor the existing scripts to use custom error codes consistently and provide detailed logging messages. This will ensure that failures are handled gracefully and clearly, improving the robustness, maintainability, and debuggability of the scripts.

Objectives:

1. Define and Use Custom Error Codes:

   • Expand the set of custom error codes to cover all potential failure scenarios.
   • Ensure each script uses these error codes consistently for clear and actionable error messages.

2. Identify Critical Failure Points:

   • Determine where the scripts should fail immediately upon encountering errors.
   • Implement consistent failure handling at these critical points.

3. Implement Detailed Logging:

   • Enhance logging to include detailed messages with error codes.
   • Provide clear context for each error to aid in troubleshooting and diagnosis.

Proposed Solution:

1. Expand Custom Error Codes:
   • Define new error codes for scenarios not currently covered, such as:
     • Connection timeouts
     • lftp/rsyn failures
     • Data validation errors
     • File system errors (e.g., permissions issues, missing files)
     • Script dependency failures
2. Map Error Codes to Failure Points:
   • Create a mapping of error codes to specific failure points in each script. For example:
     • FTP Connection Issues: EXIT_CONN_FAIL for connection failures.
     • Download Failures: EXIT_DOWNLOAD_FAIL for issues during file download.
     • Validation Failures: EXIT_VALIDATION_FAIL for data validation errors.
     • Metadata Generation Issues: EXIT_METADATA_FAIL for failures in generating metadata.
     • Archiving/Compression Errors: EXIT_ZIP_FAIL for zipping issues.
     • General Script Errors: EXIT_UNKNOWN for unexpected issues not otherwise categorized.
3. Enhance Logging for Errors:
   • Update the fail function to log the error code and a detailed message.
   • Ensure each script logs the context of errors (e.g., which file, event, or step failed).

Example Config:

# Expanded Exit Codes
EXIT_SUCCESS=0                # Successful completion
EXIT_INVALID_ARGS=1001        # Invalid arguments provided
EXIT_CONN_FAIL=1002           # Connection failure to external services
EXIT_DOWNLOAD_FAIL=1003       # File download failure
EXIT_VALIDATION_FAIL=1004     # Data validation failure
EXIT_METADATA_FAIL=1005       # Metadata generation failure
EXIT_ZIP_FAIL=1006            # Compression/zipping failure
EXIT_PERMISSION_FAIL=1007     # File system permission issues
EXIT_MISSING_FILE=1008        # Required file not found
EXIT_DEPENDENCY_FAIL=1009     # Dependency script or service failure
EXIT_UNKNOWN=1099             # Unknown error

# Export exit codes for use in other scripts
export EXIT_SUCCESS
export EXIT_INVALID_ARGS
export EXIT_CONN_FAIL
export EXIT_DOWNLOAD_FAIL
export EXIT_VALIDATION_FAIL
export EXIT_METADATA_FAIL
export EXIT_ZIP_FAIL
export EXIT_PERMISSION_FAIL
export EXIT_MISSING_FILE
export EXIT_DEPENDENCY_FAIL
export EXIT_UNKNOWN

Tasks:

• [ ] Define and document a comprehensive set of custom error codes in commons.sh.
• [ ] Identify and map critical failure points in each script to these error codes.
• [ ] Update each script to use the fail function with appropriate error codes and detailed messages.
• [ ] Enhance the logging mechanism to include error codes and detailed messages.
• [ ] Test the updated scripts to ensure they handle errors as expected and provide clear feedback.
• [ ] Update documentation to include information on the cleanup process and configuration.

[nicolemah99]

Critical Failure Points

• Invalid Config/critial values
• Test Connection
• Download
• File Validation
• Zipping Failure
• rsync failure

If any of these fail, the script should log and attempt to move on to next meter/set of directories. If some values for meters and directories are missing, the script can move to the next instead of failing.