Improve monitoring capability of Integration service

[himeshr]

To improve monitoring capability of Integration service:

• [x] 0. write tests to create a suite to ensure proper classification of errors
• [x] 1. Segregate and stop only specific task group
• [x] 2. Let the task stop if there are uncaught errors
• [x] 3. Build logic for processing error and segregating between Business errors and other errors
• [x] 4. Store Business ErrorType with matching Exception msg, to determine if an exception is not a business error
• [x] 5. Create separate bugsnag
• [x] 6. Create internal group for avni-int
• [x] 7. Store error message in ErrorRecordLog
• [x] 8. Show error message in int-web-app, on clicking Edit button with proper fields keys
• [x] 9. Use metabase for alerts for new errors occurred since lastCutOffTime, schedule periodic trigger
• [x] 10. Show correct date time string in int-web-app error records screen
• [x] 11. Filter on integration system

[petmongrels]

@himeshr @vinayvenu I think we have mis implemented the error handling in Goonj. We have blanket catch exception which will also catch any code issues. This means that the integration will complete even if there is a coding issue, network issue. For these errors we have to stop and not process further as their is not point in processing records if the systems are unavailable or code has a bug. ErrorRecords are useful only if there are known application scenarios.

This way we will create too many errors and use the error log as Bugsnag. Lets discuss on Monday.

[himeshr]

Current status of tasks in-progress:

1. Code changes are in draft for error-msg storing in error_record_log table (Unable to upload patch here, ping me if needed)
2. Draft Query created in Metabase, check https://reporting.avniproject.org/collection/449-avni-integration

[himeshr]

Code changes have been deployed to staging and prod integration servers.