spencerdcarlson
commented
4 years ago If we attempt to post to ES and encounter an error here is an example of the logged output:
2020/03/28 15:28:48 ERROR: executionId: 1 response has errors
2020/03/28 15:28:48 ERROR: executionId: 1, time: 75 ms
2020/03/28 15:28:48 ERROR: executionId: 1, itemResponse: &{Index:pg-2020-03-28 Type:pglog Id:iT8MI3EBrPk_9pZ7GpTa Version:0 Status:400 Result: ForcedRefresh:false Found:false Error:0xc0000fcd20 GetResult:<nil>}
2020/03/28 15:28:48 ERROR: executionId: 1, itemResponse.Error: &{Type:illegal_argument_exception Reason:mapper [detail] of different type, current_type [text], merged_type [ObjectMapper] ResourceType: ResourceId: Index: Phase: Grouped:false CausedBy:map[] RootCause:[] FailedShards:[]}
2020/03/28 15:28:48 executionId: 1, request: {"index":{"_index":"pg-2020-03-28","_type":"pglog"}}
{"@timestamp":"2017-10-17T15:50:35.840+0000","application_name":"SomeApp","avg_duration_ms":0.051,"code_action":"search","code_application":"Rails","code_controller":"users","code_line":"/usr/local/rvm/rubies/ruby-2.3.7/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'","code_source":[{"action":"search","application":"Rails","controller":"users","line":"/usr/local/rvm/rubies/ruby-2.3.7/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'"}],"command_tag":"SELECT","database_name":"sampledb","day_of_week":"Tuesday","day_of_week_int":2,"detail":[{},""],"error_severity":"LOG","max_duration_ms":0.051,"min_duration_ms":0.051,"prepared":"\u003cunnamed\u003e","prepared_step":"execute","process_id":72320,"query":"select * from servers where id IN ('1', '2', '3') and name = 'localhost'","redis_key":"postgres_test","remote_host":"127.0.0.1","session_id":"59e4d54b.11a80","session_line_num":18,"session_start_time":"2017-10-17T15:50:35+0000","total_count":1,"total_duration_ms":0.051,"unique_sha":"efd3e221599dbd2c9a2594d23fa837417daa62e5","unique_string":"select * from servers where id in ($1, $2, $3) and name = $4","user_name":"new_samplepayload","virtual_transaction_id":"2/48859"}
Added some ES functions:
Improved testing:
Looks like we can give the bulk processor a callback function, maybe we can log out failed messages there - BulkProcessorService.AfterI implemented the BulkProcessorService.After and log out errors when we fail to publish to elastic search.