Mapper parsing exceptions with date fields

[rmundel]

Software	Version
PHP	7.4.15
Elasticsearch	7.10.2
Laravel	6.20.16

Hey, first of all, awesome work!

I'm playing around with the new elastic packages and I'm getting some issues trying to parse date fields with the follow config:

$mapping->date('created_at', [
          'format' => 'strict_date_optional_time_nanos',
]);

It reports a successfull job but debugging the underlying elastic request gives me the following output:

[2021-02-15 12:36:37] production.DEBUG: {"took":1,"errors":true,"items":[{"index":{"_index":"anoreg_mt_vendas","_type":"_doc","_id":"19","status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse field [created_at] of type [date] in document with id '19'. Preview of field's value: '2020-11-16 17:07:36'","caused_by":{"type":"illegal_argument_exception","reason":"failed to parse date field [2020-11-16 17:07:36] with format [strict_date_optional_time_nanos]","caused_by":{"type":"date_time_parse_exception","reason":"Text '2020-11-16 17:07:36' could not be parsed, unparsed text found at index 10"}}}}}]}  

[rmundel]

I think the default config for the date mapping could be 'format' => 'yyyy-MM-dd HH:mm:ss'. Also, there is a way to report the elastic errors so that the Scout job fails?

[babenkoivan]

Hey @rmundel, sorry for the late reply.

You should define the date format when creating an index. Currently, not all errors returned by ES result in an exception, I'm investigating it.

[rmundel]

Hey man! Thanks for the work btw. So, after investigating a bit, I added some code inside the ES package and found out that indeed it was not returning exceptions, but the index was rejecting the data because ou some fields validations (I was sending the wrong date format). I think that the job should fail right?

I lost the errors log but was somthing like this:

Success log example:

[2021-02-19 10:17:06] production.DEBUG: {"took":0,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}}  

Failure log example:

[2021-02-19 10:17:06] production.DEBUG: {"took":0,"timed_out":false,"_shards":{"total":1,"successful":0,"skipped":0,"failed":1},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}} 

I think this issue should be in the scout driver repo right?

Thanks.

[rmundel]

Maybe the ES Scout Driver could read the ES response and fail the job?

[babenkoivan]

@rmundel, definitely. If there is an error, the document manager should throw an exception. Unfortunately, I didn't have time yet to look into it.

[babenkoivan]

@rmundel I've created an issue in the official client repository, let's see how it goes. If it's not resolved in the client library, then I will add extra handling on the adapter level.

[rmundel]

Hey @babenkoivan, no worries! Thanks for the effort.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days