Closed abkrim closed 7 months ago
Have run your tests and these seem to fall into elasticsearch quirks. The package just makes the DSL queries but you will still need to pass in the correctly typed values and make sure you save the correct values.
If set ignore_malformed
to true
then ES will save whatever value you give it, but you may not be able to query it since it will effectively be unmapped. I would avoid that to ensure you have the correct format.
For timestamps, you'll need to pass it in as a string or in milliseconds if you want to use int, I don't know why, but you can test directly on ES:
Given your mapping - the below queries all work:
$test = [];
$test[] = ElasticTest::where('custom_date', '>=', (string)$ts)->count();
$test[] = ElasticTest::where('custom_date', '>=', $ts * 1000)->count();
$test[] = ElasticTest::where('custom_date', '>=', Carbon::parse($ts)->format('Y-m-d H:i:s'))->count();
$test[] = ElasticTest::where('default_date', '>=', Carbon::parse($ts))->count();
$test[] = ElasticTest::where('default_date', '>=', $ts * 1000)->count();
$test[] = ElasticTest::where('default_date_forced', '>=', Carbon::parse($ts))->count();
$test[] = ElasticTest::where('default_date_forced', '>=', Carbon::parse($ts)->getTimestampMs())->count();
$test[] = ElasticTest::where('default_date_forced', '>=', $ts * 1000)->count();
$test[] = ElasticTest::where('epoch_date', '>=', $ts * 1000)->count();
$test[] = ElasticTest::where('epoch_date', '>=', Carbon::parse($ts)->getTimestampMs())->count();
$test[] = ElasticTest::where('epoch_date', '>=', (string)$ts)->count();
I would just stick to saving date-time directly with Carbon, and query using Cabon. It's consistent, works with timezones and easy to work with.
I can't find the bug within this package - just weird Elasticsearch formatting. Did I miss something?
Hello Much appreciate your time.
The truth is that the topic is a real headache, especially when it comes to indices with a huge amount of data.
In the next remapping that I am preparing, after consulting the endpoints and the other developers on how they are using JS and the API, I consult them and see that everyone uses timestamp and that it is only the project manager who needs accessibility via eloquent or ISO type formats, I will do what you say in your manual.
Use the default value, since the ignore_malformed
is already outdated from two years ago because a method was put in place to clean the logs with malformed data, so there is no data with malformed data.
This change, given the implication and the fact that it is about production, will force me to do many tests, which I hope pass properly.
Cuando termine, añadiré aqui, mis resultados e impresiones
Once again, my greatest thanks and gratitude for such a productive package and your time.
Thanks @abkrim, your feedback is great and useful for developing a more robust package.
I'm going to make this an enhancement issue and convert whereDate()
to work with carbon input (and validate accordingly), then add a new method whereTimestamp()
whereby I can intercept the input and ensure the correct type casting. I'll keep this open until this feature has been published. Feel free to comment further here until then.
Hi @abkrim, new releases have been published that include whereTimestamp()
Thanks
Package version
ex: v3.10.0 Elasticsearch: 8.12.1
Describe the bug
After some issues with queries with date times, create a test seeder to see the question.
To Reproduce
Steps to reproduce the behavior:
Create a index for test purposes
Create Model
Create factory
Create seeder
Run migration
Check index mapping
Run migrations
Expected behavior
Working pass
default_date and default_date_forced acceptable value timestamp
They both fail. Show all instead of half.
epoch_date fails with timestamp if not casting to string
Screenshots: If applicable, add screenshots to help explain your problem.
Component Versions (Paste in the
require
section from your composer.json file):Additional context:
I don't know if I'm right, but well, I preferred to spend some time, after the last re-examination, being asked by my team about the methodology, since it was necessary:
And certainly after reindexing 100 million docs, I found that I had not done enough testing)
Acknowledgments