pdphilip / laravel-elasticsearch

Laravel Elasticsearch: An Elasticsearch implementation of Laravel's Eloquent ORM
MIT License
86 stars 16 forks source link

[Implemented] firstOrCreateWithOutRefresh or similar to improve speed #19

Closed abkrim closed 5 months ago

abkrim commented 5 months ago

Is your feature request related to a problem? Please describe. I use your package in an intensive system. After create a code to get for one index and migrate to another (local) based on docker I see a big problem with Model::firstOrCreate()

First I thought it was the fault of the previous query to find out if the doc existed, and then creating it if it did not exist, was the cause.

But after doing a better job in the docker via no.

The time difference between using Model::createWithoutRefresh($doc); and Model::firstOrCreate() was overwhelming.

So I tried another approach.

$exists = MOdel::where('log_id', $doc['log_id'])->exists();
if (! $exists) {
   WorkAnalyzer::createWithoutRefresh($doc);
}

And voila. Fly again.

I've reviewed the code to see if I could get around it, creating or extending it if necessary, to create something like firstOrCreateWithoutRefresh(), but I feel unable. So I gave up the idea.

And even with a little conversation, if you could advise me, I could try to create the PR, with the tests on the object.

Once again delighted with your package.

pdphilip commented 5 months ago

Hey @abkrim, to make sure I understand: You managed to solve the problem with the snippet you shared, but would like it to be a feature firstOrCreateWithoutRefresh() in the package?

abkrim commented 5 months ago

Yes.

I think is a good feature.

Basically. Run Model::firstOrCreate() need 1 hour in 1000 docs, and if use workaround (first where, and after createWithoutRefresh) get only 20 seconds.

pdphilip commented 5 months ago

Noted. Thanks for the suggestion.

If I implment this then it will be in the next major (3.x) realease which I'm in the middle of building.

Will provide feedback here of the outcome.

abkrim commented 5 months ago

Looking forward to seeing it, and thus seeing the code to see how it was done. I'm learning a lot with your code.

Thank you very much for your work.

pdphilip commented 5 months ago

@abkrim - this feature has been added in v3, docs

Check it out!