[Fixed] A possible issue preventing the package from working with Filament?

[abkrim]

Package version

versions : * v3.10.0

Package implicate

filament/filament: v3.2.62 (latest)

Describe the bug

A possible issue preventing the package from working with Filament

I wanted to create a resource for my model, and to my surprise, when I went to access the resource (List), I encountered the error

Exception
PHP 8.2.16

10.48.4
SSL CA bundle not found: ssl/http_ca_local.crt

Upon reviewing, the first test I performed was to go to tinker to check the status of Elastic's installation and its interaction with the package

> $wa = WorkAnalyzer::orderBy('datetime', 'asc')->first()
= App\Models\WorkAnalyzer {#10033
    _id: "qjPMSY4Bh1MJNmfqLuZ3",
    log_id: "01HQGZBVRAW7YQZXK4DGG0KB0V",
    datetime: 1708889203,
    provider: "prilux",
    powrea_var: 2160.0,
    pa1_w: 0.0,
    pa2_w: 6740.0,
    pa3_w: 8720.0,
    pr1_var: 500.0,
    pr2_var: 960.0,
    pr3_var: 700.0,
    iac1_a: 40760.0,
    iac2_a: 28720.0,
    iac3_a: 37300.0,
    vac1_v: 240.9,
    vac2_v: 240.0,
    vac3_v: 237.1,
    pf1: 0.98,
    pf2: 0.97,
    pf3: 0.98,
    status_code: 200,
    modem_id: 3323,
    timestamps: [
      "job" => "2024-02-25T20:30:06+00:00",
      "delayed_at" => null,
    ],
  }

> $wa->datetime;
= "2024-02-25 19:26:43"

> WorkAnalyzer::count()
= 1698367

Exception

SSL CA bundle not found: ssl/http_ca_local.crt (View: /Users/abkrim/Sites/sitelight/vendor/filament/tables/resources/views/index.blade.php) (View: /Users/abkrim/Sites/sitelight/vendor/filament/tables/resources/views/index.blade.php)
vendor/pdphilip/elasticsearch/src/DSL/Bridge.php#579


It struck me as odd to see the file where the error was encountered,

public function _countAggregate($wheres, $options, $columns): Results
{
    try {
        $params = $this->buildParams($this->index, $wheres);
        $process = $this->client->count($params);

        return $this->_return($process['count'] ?? 0, $process, $params, $this->_queryTag(__FUNCTION__));
    } catch (Exception $e) {

        $result = $this->_returnError($e->getMessage(), $e->getCode(), [], $this->_queryTag(__FUNCTION__));
        throw new Exception($result->errorMessage);
    }

}

He buscado algo mas sobre el tema, peor no localizo nada. <

Best regards.

Screenshots:

Component Versions (Paste in the require section from your composer.json file):

  "require": {
  "php": "^8.1",
        "ext-pdo": "*",
        "ext-redis": "*",
        "ext-zlib": "*",
        "archtechx/enums": "^0.3.0",
        "babenkoivan/elastic-scout-driver": "^3.0",
        "babenkoivan/elastic-scout-driver-plus": "^4.2",
        "barryvdh/laravel-debugbar": "^3.13",
        "barryvdh/laravel-dompdf": "^2.0",
        "binarytorch/larecipe": "^2.5",
        "codeat3/blade-google-material-design-icons": "^1.18",
        "codeat3/blade-iconpark": "^1.5",
        "codeat3/blade-jam-icons": "^1.5",
        "dompdf/dompdf": "^2.0",
        "elasticsearch/elasticsearch": "^8.10",
        "filament/filament": "^3.0-stable",
        "geokit/geokit": "^1.3",
        "guzzlehttp/guzzle": "^7.2",
        "laravel-notification-channels/telegram": "^4.0",
        "laravel/framework": "^10.0",
        "laravel/horizon": "^5.17",
        "laravel/pail": "^1.0",
        "laravel/prompts": "^0.1.13",
        "laravel/sanctum": "^3.2",
        "laravel/scout": "^9.8",
        "laravel/tinker": "^2.7",
        "leandrocfe/filament-apex-charts": "^3.1",
        "livewire/livewire": "^3.0",
        "lorisleiva/laravel-actions": "^2.4",
        "maatwebsite/excel": "^3.1",
        "matanyadaev/laravel-eloquent-spatial": "^3.1",
        "novadaemon/filament-pretty-json": "^2.0",
        "owen-it/laravel-auditing": "^13.3",
        "pdphilip/elasticsearch": "~3.10",
        "predis/predis": "^2.2",
        "ramsey/uuid": "^4.7",
        "robinvdvleuten/ulid": "^5.0",
        "saade/filament-laravel-log": "^3.0",
        "silber/bouncer": "^1.0",
        "spatie/laravel-backup": "^8.4",
        "spatie/laravel-data": "^2.1",
        "spatie/laravel-health": "^1.27",
        "spatie/laravel-query-builder": "^5.1",
        "spatie/laravel-rate-limited-job-middleware": "^2.2",
        "spatie/laravel-ray": "^1.35",
        "spatie/laravel-settings": "^2.8",
        "spatie/laravel-translatable": "^6.1",
        "spatie/simple-excel": "^3.0",
        "vlucas/phpdotenv": "^5.4"
  },

Best regards.

Edit for more information

If I use in ES_SSL_CERT a fullpath works.

Well, the certificate problem passes but others appear.

And I have verified that it worked with the complete path both in the tinker and in my already-created commands.

But I think that is a problem that is not defined because, in principle, everything I have at the configuration level uses a path relative to the path of the app.

[abkrim]

Relative to second problem

By taking as a solution to enter the full path, the action is executed but, I understand that during it, there are a series of jobs from the package, which use the _id field (aggregations?)

As I understand the use of _id for these issues, it was deprecated in version 7.6 and disabled by default in versions 8.X:

{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled"
            }
        ],
        "type": "search_phase_execution_exception",
        "reason": "all shards failed",
        "phase": "query",
        "grouped": true,
        "failed_shards": [
            {
                "shard": 0,
                "index": "analyzers-2022111701",
                "node": "dTTpCTBqRbSYbWfW4vyFUg",
                "reason": {
                    "type": "illegal_argument_exception",
                    "reason": "Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled"
                }
            }
        ],
        "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled",
            "caused_by": {
                "type": "illegal_argument_exception",
                "reason": "Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled"
            }
        }
    },
    "status": 400
}

When I wanted to activate it, I found literature on the issue, in the changes to version 8.0 of Elasticsearch.

Migrating to 8.0

Aggregating and sorting on _id is disallowed by default. Details
>Previously, it was possible to aggregate and sort on the built-in _id field by loading an expensive data structure called fielddata. This was deprecated in 7.6 and is now disallowed by default in 8.0. Impact
>Aggregating and sorting on _id should be avoided. As an alternative, the _id field’s contents can be duplicated into another field with docvalues enabled (note that this does not apply to auto->generated IDs).




[pdphilip]

For the cert issue, this would be environment-related. This value is passed directly to the elasticsearch PHP client. If you want to, edit the config in the database.php file to use the apps root for example.

I'm not familiar with FIlament, but that error is usually thrown when you try to chunk (which uses the _id) for sorting by default. See: https://elasticsearch.pdphilip.com/chunking

chunkById is the alternative.

Can you check further don't your stack trace to see the query that it was trying to run?

[abkrim]

Thank you so much.

For the strange question of the certificate path, did I opt for 'ssl_cert' => env('ES_SSL_CERT')? base_path(env('ES_SSL_CERT')) : null, and it works perfectly across the board.

For the other, my apologies because I can't remember the chunking issue.

And if the real reason is how Filament works the construction of the tables, in which it works on a scan that, from what I see in its code, the use of Model::chunkById is quite unacceptable:

Even so, I have written on their Discord channel and I hope to make some progress. Otherwise, I will have to create my own table with Livewire directly.

Also, I have tried to use protected $primaryKey = 'log_id'; but not work. log_id is my internal ulid field in indexes of Elasticsearch.

Thank you very much for your answer.

[abkrim]

Hello. Nothing. I investigated the matter and it is not a Filament issue itself but rather a Livewire issue.

The implication is harsh, since I did not find a way, since the chunkById and chunk methods do not work for mounting the Livewire (or Filament) instance.

So with both left out, the only thing left is to work with Blade and add certain functionalities with Alpine.js

Thanks for everything.

[pdphilip]

Thanks for the update. Can you give me a simple way to replicate this breakage using livewire?

Let's see if there's a way forward.

[abkrim]

Your help appreciated.

I continued investing and I did not see the error, because I am untrained with Livewire.

And the truth is I'm surprised since one thing is the Filament system and another is Liviwire, where you put together the code with what you need.

So effectively in Livewire, you can perfectly create a page with the list, widgets for searches, etc.

Route::get('/work-analyzers', App\Livewire\WorkAnalyzers\ListWorkAnalyzers::class);

<?php

namespace App\Livewire\WorkAnalyzers;

use App\Models\WorkAnalyzer;
use Illuminate\Contracts\View\View;
use Livewire\Component;
use Livewire\WithPagination;

class ListWorkAnalyzers extends Component
{
    use WithPagination;

    public function render(): View
    {
        return view('livewire.work-analyzers.list-work-analyzers', [
            'workAnalyzers' => WorkAnalyzer::paginate(10),
        ]);
    }
}

<div>
    @foreach ($workAnalyzers as $workAnalyzer)
        <div wire:key="{{ $workAnalyzer->lod_id }}">
            <p>{{ $workAnalyzer->_id }}</p>
        </div>
    @endforeach

    <div>
        {{ $workAnalyzers->links() }}
    </div>
</div>

I found something but no matter how much I turned it over, it didn't work. But it nevertheless encourages me to think that I can click on Filament to see if it includes the facility to integrate your package. It would be great, since RAd with Filamente is brutal, compared to having to deploy tables and forms with Livewire dry.

Implement en Filament Laravel Socut

Best regards and happy weekend

[pdphilip]

@abkrim, I've reworked the chunkById method to work independently of the _id field. Currently on Dev branch: https://github.com/pdphilip/laravel-elasticsearch/tree/dev

Please can you try this dev version and let me know if that solved this issue. Once confirmed it will be in the next release.

Thank you

[abkrim]

Oh my God.

This afternoon I will update my installation to test it. Sure.

A lot of thanks in advance.

[abkrim]

Hello.

I opened a new project to leave only the theme in a row with the model resource that uses one of the indexes, and not have noise around it.

But I don't know if I have to do anything else.

I tried

• Default model
• Model indicating that my first field is a different log_id, which is an internal ulid that I generate in each doc when I send it.
• I have not been able to apply the tip that exists for Laravel Scout because I cannot find the proposed methods.

❯ composer info pdfphilip/elasticsearch
name : pdfphilip/elasticsearch
description : An Elasticsearch implementation of Laravel's Eloquent ORM
keywords: database, elastic, elasticsearch, eloquent, laravel, model
versions: * dev-dev
type: library
license : MIT License (MIT) (OSI approved) https://spdx.org/licenses/MIT.html#licenseText
homepage: https://github.com/pdphilip/laravel-elasticsearch
source: []

I don't know if this information will contribute anything.

Best regards.

[pdphilip]

Hey @abkrim, were you not able to pull in the dev version, or is it that the problem is still the same even with the dev version?

[abkrim]

Hello @pdphilip

Yes I use a package based on laravel-elasticsearch\dev

And with this package used in a fresh Laravel 11 with only models extended eloquent with your package, get the same problem.

400 Bad Request: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting:

protected function _returnSearch($params, $source)

    {

        if (empty($params['size'])) {

            $params['size'] = $this->maxSize;

        }

        try {

            $process = $this->client->search($params);

            return $this->_sanitizeSearchResponse($process, $params, $this->_queryTag($source));

        } catch (Exception $e) {

            $result = $this->_returnError($e->getMessage(), $e->getCode(), $params, $this->_queryTag(__FUNCTION__));

            throw new Exception($result->errorMessage); // Here line

        }

    }

I'm sorry if I don't express myself correctly.

[pdphilip]

Ok, so it's not chunking. Can you look further down the trace to see what the eloquent query was that was being called? See if you're able to get the parameters as well.

My guess is that it's trying to order by _id somewhere...

[pdphilip]

I'm not able to recreate this in my testing with Filament - using ES models seems to work fine for me with Laravel 11.

Can you share a private repo with me so I can see exactly what you're seeing?

[abkrim]

I'm sorry for not having responded sooner.

I have been sick, and I couldn't even get close to the team.

Since I cannot give private access to the project since it is not mine, and my client is very strict with the security of the project, I have opted for a clone, in which I only reproduce one of the indexes.

However, when I updated the dev today, I saw that there are new features and these do not seem to be in the documentation.

Since I wanted to adapt the repository to the problem, I created a seeder to fill the model a little in order to see the problem.

But my surprise is that the seeder throws me an exception:

a db:seed --class WorkAnalyzerSeeder

   INFO  Seeding database.  

   TypeError 

  PDPhilip\Elasticsearch\Connection::getIndexPrefix(): Return value must be of type string, null returned

  at /Users/abkrim/Sites/packages/laravel-elasticsearch/src/Connection.php:42
     38▕     }
     39▕     
     40▕     public function getIndexPrefix(): string
     41▕     {
  ➜  42▕         return $this->indexPrefix;
     43▕     }
     44▕     
     45▕     public function setIndexPrefix($newPrefix): void
     46▕     {

  1   /Users/abkrim/Sites/packages/laravel-elasticsearch/src/DSL/Bridge.php:42
      PDPhilip\Elasticsearch\Connection::getIndexPrefix()

  2   /Users/abkrim/Sites/packages/laravel-elasticsearch/src/Connection.php:229
      PDPhilip\Elasticsearch\DSL\Bridge::__construct(Object(PDPhilip\Elasticsearch\Connection))

I put this a public gitlab report in my own server. https://gitlab.castris.com/root/sitelight_v11

[pdphilip]

Have updated dev branch to allow null prefixes, or just edit the vendor directly for testing: https://github.com/pdphilip/laravel-elasticsearch/commit/3577bb31b766dee28f3148c6b2206bd8fd65ced8

Had a quick look at your sample repo, seems ok:

Am I missing something?

[abkrim]

Oh... my God.

I've updated dev with changes and same problem.

400 Bad Request: all shards failed - Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled

Run seeder (I updated for use createWithoutRefresh)

Go to https://sitelight_v11.test/admin/work-analyzers and get error.

400 Bad Request: all shards failed - Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled

It is possible that in your test Elasticsearch has Fielddata access on the _id field allowed?

Best regards.

[pdphilip]

My bad! Try now, latest on dev

[abkrim]

Oh... I feel happy.

Incredible your work.

Now works fine.

Your package and your work deserve a lot of stars.

Many thanks.

[pdphilip]

Thanks @abkrim, these are great upgrades for the package - glad you brought it to my attention.

[pdphilip]

Hey @abkrim, the latest release has this fixed

• Laravel 10/11 v4.0.1
• Laravel 9 v3.9.1

Thanks again