pdphilip / laravel-elasticsearch

Laravel Elasticsearch: An Elasticsearch implementation of Laravel's Eloquent ORM
https://elasticsearch.pdphilip.com/
MIT License
94 stars 17 forks source link

[HELP] Strange behavior in tests 401 Unauthorized: unable to authenticate user [elastic] for REST request [/test-analyzers/_doc] #51

Closed abkrim closed 2 days ago

abkrim commented 2 days ago

Package version

v4.5.0

Describe the bug

I'm very happy with the package, but I'm refactoring some things I had using the original Elasticsearch library, and when refactoring to use model X, the problem comes from the tests.

When I try to create a doc I get an authentication failure on my local test node, telling me that the user does not exist

Errors with create or createWithoutRefresh

401 Unauthorized: unable to authenticate user [elastic] for REST request [/test-analyzers/_doc?refresh=wait_for] - unable to authenticate user [elastic] for REST request [/test-analyzers/_doc?refresh=wait_for]

401 Unauthorized: unable to authenticate user [elastic] for REST request [/test-analyzers/_doc] - unable to authenticate user [elastic] for REST request [/test-analyzers/_doc]

To Reproduce

Steps to reproduce the behavior:

Test

test('response fails with failed via server error', function () {
    Bus::fake();

    $indice = 'analyzers';
    $this->refreshTestingIndex($indice); // This delete test-analyzers and recreate index test-analyzers. Is a indice with strict mapping

    ray(config('database.connections.elasticsearch'))
       ->green(); // Debug for verify connection in test mode and show that elasticsearch connection used in model is a local
                         // with user name and password correct

    $manufacturer = Manufacturer::factory()->nameSolar()->create();

    $commandCenter = CommandCenter::factory()->create([
        'manufacturer_id' => $manufacturer->id,
    ]);

    $statusCode = '500';
    $job_time = Carbon::now();
    $delayed_at = Carbon::now()->addMinutes(2);
    $ulid = generateUlid();

    $stubData = json_decode(file_get_contents(base_path('tests/stubs/responses/analyzer_400.json')), true);

    $stubData['data']['datetime'] = now()->timestamp;

    Http::fake(['*' => Http::response($stubData, $statusCode)]);

    $call = new CallAnalyzerAction;

    $call->handle($ulid, $commandCenter, $job_time, $delayed_at);
});

Code below is question of test. Call to external API get a falied() response and must be save in Elasticsearch

protected function logErrorInElastic(Response $response): void
    {
        try {
            // Not work with library pdphilip/laravel-elasticsearch for a problem with authentication
            $workAnalyzer = WorkAnalyzer::create([
                'datetime' => now(),
                'modem_id' => $this->model->id,
                'provider' => $this->model->manufacturer?->provider,
                'log_id' => $this->ulid,
                'status_code' => $response->status(),
                'timestamp' => [
                    'job' => $this->job_time,
                    'delayed_at' => $this->delayed_at,
                ],
                'errors' => [
                    'message' => $response->json('message') ?? null,
                    'error' => $response->json('data.error') ?? null,
                ],
            ]);
        } catch (Exception $e) {
            ray($e->getMessage())->red(); // Error throw
            Log::error('Error saving error into Elasticsearch: '.$e->getMessage());
        }
    }

Error throw

401 Unauthorized: unable to authenticate user [elastic] for REST request [/test-analyzers/_doc] - unable to authenticate user [elastic] for REST request [/test-analyzers/_doc]

Expected behavior

Well, I tested it in Tinker and it works in production and locally too. Simple, I expected it to save the document and return the data something like in Tinker.

$wa = WorkAnalyzer::create(['datetime' => 1664040526,'modem_id' => 999999])
= App\Models\WorkAnalyzer {#10108
datetime: Illuminate\Support\Carbon @1664040526 {#16255
date: 2022-09-24 17:28:46.0 UTC (+00:00),
},
modem_id: 999999,
_id: "5yj715IB-ZP47e-ynrVN",
}

Screenshots: If applicable, add screenshots to help explain your problem.

Component Versions (Paste in the require section from your composer.json file):

  "require": {
  "php": "^8.1",
        "ext-pdo": "*",
        "ext-redis": "*",
        "ext-zlib": "*",
        "archtechx/enums": "^0.3.0",
        "babenkoivan/elastic-scout-driver": "^3.0",
        "babenkoivan/elastic-scout-driver-plus": "^4.2",
        "barryvdh/laravel-debugbar": "^3.13",
        "barryvdh/laravel-dompdf": "^2.0",
        "binarytorch/larecipe": "^2.5",
        "codeat3/blade-google-material-design-icons": "^1.18",
        "codeat3/blade-iconpark": "^1.5",
        "codeat3/blade-jam-icons": "^1.5",
        "codeat3/blade-simple-icons": "^5.14",
        "dompdf/dompdf": "^2.0",
        "elasticsearch/elasticsearch": "^8.10",
        "filament/filament": "^3.0-stable",
        "geokit/geokit": "^1.3",
        "guzzlehttp/guzzle": "^7.2",
        "laravel-notification-channels/telegram": "^4.0",
        "laravel/framework": "^10.0",
        "laravel/horizon": "^5.17",
        "laravel/pail": "^1.0",
        "laravel/prompts": "^0.1.13",
        "laravel/sanctum": "^3.2",
        "laravel/scout": "^9.8",
        "laravel/tinker": "^2.7",
        "leandrocfe/filament-apex-charts": "^3.1",
        "livewire/livewire": "^3.0",
        "lorisleiva/laravel-actions": "^2.4",
        "maatwebsite/excel": "^3.1",
        "matanyadaev/laravel-eloquent-spatial": "^3.1",
        "monooso/unobserve": "5.x",
        "novadaemon/filament-pretty-json": "^2.0",
        "owen-it/laravel-auditing": "^13.3",
        "pdphilip/elasticsearch": "~4",
        "predis/predis": "^2.2",
        "ramsey/uuid": "^4.7",
        "robinvdvleuten/ulid": "^5.0",
        "saade/filament-laravel-log": "^3.0",
        "silber/bouncer": "^1.0",
        "spatie/laravel-backup": "^8.4",
        "spatie/laravel-data": "^2.1",
        "spatie/laravel-health": "^1.27",
        "spatie/laravel-query-builder": "^5.1",
        "spatie/laravel-rate-limited-job-middleware": "^2.2",
        "spatie/laravel-ray": "^1.36",
        "spatie/laravel-settings": "^2.8",
        "spatie/laravel-translatable": "^6.1",
        "spatie/simple-excel": "^3.0",
        "vlucas/phpdotenv": "^5.4",
        "wireui/wireui": "^2.2"
  },

I have reviewed both the .env and the .env.testing and laso debug wuth show me a correct configuration:

[
  "driver" => "elasticsearch"
  "auth_type" => "http"
  "hosts" => 0 => "https://localhost:9200"
  "username" => "elastic"
  "password" => "PnwodxL+H2TZalaJNHYT"
  "cloud_id" => ""
  "api_id" => ""
  "api_key" => ""
  "ssl_cert" => "/Users/abkrim/Sites/sitelight/ssl/http_ca_local.crt"
  "index_prefix" => ""
  "query_log" => array:2 [▼
    "index" => "laravel_query_logs"
    "error_only" => true
  ]
]

Apreciate some help or ideas for debug

More debug

test('response fails with failed via server error', function () {
    Bus::fake();

    $indice = 'analyzers';
    $this->refreshTestingIndex($indice);

    $wa = new \App\Models\WorkAnalyzer;
    ray($wa->getConnectionName(), config('database.connections.elasticsearch'))->die();

    ray(config('database.connections.elasticsearch'))->green();

I have added a debug for check

  1. refreshTstingIndex delete and create a empty index with name test-analyzers. Not problem with other index for work with fake data and seeding, named work-analyzers
  2. Connjection setup is correct.

    
    array:11 [▼
    "driver" => "elasticsearch"
    "auth_type" => "http"
    "hosts" => array:1 [▶]
    "username" => "elastic"
    "password" => "PnwodxL+H2TZalaJNHYT"
    "cloud_id" => ""
    "api_id" => ""
    "api_key" => ""
    "ssl_cert" => "/Users/abkrim/Sites/sitelight/ssl/http_ca_local.crt"
    "index_prefix" => ""
    "query_log" => array:2 [▼
    "index" => "laravel_query_logs"
    "error_only" => true
    ]
    ]```   
pdphilip commented 2 days ago

Hey @abkrim , it's hard to know how to isolate the issue. Can you give me a simple way to recreate this with:

Thanks

abkrim commented 2 days ago

Model

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Relations\BelongsTo;
use PDPhilip\Elasticsearch\Eloquent\Model;

class WorkAnalyzer extends Model
{
    const CREATED_AT = null;

    const UPDATED_AT = null;

    const MAX_SIZE = 2000;

    protected $connection = 'elasticsearch';

    /**
     * Check __construct method
     *
     * @var string
     */
    protected $index = 'work-analyzers';

    protected $casts = [
        'datetime' => 'datetime',
    ];

    protected $guarded = [];

    public function getConnectionName(): string
    {
        return $this->connection;
    }

    public function __construct(array $attributes = [])
    {
        parent::__construct($attributes);

        $mapping = require base_path('elk/mappings/work-analyzers.php');

        $this->fillable = $this->processMapping($mapping['properties']);

        if (app()->environment('testing')) {
            $this->index = 'test-analyzers';
        }
    }

    private function processMapping(array $properties): array
    {
        foreach ($properties as $key => $value) {
            if (isset($value['type']) && $value['type'] === 'runtime') {
                unset($properties[$key]);
            } elseif (isset($value['script'])) {
                unset($properties[$key]);
            }
        }

        return array_keys($properties);
    }

    public function commandCenter(): BelongsTo|\PDPhilip\Elasticsearch\Relations\BelongsTo
    {
        return $this->belongsTo(CommandCenter::class, 'modem_id', 'id');
    }
}

Mapping

<?php

return [
    'runtime' => [
        'total_consumption_ok' => [
            'type' => 'double',
            'script' => [
                'source' => "boolean isNumber(def input) {
                            if (input instanceof Number) {
                                return true;
                            }
                            if (input instanceof String) {
                                try {
                                    Double.parseDouble(input);
                                    return true;
                                } catch (NumberFormatException e) {
                                    return false;
                                }
                            }
                            return false;
                        }
                        double sum = 0;
                        if (doc['pa1_w'].size() != 0 && isNumber(doc['pa1_w'].value) && doc['pa1_w'].value > 0) {
                          sum += doc['pa1_w'].value;
                        }
                        if (doc['pa2_w'].size() != 0 && isNumber(doc['pa2_w'].value) && doc['pa2_w'].value > 0) {
                          sum += doc['pa2_w'].value;
                        }
                        if (doc['pa3_w'].size() != 0 && isNumber(doc['pa3_w'].value) && doc['pa3_w'].value > 0) {
                          sum += doc['pa3_w'].value;
                        }
                        emit(sum);",
                'lang' => 'painless',
            ],
        ],
    ],
    'properties' => [
        // Internal
        'log_id' => [
            'type' => 'keyword',  // int to keyword
        ],
        // Status
        'status_code' => [
            'type' => 'keyword',
        ],
        'message' => [
            'type' => 'keyword',
            'index' => false,
            'doc_values' => false,
        ],
        'mod_bus_error' => [
            'type' => 'keyword',
            'index' => false,
            'doc_values' => false,
        ],
        //
        'modem_id' => [
            'type' => 'keyword',
        ],

        'datetime' => [
            'type' => 'date',
            'format' => 'epoch_second||yyyy-MM-dd HH:mm:ss||strict_date_optional_time||epoch_millis',
            'ignore_malformed' => true,
        ],
        'provider' => [
            'type' => 'keyword',
        ],
        'response_time' => [
            'type' => 'float',
            'index' => false,
            'doc_values' => false,
        ],
        // End internal
        // Hay muchos sin date al principio
        'ip' => [
            'type' => 'ip',
            'doc_values' => false,
            'index' => false,
        ],
        'eneact_kwh' => [
            'type' => 'float',
        ],
        'enerea_kvarh' => [
            'type' => 'float',
        ],
        'eneapa_kvah' => [
            'type' => 'float',
        ],
        'eneact1_kwh' => [
            'type' => 'float',
        ],
        'eneact2_kwh' => [
            'type' => 'float',
        ],
        'eneact3_kwh' => [
            'type' => 'float',
        ],
        'enerea1_kvarh' => [
            'type' => 'float',
        ],
        'enerea2_kvarh' => [
            'type' => 'float',
        ],
        'enerea3_kvarh' => [
            'type' => 'float',
        ],
        'eneapa1_kvah' => [
            'type' => 'float',
        ],
        'eneapa2_kvah' => [
            'type' => 'float',
        ],
        'eneapa3_kvah' => [
            'type' => 'float',
        ],
        'powact_w' => [
            'type' => 'float',
        ],
        'powrea_var' => [
            'type' => 'float',
        ],
        'powapa_va' => [
            'type' => 'float',
        ],
        'freq_hz' => [
            'type' => 'float',
        ],
        'pa1_w' => [
            'type' => 'float',
        ],
        'pa2_w' => [
            'type' => 'float',
        ],
        'pa3_w' => [
            'type' => 'float',
        ],
        'pr1_var' => [
            'type' => 'float',
        ],
        'pr2_var' => [
            'type' => 'float',
        ],
        'pr3_var' => [
            'type' => 'float',
        ],
        'pp1_va' => [
            'type' => 'float',
        ],
        'pp2_va' => [
            'type' => 'float',
        ],
        'pp3_va' => [
            'type' => 'float',
        ],
        'iac1_a' => [
            'type' => 'float',
        ],
        'iac2_a' => [
            'type' => 'float',
        ],
        'iac3_a' => [
            'type' => 'float',
        ],
        'vac1_v' => [
            'type' => 'float',
        ],
        'vac2_v' => [
            'type' => 'float',
        ],
        'vac3_v' => [
            'type' => 'float',
        ],
        'lvac1_v' => [
            'type' => 'float',
        ],
        'lvac2_v' => [
            'type' => 'float',
        ],
        'lvac3_v' => [
            'type' => 'float',
        ],
        'pf1' => [
            'type' => 'float',
        ],
        'pf2' => [
            'type' => 'float',
        ],
        'pf3' => [
            'type' => 'float',
        ],
        'v_event' => [
            'type' => 'byte',
        ],
        'pf' => [
            'type' => 'float',
        ],
        'pa1_m_w' => [
            'type' => 'float',
        ],
        'pa1_x_w' => [
            'type' => 'float',
        ],
        'pa1_a_w' => [
            'type' => 'float',
        ],
        'pa2_m_w' => [
            'type' => 'float',
        ],
        'pa2_x_w' => [
            'type' => 'float',
        ],
        'pa2_a_w' => [
            'type' => 'float',
        ],
        'pa3_m_w' => [
            'type' => 'float',
        ],
        'pa3_x_w' => [
            'type' => 'float',
        ],
        'pa3_a_w' => [
            'type' => 'float',
        ],
        'pr1_m_var' => [
            'type' => 'float',
        ],
        'pr1_x_var' => [
            'type' => 'float',
        ],
        'pr1_a_var' => [
            'type' => 'float',
        ],
        'pr2_m_var' => [
            'type' => 'float',
        ],
        'pr2_x_var' => [
            'type' => 'float',
        ],
        'pr2_a_var' => [
            'type' => 'float',
        ],
        'pr3_m_var' => [
            'type' => 'float',
        ],
        'pr3_x_var' => [
            'type' => 'float',
        ],
        'pr3_a_var' => [
            'type' => 'float',
        ],
        'vac1_m_v' => [
            'type' => 'float',
        ],
        'vac1_x_v' => [
            'type' => 'float',
        ],
        'vac1_a_v' => [
            'type' => 'float',
        ],
        'vac2_m_v' => [
            'type' => 'float',
        ],
        'vac2_x_v' => [
            'type' => 'float',
        ],
        'vac2_a_v' => [
            'type' => 'float',
        ],
        'vac3_m_v' => [
            'type' => 'float',
        ],
        'vac3_x_v' => [
            'type' => 'float',
        ],
        'vac3_a_v' => [
            'type' => 'float',
        ],
        'iac1_m_a' => [
            'type' => 'float',
        ],
        'iac1_m_x' => [
            'type' => 'float',
        ],
        'iac1_x_a' => [
            'type' => 'float',
        ],
        'iac1_a_a' => [
            'type' => 'float',
        ],
        'iac2_m_a' => [
            'type' => 'float',
        ],
        'iac2_m_x' => [
            'type' => 'float',
        ],
        'iac2_x_a' => [
            'type' => 'float',
        ],
        'iac2_a_a' => [
            'type' => 'float',
        ],
        'iac3_m_a' => [
            'type' => 'float',
        ],
        'iac3_m_x' => [
            'type' => 'float',
        ],
        'iac3_x_a' => [
            'type' => 'float',
        ],
        'iac3_a_a' => [
            'type' => 'float',
        ],
        'cos1_m' => [
            'type' => 'float',
        ],
        'cos1_a' => [
            'type' => 'float',
        ],
        'cos1_x' => [
            'type' => 'float',
        ],
        'cos2_m' => [
            'type' => 'float',
        ],
        'cos2_a' => [
            'type' => 'float',
        ],
        'cos2_x' => [
            'type' => 'float',
        ],
        'cos3_m' => [
            'type' => 'float',
        ],
        'cos3_x' => [
            'type' => 'float',
        ],
        'cos3_a' => [
            'type' => 'float',
        ],
        'errors' => [
            'type' => 'object',
            'enabled' => false,
        ],
        'timestamps' => [
            'type' => 'object',
            'enabled' => false,
        ],
        'slim_data' => [
            'type' => 'object',
            'enabled' => false,
        ],
    ],
];

Data for fail

$workAnalyzer = WorkAnalyzer::create([
                'datetime' => now(),
                'modem_id' => 1
                'provider' => 'provider'
                'log_id' => '01JB8XY3SEXKT73GFJTPM0G5BE',
                'status_code' => 500,
                'timestamp' => [
                    'job' => 1730204454,
                    'delayed_at' => 1730204455,
                ],
                'errors' => [
                    'message' => 'Error....."
                    'error' => null,
                ],
            ]);

But with all step I have added a debug for check add at same time taht your reply, I think is another problem for get correct connection.

a simple

  test('response fails with failed via server error', function () {
    Bus::fake();

    $indice = 'analyzers';
    $this->refreshTestingIndex($indice);

    $wa = new \App\Models\WorkAnalyzer;
    ray($wa->getConnectionName(), config('database.connections.elasticsearch'));
  ray(\App\Models\WorkAnalyzer::first()); // This fails and connection name and confi is correct.

Thanks you.

abkrim commented 2 days ago

Sorry, sorry.

I'm embarrassed. The mistake was mine,

I had forgotten because I didn't use in my current code that to use your library, I had more configurations in the .env.testing and not see this details in debug.

ELK_DOCKER_HOST=https://localhost:9200
ELK_DOCKER_CA_BUNDLE=http_ca_local.crt
ELK_DOCKER_USER=elastic
ELK_DOCKER_PASSWORD='FORGET_UPDATE_THIS_PASSWORD_2'
# pdphilip/Elastic ES_AUTH_TYPE=http ES_HOSTS="https://localhost:9200" ES_USERNAME=elastic ES_PASSWORD='FORGET_UPDATE_THIS_PASSWORD_LAST_2' ES_CLOUD_ID= ES_API_ID= ES_API_KEY= ES_SSL_CERT=ssl/http_ca_local.crt ES_INDEX_PREFIX=

Sorry to have bothered you.

A cordial greeting.