teamtnt / laravel-scout-tntsearch-driver

Driver for Laravel Scout search package based on https://github.com/teamtnt/tntsearch
MIT License
1.09k stars 142 forks source link

Cannot search IP Adresses #349

Closed Julien-cpsn closed 1 year ago

Julien-cpsn commented 1 year ago

Hello !

I've been using TNTsearch for about a week now and it works great, but i'm having a really big problem. I cannot search on IP Adresses in my database, and they aren't indexed (even in the wordlist). It works with everything else, like the port and all my other models.

I've tried many IP Adreeses, to escape every single character, convert it with ip2long(), remove dots but nothing works...

Here's my table definition :

create table "fluxes_IP"
(
    id              bigserial primary key,
    "ip_source_SSM" inet,
    multicast       inet,
    port            integer,
    created_at      timestamp(0),
    updated_at      timestamp(0)
);

Here my model definition :

<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;

class FluxIP extends Model
{
    use HasFactory, Searchable;

    static $frenchName = 'Flux IP';
    protected $table = "fluxes_IP";
    protected $hidden = ['created_at', 'updated_at'];
    protected $fillable = [
      'ip_source_SSM',
      'multicast',
      'port'
    ];

    /* ----=== RELATIONS ===---- */
    public function flux() {
        return $this->morphOne(Flux::class, 'fluxable');
    }

    /* ----=== SCOPES ===---- */
    public function scopeFindBySourceMulticastPortSid($query, $ip_source_SSM, $multicast, $port, $sid) {...}

    public function __toString()
    {
        return ($this->ip_source_SSM ?:'ø') . ' - ' . ($this->multicast ?:'ø') . ' - ' . ($this->port ?:'ø');
    }

    /* ----=== SEARCH ===---- */
    /**
     * Get the indexable data array for the model.
     *
     * @return array
     */
    public function toSearchableArray(): array
    {
        return $this->only([
            'id',
            'ip_source_SSM',
            'multicast',
            'port'
        ]);
    }
}

My searching code (pretty basic) :

dd(FluxIP::search('0.0.0.0')->get())

It returns more than 500 results, but this IP Address only appears 9 times in my database. I have more than 24.000 rows in this table.

Does someone have a solution or a suggestion? It's blocking everything for an important project.

Versions : laravel-scout-tntsearch-driver 12.1 tntsearch 3.0 PHP 7.4 PostGreSQL 13.4

Julien-cpsn commented 1 year ago

I can't show any IP address because it is personnal data, sorry for the inconvenience

nticaric commented 1 year ago

To better understand the issue, take a look a the Tokenizer class. It will break down the IP address by dots and you'll get bunch of numbers as tokens, which is not helpful in your case. What you can do is to base64 enocode the address:

    public function toSearchableArray()
    {
        return [
            'id'   => $this->id,
            'ip'   => base64_encode($this->ip),
            'port' => $this->port
        ];
    }

And while doing a search do FluxIP::search(base64_encode('0.0.0.0'))->get()

Don't forget to reindex after changing the toSearchableArray method

nticaric commented 1 year ago

Another option is writing a custom tokenizer specially for IP addresses

Julien-cpsn commented 1 year ago

Thanks for your reactivity ! Does work really great for a single model search :)

But now I am unfortunately getting another problem :( The query is getting weird when there is dots in the search string (eg. IP Address obviously)

Since I'm searching on different models with the same search terms it gets kinda ridiculous

The IP Address "0.0.0.0" now finds the FluxIP that are concerned (thanks!) but it also returns some more results from other models that aren't supposed to be here but because of the dots. Any suggestion?