Open lis975 opened 5 months ago
lis975
commented
4 months ago my additional observation is: select count(*) from ticket returns above 7000000 records, but index ticket contains above 21000000 documents (approx 3x more than tickets in database). Maybe there is a problem in selecting tickets for indexing? The rest of indexes have almost the same count of objects (article, ticket_history).
lis975
commented
1 month ago I am still looking for a solution to this problem. Any hints? Maybe it's a problem with the configuration of my system. I've already done some experiments: re-establishing indexes, re-indexing, even setting up templates on the ES side. I still get the same error every time system sends changes to ticket to be indexed, making the whole ES engine useless for me.
My solution at the moment is: when you have ticket attachments on disk, you should set SearchEngine::Settings::Index::ES::Ticket###000-Framework IndexAttachments => 0 to avoid errors.
hanneshal
commented
1 month ago Hi, I tranferred the issue to the correct repo and we will have a look. We actually use it in the same way / config, but we do not have close to your ticket count.
We get back to you
Environment
Expected behavior
Elasticsearch plugin will keep the index up to date.
Actual behavior
for every new ticket crated in system error occurs:
ERROR: OTRS-otrs.Console.pl-Maint::Search::ES::IndexQueueDataProcess-70 Perl: 5.26.3 OS: linux Time: Mon Mar 18 09:26:20 2024
Message: Query failed for engine: Kernel::System::Search::Engine::ES. Message: [Request] ** [[https://xxxxxxxxxxxxxxxxx:9200]-[400]https://xxxxxxxxxxxxxxxxx:9200]-[400] Bad Request, called from sub Search::Elasticsearch::Transport::perform_request at /opt/otrs/Kernel/System/Search/Engine/ES.pm line 388. With vars: {'request' => {'qs' => {'pipeline' => 'attachment_nested'},'path' => 'ticket/_update_by_query','body' => {'query' => {'terms' => {'TicketID' => [9999891]}}},'method' => 'POST','ignore' => [],'mime_type' => 'application/json','serialize' => 'std'},'status_code' => 400,'body' => {'version_conflicts' => 0,'throttled_until_millis' => 0,'throttled_millis' => 0,'retries' => {'search' => 0,'bulk' => 0},'timed_out' => bless( do{(my $o = 0)}, 'JSON::PP::Boolean' ),'deleted' => 0,'took' => 8,'requests_per_second' => '-1','updated' => 0,'batches' => 1,'noops' => 0,'failures' => [{'index' => 'ticket','status' => 400,'id' => '9999891','cause' => {'type' => 'script_exception','reason' => 'runtime error','caused_by' => {'type' => 'class_cast_exception','reason' => 'Cannot cast java.lang.Integer to java.lang.String'},'position' => {'start' => 167,'end' => 248,'offset' => 194},'script_stack' => ['java.base/java.lang.Class.cast(Class.java:4067)','ArticleID = Articles.get(i).ArticleID; ArrayList ',' ^---- HERE'],'script' => ' ...','lang' => 'painless'}}],'total' => 1}}
Traceback (3035849): Module: Kernel::System::Search::Engine::ES::QueryExecute Line: 204 Module: Kernel::System::Search::Object::Engine::ES::Ticket::ObjectIndexGeneric Line: 1507 Module: Kernel::System::Search::Object::Engine::ES::Ticket::ObjectIndexUpdate Line: 1024 Module: Kernel::System::Search::ObjectIndexUpdate Line: 564 Module: Kernel::System::Search::Object::Base::ObjectIndexQueueHandle Line: 1203 Module: Kernel::System::Console::Command::Maint::Search::ES::IndexQueueDataProcess::Run Line: 97 Module: (eval) Line: 461 Module: Kernel::System::Console::BaseCommand::Execute Line: 455 Module: (eval) Line: 143 Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Cron::Run Line: 122 Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Run Line: 236 Module: (eval) Line: 332 Module: main::Start Line: 332 Module: /opt/otrs/bin/otrs.Daemon.pl Line: 153
How to reproduce
Steps to reproduce the behavior:
Additional information
My system is very large. Ticket index is about 1,5TB. Article: 61,6GB, ticket_history: 41,8GB. ES configuration persists of 3 nodes, ES version is 8.12
Screenshots