Can't seem to get full text search working using SOAP

frdmn commented 8 years ago

Hello,

i struggle since about 3 months to get the full text search working via SOAP.

Already tried asking for help in the official otterhub forum, but noone seems to be able to help me here: OtterHub - OTRS Community | Full text search via SOAP and GenericTicketConnector

Here's what I use at the moment:

tickets = client.ticket_search(
    ContentSearch = 'OR',
    From = '%' + args['input'] + '%',
    To = '%' + args['input'] + '%',
    Cc = '%' + args['input'] + '%',
    Subject = '%' + args['input'] + '%',
    Body = '%' + args['input'] + '%',
    Title = '%' + args['input'] + '%'
)

... based on the example code in the Ticket.pm file in the OTRS source: https://github.com/OTRS/otrs/blob/1e908159a5dbdcfb94cc35d13bf15b04ac3e6a24/Kernel/System/LinkObject/Ticket.pm#L271-L282, but for some reason it always returns an empty array.

However, it works fine when I try a more simple search like:

tickets = client.ticket_search(
    Title = '%' + args['input'] + '%'
)

I want to search like the full text search in the web ui though. Any ideas?

ewsterrenburg commented 8 years ago

As far as I can see, ContentSearch = 'OR' is being ignored by the OTRS Generic Ticketconnector.

When I do some testing, the results appear as if only results are given back for which each of the conditions specified is True. This behavior occurs both calling the TicketConnector from Python and when testing the GenericTicket connector directly using SoapUI.

@frdmn could you do some testing if you agree with me that the ticket connector checks for each condition regardless of the value supplied for ContentSearch? If so, we could file a bug report with OTRS itself.

frdmn commented 8 years ago

@ewsterrenburg I would love to debug/test this with you! I already spent several months trying to get a proper response from the OTRS team, but no-one really could help me out yet.

Let me know when and how you need my help :+1:

ewsterrenburg commented 8 years ago

Doesn't sound really hopeful, since I suspect the issue is located within OTRS itself...

@frdmn Could you please check if a request of the following nature only returns you the tickets for which the string being searched for (I used "test", use something that is present in your tickets ;)) occurs both in the subject and body of the ticket. If so, I will put it up on the OTRS mailing list

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns="http://www.otrs.org/TicketConnector/">
  <soapenv:Header/>
  <soapenv:Body>
  <TicketSearch>
           <UserLogin>username</UserLogin>
         <Password>password</Password>
  <Body>test</Body>
  <Subject>test</Subject>
  <ContentSearch>OR</ContentSearch>
</TicketSearch>
  </soapenv:Body>
</soapenv:Envelope>

frdmn commented 8 years ago

Not sure if I do this right, but I setup SoapUI, downloaded the WSDL, imported it and tried the following request:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns="http://www.otrs.org/TicketConnector/">
  <soapenv:Header/>
  <soapenv:Body>
    <TicketSearch>
     <UserLogin>jonas.friedmann</UserLogin>
     <Password>[password]</Password>
     <Body>RapidSSL</Body>
     <Subject>RapidSSL</Subject>
     <ContentSearch>OR</ContentSearch>
   </TicketSearch>
 </soapenv:Body>
</soapenv:Envelope>

Which returns the following (error) response:

<soap:Envelope soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soap:Body>
      <Fault xmlns="http://www.otrs.org/TicketConnector/">
         <faultcode>Server</faultcode>
         <faultstring>Namespace from SOAPAction 'http://www.otrs.org/TicketConnector' does not match namespace from configuration 'http://www.otrs.org/TicketConnector/'</faultstring>
      </Fault>
   </soap:Body>
</soap:Envelope>

Did I miss anything?

ewsterrenburg commented 8 years ago

Don't know, it's working for me. But you could also test it using your python code I guess...

frdmn commented 8 years ago

Interesting, works for me via python-otrs but not via raw SOAP. I'm going to take a look in the Webservice Debugger and try to compare both requests to see the differences. Stay tuned :)

frdmn commented 8 years ago

Here's the request received from the debugger:

via python-otrs (works fine): http://paste.frd.mn/ocigo
via SoapUI.app (doesn't work): http://paste.frd.mn/xeguy

Comparison:

2,3c2,3
<   'CONTENT_LENGTH' => '422',
<   'CONTENT_TYPE' => 'text/xml;charset=UTF-8',
---
>   'CONTENT_LENGTH' => '388',
>   'CONTENT_TYPE' => 'text/xml;charset=utf-8',
7,8c7,8
<   'HTTP_ACCEPT_ENCODING' => 'gzip,deflate',
<   'HTTP_CONNECTION' => 'Keep-Alive',
---
>   'HTTP_ACCEPT_ENCODING' => 'identity',
>   'HTTP_CONNECTION' => 'close',
10,11c10
<   'HTTP_SOAPACTION' => '"http://www.otrs.org/TicketConnector/TicketSearch"',
<   'HTTP_USER_AGENT' => 'Apache-HttpClient/4.1.1 (java 1.5)',
---
>   'HTTP_USER_AGENT' => 'Python-urllib/2.7',
19c18
<   'REMOTE_PORT' => '56885',
---
>   'REMOTE_PORT' => '56379',
30c29,30
<   'SERVER_SOFTWARE' => 'Apache/2.2.16 (Debian)'
---
>   'SERVER_SOFTWARE' => 'Apache/2.2.16 (Debian)',
>   'SSL_TLS_SNI' => 'ticket.intern.company.de'

My guess is that the HTTP_SOAPACTION makes the request invalid out of SoapUI. I have no clue how that got in there though - I assume because I imported the WSDL?

frdmn commented 8 years ago

@ewsterrenburg Finally got it working! Apparently the SOAPACTION is a SoapUI-specific behaviour(?). I just tried to send the raw HTTP request using Postman and that immediately worked like a charm.

Regarding your actual question: I've picked a ticket which contains a really unique ticket number from a remote ticket system (2016031410000296) only in the subject, but not in the body.

Now I tried a SOAP request like this:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns="http://www.otrs.org/TicketConnector/">
  <soapenv:Header/>
  <soapenv:Body>
    <TicketSearch>
     <UserLogin>jonas.friedmann</UserLogin>
     <Password>[password]</Password>
     <Body>2016031410000296</Body>
     <Subject>2016031410000296</Subject>
     <ContentSearch>OR</ContentSearch>
   </TicketSearch>
 </soapenv:Body>
</soapenv:Envelope>

And this is what I've received:

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <soap:Body>
        <TicketSearchResponse xmlns="http://www.otrs.org/TicketConnector/" xsi:nil="true" />
    </soap:Body>
</soap:Envelope>

Ergo, the response is completely empty even though, it should actually list at least one ticket ID.

To cross check this problem I removed either <Body> or <Subject>, and it returned the proper ticket ID as expected.

I guess this confirms our assumptions, right?

ewsterrenburg commented 8 years ago

@frdmn I agree. Will you report this to the OTRS development team or should I do so?

frdmn commented 8 years ago

@ewsterrenburg I am not really familiar with mailing lists, so it would be nice if you could submit it. If you have more important stuff to do right now, I can also try to o it :)

ewsterrenburg commented 8 years ago

Posted it on the OTRS mailinglist

frdmn commented 8 years ago

Thank you! Now let's wait patiently for a response :)

ewsterrenburg commented 8 years ago

@frdmn no reaction on the mailing list. Reported it with the otrs bugtracker: http://bugs.otrs.org/show_bug.cgi?id=11981

frdmn commented 8 years ago

🙂👍

Von meinem iPhone gesendet

Am 04.04.2016 um 15:43 schrieb ewsterrenburg notifications@github.com:

@frdmn no reaction on the mailing list. Reported it with the otrs bugtracker: http://bugs.otrs.org/show_bug.cgi?id=11981

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

frennkie commented 8 years ago

I have been working on a Python implementation for the OTRS API over the last few days which - I'll have to admit - actually does the same thing as you have done here. But I really did not want to touch that SOAP stuff and am using the REST interface. i believe I have run into the same issue with my project as you have:

In [10]: client.ticket_search(Subject="%worldwide%")
DEBUG:pyotrs.client:{}
DEBUG:pyotrs.client:search found no results
Out[10]: []

In [11]: client.ticket_search(Body="%worldwide%")
DEBUG:pyotrs.client:{u'TicketID': [u'1']}
Out[11]: [u'1']

In [12]: client.ticket_search(Body="%worldwide%",Subject="%worldwide%",ContentSearch="OR")
DEBUG:pyotrs.client:{}
DEBUG:pyotrs.client:search found no results
Out[12]: []

The only ticket I have in my system is the initial "Welcome to OTRS" which has "worldwide" in it's body, so the ticket should also be found in my [12] example, right?!

ewsterrenburg commented 8 years ago

@frennkie I agree with you on this one and expect that ticket to be found... It keeps being quiet on both OTRS mailinglist and Bugtracker though.

Since @frdmn has similar experiences of proper responses lacking, priority for the dev team seems to be low. Fortunately, I am not using full text searches anyway.

frennkie commented 8 years ago

@ewsterrenburg I assume you have absolutely no intention on using/switch to REST as you already have a full SOAP implementation, right? I have not yet published my code but am planning to do so (possibly as "PyOTRS"). Any objections from your side?

ewsterrenburg commented 8 years ago

@frennkie

No objections at all and neither any intention to use/switch to REST at the moment. Generally I like REST better as well, yet there were two reasons have chosen SOAP:

Me being totally clueless how to set up the REST service within OTRS
A perfectly fine solution for SOAP was already available created by @JocelynDelalande which I could extend.

frennkie commented 8 years ago

@ewsterrenburg I think you got the wrong frennkie there.. I'm @frennkie ;-) Otherwise I completely understand you. Setting up the OTRS REST service is actually very easy and done in two minutes. All you need to do is download the example .yml config from the OTRS repo and upload it as a new web service: https://github.com/OTRS/otrs/tree/master/development/webservices

ewsterrenburg commented 8 years ago

@frennkie ups, so many fren(n)kies in this world...

Ok, that doesn't seem difficult. Nonetheless I was being clueless when I installed OTRS while the SOAP service worked right away. So, maybe it's possible to switch someday, yet for now SOAP service does it's job perfectly fine :)

frdmn commented 8 years ago

@frennkie Perhaps you could "+1" the issue in the bug tracker so the OTRS dev team notice the demand of a fix for this problem.

Right now, my original project is kind of abandoned because it depends on a full text search in OTRS :/

I already thought about just scraping the web page instead for now...

frennkie commented 8 years ago

@frdmn I already posted a follow up in the OTRS tracker to Bug 11981.

frdmn commented 8 years ago

@frennkie Oh, I saw Robert's comments but didn't realize it's you. Thanks buddy! :+1:

ewsterrenburg commented 8 years ago

Issue is clear. Is now fixed in OTRS (at least in OTRS5). Will close this issue.

frdmn commented 8 years ago

👍

Filterer commented 6 years ago

Hi all,

I know that this issue is closed but I have the same problem when trying to search tickets based on their article fields (subject, body etc). I m having OTRS 6 on ubuntu and I am executing queries with curl in the terminal. OTRS is also configured as an Invoker with REST and I am using the common GenericTicketConnectorREST webservice with the basic ticket and session functions.

For example: This query works and searches for tickets with a specific title and the ones that belong to the 'Raw' queue. curl "http://localhost/otrs/nph-genericinterface.pl/Webservice/GenericTicketConnectorREST/Ticket?SessionID=xLHFqyIbO9OUDZypaFGSVDbe3sA0v3NF&Title=some+title&Queues=Raw"

But this one that searches for tickets with a specific subject it doesn't work and returns all the existing tickets on the OTRS server. curl "http://localhost/otrs/nph-genericinterface.pl/Webservice/GenericTicketConnectorREST/Ticket?SessionID=xLHFqyIbO9OUDZypaFGSVDbe3sA0v3NF&Subject=some+subject"

Same problem exists for the From, To, CC and Body. ContentSearch and FullTextIndex don't make a difference as well. Also AttachmentName doesn't seem to work as well but that is not article related I guess.

Any help will be appreciated.

ewsterrenburg commented 6 years ago

@Filterer Am I correct that you are not experiencing an issue with the python-otrs library, yet with OTRS itself?

If so, could you please try the OTRS mailing list or one of the OTRS related forums for help...

ewsterrenburg / python-otrs

Can't seem to get full text search working using SOAP #10