search

HashtagSell / posting-api

API for storage and retrieval of posting details for Hashtagsell
0 stars 0 forks source link

PersistenceError: unable to find postings #43

Closed that1guy closed 9 years ago

that1guy commented 9 years ago

Some search queries return 500 error (for example the word "sectional" or "kart"). I can reproduce this issue quite often if I pick a rare keyword from the heading of a posting that was JUST ingested. I've tried flushing elasticsearch and restarting all services..

Get Request
http://localhost:4043/v1/postings/?start=0&count=35&filters[mandatory][contains][heading]=sectional&filters[optional][exact][categoryCode]=SFUR&geo[lookup]=true&geo[min]=0&geo[max]=12890000
Node logs
    06:26:13.689Z  INFO posting-api: GET /v1/groupings/popular?query=sectional
        06:26:13.692Z TRACE posting-api: sectional
    GET /favicon.ico 304 20.241 ms - -
    2015-04-23T23:26:14.343-0700 [conn20] command hashtagsell-posting-v1.$cmd command: aggregate { aggregate: "postings", pipeline: [ { $match: { $or: [ { heading: /\bsectional\b/sim }, { body: /\bsectional\b/sim } ] } }, { $group: { _id: "$categoryCode", count: { $sum: 1 } } }, { $sort: { count: -1 } } ] } keyUpdates:0 numYields:1 locks(micros) r:954939 reslen:482 649ms
        06:26:14.344Z TRACE posting-api: findPopular completed in 652 milliseconds
        06:26:14.357Z  INFO posting-api: GET /v1/postings/?start=0&count=35&filters[mandatory][contains][heading]=sectional&filters[optional][exact][categoryCode]=SFUR&geo[lookup]=true&geo[min]=0&geo[max]=100000
        >> 2015/04/23 23:26:14 "HTTP/1.1" 200 "GET" "/json/216.38.134.18" 127.0.0.1 "" 252b in 108.118us ""
        06:26:14.361Z TRACE posting-api: request to geo coords for IP Address 127.0.0.1 completed in 3 milliseconds
        06:26:14.361Z TRACE posting-api: local IP address 127.0.0.1 will be replaced with 216.38.134.18 for request
        06:26:14.371Z ERROR posting-api: 500: GET /v1/postings/?start=0&count=35&filters[mandatory][contains][heading]=sectional&filters[optional][exact][categoryCode]=SFUR&geo[lookup]=true&geo[min]=0&geo[max]=100000
        06:26:14.371Z ERROR posting-api: unable to find postings
        err: {
          "name": "PersistenceError",
          "message": "unable to find postings",
          "sourceError": {
            "statusCode": 400
          },
          "statusCode": 500,
          "options": {
            "count": "35",
            "filters": {
              "mandatory": {
                "contains": {
                  "heading": "sectional"
                }
              },
              "optional": {
                "exact": {
                  "categoryCode": "SFUR"
                }
              }
            },
            "geo": {
              "lookup": "true",
              "min": 0,
              "max": 100000,
              "ip": "216.38.134.18",
              "coords": [
                -122.399,
                37.799
              ]
            },
            "sort": {},
            "start": "0"
          }
        }
Mongo dump, just in case
https://drive.google.com/file/d/0B7Ul13wcHjNIUTJFbkRRUEdYX2s/view?usp=sharing
brozeph commented 9 years ago

Found that if one optional filter is added, the query to ElasticSearch fails, but if multiple optional filters are added, the query operates as expected. I fixed the abstraction layer by ignoring optionals if only one is supplied.