fergiemcdowall / search-index

A persistent, network resilient, full text search library for the browser and Node.js
MIT License
1.39k stars 149 forks source link

Identical JSON creates duplicate document count. #80

Closed eklem closed 9 years ago

eklem commented 9 years ago

Using this code to add documents to the index

  si.add({'batchName': config.batchname, 'filters': config.filters}, result.data, function(err) {
    if (!err) console.log('indexed!')
  })

First time I get 88 as count when I search for "*", second time I get 173 documents, third time I get "258". Only count seems off, amount of results actually shown in a given search stays the same.

Data I index:

[ { date: '2014-11-10T22:20:00+01:00',
    user: [ 'eklem' ],
    text: 'What in your online life would be interesting for you to have #searchable ? #search #events #sites',
    link: 'http://ift.tt/1uUH33u',
    id: '3ABtU',
    tags: [ 'searchable', 'search', 'events', 'sites' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-13T10:22:00+01:00',
    user: [ 'eklem', 'labs_finn_no', 'IFTTT' ],
    text: '@labs_finn_no På tide å utvikle triggere for @IFTTT ? https://t.co/n9OAv49fXS Tenk på all moroa folk kan gjøre da!',
    link: 'http://twitter.com/eklem/status/532825696486563840',
    id: '1bwKxq',
    tags: null,
    links: [ 'https://t.co/n9OAv49fXS' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-13T10:39:00+01:00',
    user: [ 'eklem', 'labs_finn_no', 'SchibstedGroup', 'SchibstedGrowth' ],
    text: '@labs_finn_no Eller få @SchibstedGroup / @SchibstedGrowth til å utvikle eller investere i noe tilsvarende?',
    link: 'http://twitter.com/eklem/status/532830002002546689',
    id: '3png34',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-13T10:55:00+01:00',
    user: [ 'eklem', 'ramyo', 'Yggdrasil2015' ],
    text: 'RT @ramyo: Har akkurat foreslått et foredrag til @Yggdrasil2015: “Hvordan få til god designkritikk”. Håper det blir antatt – Yggdrasil er alltid moro!',
    link: 'http://twitter.com/eklem/status/532834125749948416',
    id: '1MmNVn',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-15T09:34:00+01:00',
    user: [ 'eklem', 'hildecs' ],
    text: 'RT @hildecs: Nå har det kokt over for Aftenposten. "Fikk kroppen full av tungmetaller og parasitter" http://t.co/RD46U3MPgv #kvakksalverspråk',
    link: 'http://twitter.com/eklem/status/533538592912850944',
    id: 'THf0S',
    tags: [ 'kvakksalverspråk' ],
    links: [ 'http://t.co/RD46U3MPgv' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-15T09:39:00+01:00',
    user: [ 'eklem', 'hildecs', 'frmartinsen' ],
    text: '@hildecs @frmartinsen Hehe "skannet kroppen og målte energien i cellene" + tyske leger. Sikker vinner!',
    link: 'http://twitter.com/eklem/status/533539718181687296',
    id: 'YCIeK',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-15T17:30:00+01:00',
    user: [ 'eklem' ],
    text: 'Noen som savner denne fete kjerra? #inntauing #pimpmyride http://t.co/eVsEpjRQyv',
    link: 'http://twitter.com/eklem/status/533658235027595264',
    id: 'xLKc4',
    tags: [ 'inntauing', 'pimpmyride' ],
    links: [ 'http://t.co/eVsEpjRQyv' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-17T05:07:00+01:00',
    user: [ 'eklem', 'Ruter' ],
    text: '@Ruter greit å åpne stasjonen i tide til første T-bane?',
    link: 'http://twitter.com/eklem/status/534196012584554496',
    id: '1bYSFj',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-17T05:07:00+01:00',
    user: [ 'eklem', 'ruter' ],
    text: '@ruter http://t.co/2EROVvOZZv',
    link: 'http://twitter.com/eklem/status/534196157178994688',
    id: 'O6RUy',
    tags: null,
    links: [ 'http://t.co/2EROVvOZZv' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-18T09:47:00+01:00',
    user: [ 'eklem', 'Ruter' ],
    text: '@Ruter Null svar... Altså: Dere bør åpne T-banestasjonene i tide? Og vakta bør kanskje ikke bruke skinnegangen mellom perrongene i jobben?',
    link: 'http://twitter.com/eklem/status/534628973888937984',
    id: '2X3ZFV',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-18T09:50:00+01:00',
    user: [ 'eklem', 'johanhal', 'hanekamhaug' ],
    text: '@johanhal @hanekamhaug Noen outer seg selv, flere kommentarer overflødig... ?',
    link: 'http://twitter.com/eklem/status/534629594801135616',
    id: '4kzwg3',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-18T10:55:00+01:00',
    user: [ 'eklem', 'Ruter' ],
    text: '@Ruter Takker!',
    link: 'http://twitter.com/eklem/status/534646126347964416',
    id: '3mGbV2',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-18T10:58:00+01:00',
    user: [ 'eklem', 'Ruter' ],
    text: '@Ruter Takke!',
    link: 'http://twitter.com/eklem/status/534646758186708993',
    id: '41FIRP',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-18T14:25:00+01:00',
    user: [ 'eklem', 'basbrommet', 'kickstarter' ],
    text: 'RT @basbrommet: Lease A Jeans: Earth wants your jeans back by Mud Jeans http://t.co/6cnmMUZqzt via @kickstarter',
    link: 'http://twitter.com/eklem/status/534698976940158977',
    id: '3EKpMb',
    tags: null,
    links: [ 'http://t.co/6cnmMUZqzt' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-18T21:18:00+01:00',
    user: [ 'eklem', 'JensBP' ],
    text: 'RT @JensBP: Forskere mener "kristen kulturarv" er en konstruksjon og misbrukes i politisk hestehandling. http://t.co/Fm4Oi4lzeV',
    link: 'http://twitter.com/eklem/status/534802871473422337',
    id: '3YqQ0V',
    tags: null,
    links: [ 'http://t.co/Fm4Oi4lzeV' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-19T12:14:00+01:00',
    user: [ 'eklem' ],
    text: '#Google har masse interne verktøy for bedrifter. Noen som jobber med å integrere/etablere dette hos kunder?',
    link: 'http://twitter.com/eklem/status/535028298276171776',
    id: '3ZyJQP',
    tags: [ 'google' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-19T12:35:00+01:00',
    user: [ 'eklem', 'johshholmedahl' ],
    text: '@johshholmedahl Så kult! Noe blogposter om erfaringer og utbytte?',
    link: 'http://twitter.com/eklem/status/535033737957081088',
    id: '1uIsCB',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-19T15:45:00+01:00',
    user: [ 'eklem', 'GoogleforWork' ],
    text: '@GoogleforWork Is there a possibility to have one search box searching Gmail, Calendar, Contacts, Drive etc?',
    link: 'http://twitter.com/eklem/status/535081479026769920',
    id: '4wGPOl',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-19T16:03:00+01:00',
    user: [ 'eklem', 'GoogleforWork' ],
    text: '@GoogleforWork ... and Hangouts?',
    link: 'http://twitter.com/eklem/status/535086032623243265',
    id: '3GQvqi',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-20T08:59:00+01:00',
    user: [ 'eklem', 'ironskyfilm' ],
    text: '@ironskyfilm Tried to share this page, but got an error http://t.co/OhIAtOtgfQ http://t.co/1fPHD4eKbV',
    link: 'http://twitter.com/eklem/status/535341562872754176',
    id: '4avLPI',
    tags: null,
    links: [ 'http://t.co/OhIAtOtgfQ', 'http://t.co/1fPHD4eKbV' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-24T10:16:00+01:00',
    user: [ 'eklem', 'ironskyfilm' ],
    text: '@ironskyfilm Working now. Thanks!',
    link: 'http://twitter.com/eklem/status/536810615395061760',
    id: '4k17Cd',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-24T13:56:00+01:00',
    user: [ 'eklem' ],
    text: 'Will embed-view of your content replace the preview-pane in modern #search  #engine solutions? Why preview when you can have the real deal?',
    link: 'http://twitter.com/eklem/status/536866049078333440',
    id: '1Xugfm',
    tags: [ 'search', 'engine' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-24T15:35:00+01:00',
    user: [ 'eklem', 'johshholmedahl' ],
    text: '@johshholmedahl Nei med andre ord =)',
    link: 'http://twitter.com/eklem/status/536890737192075264',
    id: '2CMM13',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-25T14:39:00+01:00',
    user: [ 'eklem' ],
    text: 'Idea: Your life, indexed and available through Norch, NOde seaRCH - http://t.co/pkEs4TW7Xb',
    link: 'http://twitter.com/eklem/status/537239093810176001',
    id: 'ANeTb',
    tags: null,
    links: [ 'http://t.co/pkEs4TW7Xb' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-25T20:34:00+01:00',
    user: [ 'eklem', 'instructables' ],
    text: 'RT @instructables: Open Toys, hack your veggies with 3D Printing http://t.co/yQvigeVdS7 http://t.co/DAZNnK7INO',
    link: 'http://twitter.com/eklem/status/537328381512392706',
    id: '4Gdy4k',
    tags: null,
    links: [ 'http://t.co/yQvigeVdS7', 'http://t.co/DAZNnK7INO' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-25T20:58:00+01:00',
    user: [ 'eklem' ],
    text: 'Anyone that could be interested in a #search solution like this? \nhttp://t.co/8uVxaqW8o6',
    link: 'http://twitter.com/eklem/status/537334627246301184',
    id: '2HCMZb',
    tags: [ 'search' ],
    links: [ 'http://t.co/8uVxaqW8o6' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-25T21:52:00+01:00',
    user: [ 'eklem' ],
    text: 'Sharing. Statistics, so far. Google+(0),LinkedIn(1),Facebook(4),Twitter(5),Hacknews(12),Reddit(30)',
    link: 'http://twitter.com/eklem/status/537348165750767617',
    id: '2fCK33',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-26T09:41:00+01:00',
    user: [ 'eklem', 'inconvergent' ],
    text: '@inconvergent What happened?',
    link: 'http://twitter.com/eklem/status/537526476774838272',
    id: '21HOM0',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-26T15:02:00+01:00',
    user: [ 'eklem', 'tikikate', 'RubenBolling' ],
    text: 'RT @tikikate: "Fart-sound App Maker"\n\nMT @RubenBolling\n\nRichard Scarry\'s Bush 21st Century\n\nhttp://t.co/LswGgcoekL http://t.co/sH7GdG22Dr',
    link: 'http://twitter.com/eklem/status/537607428951773184',
    id: '1fAqcM',
    tags: null,
    links: [ 'http://t.co/LswGgcoekL', 'http://t.co/sH7GdG22Dr' ],
    type: [ 'Twitter' ] },
  { date: '2014-11-27T10:11:00+01:00',
    user: [ 'eklem', 'EinarWilly', 'delveien', 'Presserom' ],
    text: 'RT @EinarWilly: Prøvde ut  nye sykkelfeltet i #Eufemiasgtate  Konklusjon. Funksjonell løsning kun om man finner opp teleportering.  @delveien @Presserom',
    link: 'http://twitter.com/eklem/status/537896398176661504',
    id: '7CGcR',
    tags: [ 'eufemiasgtate' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-11-27T10:59:00+01:00',
    user: [ 'eklem', 'thomax' ],
    text: 'RT @thomax: Bengler does it again! Sjekk denne visualiseringen http://t.co/ZZQKHULORV av klodens fremtid og hvilke tiltak som funker #IPCC #klima',
    link: 'http://twitter.com/eklem/status/537908678322835456',
    id: '2IjiO',
    tags: [ 'ipcc', 'klima' ],
    links: [ 'http://t.co/ZZQKHULORV' ],
    type: [ 'Twitter' ] },
  { date: '2014-12-01T12:58:00+01:00',
    user: [ 'eklem' ],
    text: 'Hvor er undersøkelsen om jeg faktisk ønsker en hel undersøkelse etter et enkelt kjøp på nett? Nok nå...',
    link: 'http://twitter.com/eklem/status/539388155439624193',
    id: '33owhM',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-12-02T15:54:00+01:00',
    user: [ 'eklem' ],
    text: 'Testing Search without search box: Recipe App  – Alpha version  http://t.co/FhD2WyfDsL',
    link: 'http://twitter.com/eklem/status/539794643911602176',
    id: '2vcEPt',
    tags: null,
    links: [ 'http://t.co/FhD2WyfDsL' ],
    type: [ 'Twitter' ] },
  { date: '2014-12-03T16:31:00+01:00',
    user: [ 'eklem', 'ChrSorgjerd' ],
    text: '@ChrSorgjerd Telthusbakken opp fra Maridalsveien!',
    link: 'http://twitter.com/eklem/status/540166423860219905',
    id: '25xHo5',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-12-09T10:55:00+01:00',
    user: [ 'eklem', 'labs_finn_no', 'FINN_no' ],
    text: '@labs_finn_no @FINN_no Jeg skjønner hvorfor folk ikke bruker sikker betaling. Ikke veldig brukervennlig.',
    link: 'http://twitter.com/eklem/status/542256129490059264',
    id: '20iYoY',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-12-09T10:56:00+01:00',
    user: [ 'eklem', 'labs_finn_no', 'FINN_no' ],
    text: '@labs_finn_no @FINN_no 1: Forklaring ut fra teknisk ståsted. 2: Masse tekst istedenfor selvforklarende. 3: Usikker på hva neste trinn er',
    link: 'http://twitter.com/eklem/status/542256551000801280',
    id: '4jIhpJ',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-12-09T20:38:00+01:00',
    user: [ 'eklem' ],
    text: 'Google Play Movies must be Google\'s worst #search experience? Query: "Hobbit" --> Result: "The Dark Knight Rises"',
    link: 'http://twitter.com/eklem/status/542402910881980416',
    id: 'dQ7Jw',
    tags: [ 'search' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-12-13T11:00:00+01:00',
    user: [ 'eklem', 'Reuters' ],
    text: 'RT @Reuters: A surfer drops in on a large wave at Praia do Norte. Editor\'s Choice photos: http://t.co/80WiUujjSm http://t.co/sYqam8FiYp',
    link: 'http://twitter.com/eklem/status/543707071019900928',
    id: '1nw8QH',
    tags: null,
    links: [ 'http://t.co/80WiUujjSm', 'http://t.co/sYqam8FiYp' ],
    type: [ 'Twitter' ] },
  { date: '2014-12-15T09:39:00+01:00',
    user: [ 'eklem', 'anniken', 'johanhal' ],
    text: 'RT @anniken: @johanhal Litt spesielt, i hvert fall. "Lær engelsk! For pensjonister og menn."',
    link: 'http://twitter.com/eklem/status/544411456180338688',
    id: 'p9Vzi',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-12-15T09:52:00+01:00',
    user: [ 'eklem', 'anniken', 'johanhal', 'iacob' ],
    text: '@anniken @johanhal @iacob Min erfaring er at eneste måten å få med alle er å lytte til hva de vil ha og ikke  ikke fordumme/rosa-ifisere',
    link: 'http://twitter.com/eklem/status/544414743508045824',
    id: 'bCibL',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-12-17T10:04:00+01:00',
    user: [ 'eklem', 'IFTTT' ],
    text: '@IFTTT How about a title-row when inserting to Google Drive? Would make this much closer to reality: http://t.co/qfdjmY3m0y #lifeindex',
    link: 'http://twitter.com/eklem/status/545142512869212160',
    id: '3ud5Yd',
    tags: [ 'lifeindex' ],
    links: [ 'http://t.co/qfdjmY3m0y' ],
    type: [ 'Twitter' ] },
  { date: '2014-12-17T10:56:00+01:00',
    user: [ 'eklem', 'Ingeborgborg', 'JenSorensen' ],
    text: 'RT @Ingeborgborg: Denne er veldig god. @JenSorensen http://t.co/KuhaVkEPtk',
    link: 'http://twitter.com/eklem/status/545155461163782144',
    id: 'lkTwz',
    tags: null,
    links: [ 'http://t.co/KuhaVkEPtk' ],
    type: [ 'Twitter' ] },
  { date: '2014-12-17T10:56:00+01:00',
    user: [ 'eklem', 'Gj3rt' ],
    text: 'RT @Gj3rt: Når vi blir gamle ... http://t.co/ggDPbnjuUb',
    link: 'http://twitter.com/eklem/status/545155478435942400',
    id: '2ccpHZ',
    tags: null,
    links: [ 'http://t.co/ggDPbnjuUb' ],
    type: [ 'Twitter' ] },
  { date: '2014-12-19T10:43:00+01:00',
    user: [ 'eklem', 'IFTTT' ],
    text: '@IFTTT Cool! That would bring us one step closer to a service that people will pay for =)',
    link: 'http://twitter.com/eklem/status/545877120871043072',
    id: '3I8QBi',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2014-12-23T12:35:00+01:00',
    user: [ 'eklem', 'holajoan' ],
    text: 'RT @holajoan: I assume somebody has done this before but I just couldn\'t contain. Meet Dron Draper. http://t.co/r88dti3esb',
    link: 'http://twitter.com/eklem/status/547354703336849408',
    id: '4eQ6Vw',
    tags: null,
    links: [ 'http://t.co/r88dti3esb' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-01T20:23:00+01:00',
    user: [ 'eklem' ],
    text: 'Nice chance to join the premiere of Iron Sky The Coming Race + premiere party in Helsinki/Berlin: http://t.co/E1h40z3cgA',
    link: 'http://twitter.com/eklem/status/550734183501692928',
    id: '30FCGi',
    tags: null,
    links: [ 'http://t.co/E1h40z3cgA' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-01T20:30:00+01:00',
    user: [ 'eklem', 'inconvergent' ],
    text: 'RT @inconvergent: different perspective. #generative http://t.co/v5YlhULpo7',
    link: 'http://twitter.com/eklem/status/550735918840422400',
    id: '4eeNC0',
    tags: [ 'generative' ],
    links: [ 'http://t.co/v5YlhULpo7' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-06T12:50:00+01:00',
    user: [ 'eklem', 'terokaukomaa' ],
    text: 'RT @terokaukomaa: Just completed #IronSkyTheComingRace 60 days fight succesfully in #IndieGoGo with 560.949 dollars from 8048 contributors. Huge Thank!',
    link: 'http://twitter.com/eklem/status/552432066378616832',
    id: 'MtbHY',
    tags: [ 'ironskythecomingrace', 'indiegogo' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-01-08T09:54:00+01:00',
    user: [ 'eklem', 'terjerp' ],
    text: 'RT @terjerp: Nei. Muslimer bør ikke kritisere terror fordi de er muslimer. Alle burde kritisere terror fordi vi er mennesker.',
    link: 'http://twitter.com/eklem/status/553112380482519042',
    id: 'tEK3c',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-01-08T12:55:00+01:00',
    user: [ 'eklem', 'maaemo' ],
    text: 'RT @maaemo: We\'re super excited to announce that we\'ll finally be starting our very own farm! https://t.co/hDojBq0ta7',
    link: 'http://twitter.com/eklem/status/553158023582334976',
    id: '3sN9Bs',
    tags: null,
    links: [ 'https://t.co/hDojBq0ta7' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-09T09:49:00+01:00',
    user: [ 'eklem', 'martabreen' ],
    text: 'RT @martabreen: Siri Martinsen, du tar matlysten fra meg.\nhttp://t.co/6Hd456ra4U',
    link: 'http://twitter.com/eklem/status/553473549252689920',
    id: '3QH8Sx',
    tags: null,
    links: [ 'http://t.co/6Hd456ra4U' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-13T10:25:00+01:00',
    user: [ 'eklem', 'jamesrbuk' ],
    text: 'RT @jamesrbuk: "Terrorists must never be allowed to change our way of life": every politician, usually less than 72 hours before trying to change the law.',
    link: 'http://twitter.com/eklem/status/554932319950098432',
    id: '4rwz9a',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-01-13T10:42:00+01:00',
    user: [ 'eklem', 'kimonolabs' ],
    text: 'Great service you\'ve created @kimonolabs ! Best #crawler / #screenscraper I\'ve seen. It\'ll play nice with #Norch http://t.co/O082gCObaU',
    link: 'http://twitter.com/eklem/status/554936503668051968',
    id: '22V7Lf',
    tags: [ 'crawler', 'screenscraper', 'norch' ],
    links: [ 'http://t.co/O082gCObaU' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-13T13:27:00+01:00',
    user: [ 'eklem', 'kimonolabs' ],
    text: '@kimonolabs ... and fit nicely into this idea: http://t.co/Do0RGWQdbr',
    link: 'http://twitter.com/eklem/status/554978003143622657',
    id: '4nXdc7',
    tags: null,
    links: [ 'http://t.co/Do0RGWQdbr' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-13T14:10:00+01:00',
    user: [ 'eklem', 'comperiosearch' ],
    text: 'RT @comperiosearch: Velkommen til Comperio\'s frokostseminar på Hotel Continental 21.januar!  "Digitale helsetjenester og grafteknologi" http://t.co/UkWny2qIKr',
    link: 'http://twitter.com/eklem/status/554988890487275520',
    id: '4aYWc4',
    tags: null,
    links: [ 'http://t.co/UkWny2qIKr' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-15T09:40:00+01:00',
    user: [ 'eklem' ],
    text: 'Enterprise #Search Optimization - #ESO  http://t.co/fqsljnk0Mk',
    link: 'http://twitter.com/eklem/status/555645579846746112',
    id: '1EZrEt',
    tags: [ 'search', 'eso' ],
    links: [ 'http://t.co/fqsljnk0Mk' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-16T12:56:00+01:00',
    user: [ 'eklem', 'kimonolabs' ],
    text: '@kimonolabs An idea for your advanced CSS selector editor?\nhttp://t.co/LiGB8KTz9U',
    link: 'http://twitter.com/eklem/status/556057487557025792',
    id: 'geU6C',
    tags: null,
    links: [ 'http://t.co/LiGB8KTz9U' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-16T13:31:00+01:00',
    user: [ 'eklem', 'IFTTT' ],
    text: '@IFTTT Is it possible to add two different blogs to the #wordpress channel?',
    link: 'http://twitter.com/eklem/status/556066264494637056',
    id: '1TBuJC',
    tags: [ 'wordpress' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-01-16T15:26:00+01:00',
    user: [ 'eklem', 'andersheger', 'ornulfris' ],
    text: '@andersheger @ornulfris Ekte?',
    link: 'http://twitter.com/eklem/status/556095060220338176',
    id: '1Aiwol',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-01-16T15:32:00+01:00',
    user: [ 'eklem', 'andersheger' ],
    text: 'RT @andersheger: Det er nærliggende å tro at BÅDE  Arfan Bhatti og mulla Krekar har politisk vern av Høyre/FrP- regjeringen. http://t.co/QXCHbtIhgM',
    link: 'http://twitter.com/eklem/status/556096677699129344',
    id: 'pBNMM',
    tags: null,
    links: [ 'http://t.co/QXCHbtIhgM' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-16T21:52:00+01:00',
    user: [ 'eklem', 'IFTTT' ],
    text: '@IFTTT Cool, thanks!',
    link: 'http://twitter.com/eklem/status/556192317124915200',
    id: '2sCohB',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-01-17T09:54:00+01:00',
    user: [ 'eklem', 'kimonolabs' ],
    text: '@kimonolabs Will do!',
    link: 'http://twitter.com/eklem/status/556374017427656704',
    id: '1V3orM',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-01-22T10:44:00+01:00',
    user: [ 'eklem', 'appear_in' ],
    text: 'Alt for lite egen produktutvikling i Norge. Telenor Digital er en av de få positive med bl.a. @appear_in http://t.co/SvieQxwzwX Heia-heia!',
    link: 'http://twitter.com/eklem/status/558198459648737280',
    id: '41ABFE',
    tags: null,
    links: [ 'http://t.co/SvieQxwzwX' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-27T14:14:00+01:00',
    user: [ 'eklem', 'appear_in', 'vivaldibrowser' ],
    text: 'RT @appear_in: Congrats to @vivaldibrowser for making an awesome new #browser with #WebRTC support! appear.in works perfectly :) http://t.co/16hVv5bWMt',
    link: 'http://twitter.com/eklem/status/560063260184309761',
    id: '14e7sP',
    tags: [ 'browser', 'webrtc' ],
    links: [ 'http://t.co/16hVv5bWMt' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-28T09:39:00+01:00',
    user: [ 'eklem', 'johshholmedahl', 'eklem', 'marit_ab', 'ErlingSmemo' ],
    text: 'RT @johshholmedahl: Jeg har blogget om mine 7 favorittwebfonter. http://t.co/uyrbYigbLl Utfordrer @eklem @marit_ab og @ErlingSmemo til å gjøre det samme :-) #ux',
    link: 'http://twitter.com/eklem/status/560356369296293888',
    id: '36hRB8',
    tags: [ 'ux' ],
    links: [ 'http://t.co/uyrbYigbLl' ],
    type: [ 'Twitter' ] },
  { date: '2015-01-28T09:39:00+01:00',
    user: [ 'eklem', 'johshholmedahl', 'marit_ab', 'ErlingSmemo' ],
    text: '@johshholmedahl @marit_ab @ErlingSmemo Skal jeg gjøre!',
    link: 'http://twitter.com/eklem/status/560356435830521856',
    id: '2Qiwqy',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-02T09:12:00+01:00',
    user: [ 'eklem', 'twitter' ],
    text: '@twitter just reinvented #bling tag on trends. No more of that please. Too early in the morning. Actually, no time of day that would work.',
    link: 'http://twitter.com/eklem/status/562161704852017152',
    id: '38HUqI',
    tags: [ 'bling' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-05T10:57:00+01:00',
    user: [ 'eklem', 'web_rebels', 'fergiemcdowall' ],
    text: 'RT @web_rebels: A big welcome to our first Norway based speaker; @fergiemcdowall - https://t.co/d4c7fMAUyv',
    link: 'http://twitter.com/eklem/status/563275108232744960',
    id: '1zTthM',
    tags: null,
    links: [ 'https://t.co/d4c7fMAUyv' ],
    type: [ 'Twitter' ] },
  { date: '2015-02-05T12:17:00+01:00',
    user: [ 'eklem', 'elisabethirg' ],
    text: 'RT @elisabethirg: I have for ex /* yep */ behind a rule. This meant that I didn’t understand why the rule was needed, but it apparently is – so don’t delete.',
    link: 'http://twitter.com/eklem/status/563295365475364864',
    id: '18cUFC',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-05T14:03:00+01:00',
    user: [ 'eklem' ],
    text: 'Bra forhold for bølgesurfing på #Unstad rett rund hjørnet? http://t.co/IopXGWXzFK',
    link: 'http://twitter.com/eklem/status/563321963498983424',
    id: '19z97p',
    tags: [ 'unstad' ],
    links: [ 'http://t.co/IopXGWXzFK' ],
    type: [ 'Twitter' ] },
  { date: '2015-02-09T09:07:00+01:00',
    user: [ 'eklem' ],
    text: '#Nerd analysing #beer with cool, new #tools http://t.co/Z3gpsyBM52 #kibana',
    link: 'http://twitter.com/eklem/status/564697130728964096',
    id: '19pJ2s',
    tags: [ 'nerd', 'beer', 'tools', 'kibana' ],
    links: [ 'http://t.co/Z3gpsyBM52' ],
    type: [ 'Twitter' ] },
  { date: '2015-02-13T10:02:00+01:00',
    user: [ 'eklem', 'trygve_lie', 'web_rebels' ],
    text: '@trygve_lie @web_rebels You\'ll soon figure it out =)',
    link: 'http://twitter.com/eklem/status/566160453320339456',
    id: '2rgUQM',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-13T10:05:00+01:00',
    user: [ 'eklem', 'fergiemcdowall' ],
    text: 'Nice presentation of #Norch, #Node #Search. It\'s a search engine written in JavaScript. Thnx @fergiemcdowall http://t.co/wI8z2QsGMW',
    link: 'http://twitter.com/eklem/status/566161222093910016',
    id: '3uto6G',
    tags: [ 'norch', 'node', 'search' ],
    links: [ 'http://t.co/wI8z2QsGMW' ],
    type: [ 'Twitter' ] },
  { date: '2015-02-13T10:41:00+01:00',
    user: [ 'eklem', 'Nervetattoo', 'SlackHQ', 'HipChat' ],
    text: '@Nervetattoo Really happy with @SlackHQ What\'s better with @HipChat other than you saying so =)',
    link: 'http://twitter.com/eklem/status/566170270571192320',
    id: '2fsR3I',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-13T13:47:00+01:00',
    user: [ 'eklem', 'web_rebels' ],
    text: 'Ohaaa, I\'m going to @web_rebels',
    link: 'http://twitter.com/eklem/status/566217096171102208',
    id: '3LiQ3G',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-15T11:11:00+01:00',
    user: [ 'eklem', 'fenilsen', 'SoundsFrench', 'vaartland' ],
    text: 'RT @fenilsen: Et lite blaff av realitetsorientering hos @SoundsFrench i @vaartland, eller bare ufrivillig selvironisk? http://t.co/57tJeUQaDL',
    link: 'http://twitter.com/eklem/status/566902507218419712',
    id: '22UHfp',
    tags: null,
    links: [ 'http://t.co/57tJeUQaDL' ],
    type: [ 'Twitter' ] },
  { date: '2015-02-16T12:33:00+01:00',
    user: [ 'eklem', 'web_rebels', 'IFTTT', 'googledrive' ],
    text: 'Just added a proposal for Open Mic Night at @web_rebels. Personal #Search based on #Norch @IFTTT and #Google @googledrive Fingers crossed!',
    link: 'http://twitter.com/eklem/status/567285675708674048',
    id: '5Q40F',
    tags: [ 'search', 'norch', 'google' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-16T12:57:00+01:00',
    user: [ 'eklem', 'Ingvild_S', 'oslokommune' ],
    text: 'RT @Ingvild_S: I dag har @oslokommune redusert antall sider på nett med 90%. Imponert over organisasjonen, og veldig glad for å ha vært med på reisen.',
    link: 'http://twitter.com/eklem/status/567291733143486464',
    id: '4b6VUM',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-19T13:46:00+01:00',
    user: [ 'eklem', 'kimonolabs' ],
    text: '@kimonolabs App builder just freezes my browsers. Both Firefox and Chrome. Any suggestion?',
    link: 'http://twitter.com/eklem/status/568391179479478272',
    id: 'Mwner',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-24T09:54:00+01:00',
    user: [ 'eklem' ],
    text: 'Media er avslørt: http://t.co/5b3EldQX23',
    link: 'http://twitter.com/eklem/status/570144846289248256',
    id: '1qe86L',
    tags: null,
    links: [ 'http://t.co/5b3EldQX23' ],
    type: [ 'Twitter' ] },
  { date: '2015-02-24T10:01:00+01:00',
    user: [ 'eklem', 'Bignate435' ],
    text: '#Node on #Raspbian through a .deb package repository\nhttps://t.co/IBfb2G4eox Works like a charm! Thanks to @Bignate435',
    link: 'http://twitter.com/eklem/status/570146377679949824',
    id: '2ZfvHH',
    tags: [ 'node', 'raspbian' ],
    links: [ 'https://t.co/IBfb2G4eox' ],
    type: [ 'Twitter' ] },
  { date: '2015-02-24T10:20:00+01:00',
    user: [ 'eklem', 'kimonolabs' ],
    text: '@kimonolabs I get it working in Chrome after waiting  a couple of minutes. Firefox ( Beta) just freezes. http://t.co/JtQFiouKtv',
    link: 'http://twitter.com/eklem/status/570151276937396225',
    id: '2hzOy8',
    tags: null,
    links: [ 'http://t.co/JtQFiouKtv' ],
    type: [ 'Twitter' ] },
  { date: '2015-02-25T09:16:00+01:00',
    user: [ 'eklem', 'kimonolabs' ],
    text: '@kimonolabs Actually, none of them work. Chrome Version 40.0.2214.111 (64-bit)  and Firefox 35.0.1 (OSX 10.10.2). Never get to create an app',
    link: 'http://twitter.com/eklem/status/570497466061209600',
    id: '1XN26e',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-02-26T21:28:00+01:00',
    user: [ 'eklem' ],
    text: 'Ohaa! Personal #Search, here we come! https://t.co/xfVQeZNpoM',
    link: 'http://twitter.com/eklem/status/571044161048203265',
    id: 'XcdJB',
    tags: [ 'search' ],
    links: [ 'https://t.co/xfVQeZNpoM' ],
    type: [ 'Twitter' ] },
  { date: '2015-03-03T21:22:00+01:00',
    user: [ 'eklem', 'mhmoeller', 'rauschma', 'eranhammer' ],
    text: '@mhmoeller @rauschma I just re-tweeted. Nice read, thanks @eranhammer',
    link: 'http://twitter.com/eklem/status/572854602317496320',
    id: 'JoR3q',
    tags: null,
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-03-07T14:20:00+01:00',
    user: [ 'eklem' ],
    text: 'Auto-indexing set up, and first version of Personal #Search running. Blog post later. #RaspberryPi2 works great!',
    link: 'http://twitter.com/eklem/status/574197935027650560',
    id: '20I2XR',
    tags: [ 'search', 'raspberrypi2' ],
    links: null,
    type: [ 'Twitter' ] },
  { date: '2015-03-10T22:40:00+01:00',
    user: [ 'eklem' ],
    text: 'Personal Search comming along nicely. Check out the search information model and what the future holds! http://t.co/Rl5ekiSNV0',
    link: 'http://twitter.com/eklem/status/575410876435464192',
    id: '1XeKrT',
    tags: null,
    links: [ 'http://t.co/Rl5ekiSNV0' ],
    type: [ 'Twitter' ] },
  { date: '2015-03-10T22:50:00+01:00',
    user: [ 'eklem', 'web_rebels' ],
    text: '@web_rebels Personal search on schedule: http://t.co/yQlQFxGRA1',
    link: 'http://twitter.com/eklem/status/575413353696395265',
    id: '3p9g6L',
    tags: null,
    links: [ 'http://t.co/yQlQFxGRA1' ],
    type: [ 'Twitter' ] } ]
eklem commented 9 years ago

Problem will be smaller with this enhancment, but it won't go away: https://github.com/fergiemcdowall/search-index/issues/78

fergiemcdowall commented 9 years ago

Ahh- yes, it would appear that docs arent being deleted as you would expect, and that the test suite is not picking this up

fergiemcdowall commented 9 years ago

There was a gap in test coverage which had allowed deletion to become broken. This should now be fixed

fergiemcdowall commented 9 years ago

Going to take the chance and close- reopen if stuff is still broken

eklem commented 9 years ago

Thanks @fergiemcdowall. Will test!

eklem commented 9 years ago

Tested, works perfectly!