[feat] add a new search option to specify which fields to include or exclude in the returned documents

[fasenderos]

Problem Description

While working with Orama server-side, I would like to be able to exclude certain fields from the search to reduce the amount of data transferred and improve efficiency

Proposed Solution

Similar to MongoDB, I was thinking of a new search option called projection that would be either inclusive (which excludes all other fields) or exclusive (which implies all other fields are included) e.g.:

import { create, insert, remove, search, searchVector } from '@orama/orama'

const db = await create({
  schema: {
    name: 'string',
    description: 'string',
    price: 'number',
    meta: {
      rating: 'number',
    },
  },
})

await insert(db, {
  name: 'Wireless Headphones',
  description: 'Experience immersive sound quality with these noise-cancelling wireless headphones.',
  price: 99.99,
  meta: {
    rating: 4.5,
  },
})

await insert(db, {
  name: 'Smart LED Bulb',
  description: 'Control the lighting in your home with this energy-efficient smart LED bulb, compatible with most smart home systems.',
  price: 24.99,
  meta: {
    rating: 4.3,
  },
})

await insert(db, {
  name: 'Portable Charger',
  description: 'Never run out of power on-the-go with this compact and fast-charging portable charger for your devices.',
  price: 29.99,
  meta: {
    rating: 3.6,
  },
})

Inclusive projection

const searchResult = await search(db, {
  term: 'headphones',
  projection: { name: 1, description: 1 }
})

// result
{
  elapsed: {
    raw: 99512,
    formatted: '99μs',
  },
  hits: [
    {
      id: '41013877-56',
      score: 0.925085832971998432,
      document: {
        name: 'Wireless Headphones',
        description: 'Experience immersive sound quality with these noise-cancelling wireless headphones.'
      }
    }
  ],
  count: 1
}

Exclusive projection

const searchResult = await search(db, {
  term: 'headphones',
  projection: { meta: 0 }
})

// result
{
  elapsed: {
    raw: 99512,
    formatted: '99μs',
  },
  hits: [
    {
      id: '41013877-56',
      score: 0.925085832971998432,
      document: {
        name: 'Wireless Headphones',
        description: 'Experience immersive sound quality with these noise-cancelling wireless headphones.',
        price: 99.99
      }
    }
  ],
  count: 1
}

If you think it might be useful, I'll be happy to open a PR.

Alternatives

No response

Additional Context

No response

[micheleriva]

Hi @fasenderos, thanks for the issue!

In Orama Cloud we have the returning statement that works like this:

const searchResult = await search(db, {
  term: 'headphones',
  returning: [ 'name', 'description']
})

would that work? It wouldn't support exclusive projection though.

[fasenderos]

Hello @micheleriva , thanks for your reply.

Yes, the returning statement is exactly what I'm looking for, and it's perfectly fine without the exclusive projection. Do you think it can be added to the search of @orama/orama ?

[micheleriva]

Yes it can definitely be added to @orama/orama, but I fear it would slow down search... removing properties from objects is expensive. It's worth it when done server-side 'cause you save on payload size

[fasenderos]

removing properties from objects is expensive. It's worth it when done server-side 'cause you save on payload size

Totally agree, doing it client-side wouldn't make any sense and it should be highlighted in the documentation. If you want, I can open a PR

[micheleriva]

yes please!