Delete document via sanity webhook not working

[sarah-j-lancaster]

Following this tutorial I have set up a serverless next function and a sanity webhook: It successfully updates algolia on create and update sanity documents but produces the following error on delete:

[POST] /api/search
11:31:44:74
2022-10-16T22:31:45.739Z    dc39b790-6c28-45f3-aa67-f1df28b87c24    ERROR   Unhandled Promise Rejection     {"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"[object Object]","reason":{"name":"ApiError","message":"null value not allowed for objectID near line:1 column:61","status":400,"transporterStackTrace":[{"request":{"data":"{\"requests\":[{\"action\":\"deleteObject\",\"body\":{\"objectID\":null}},{\"action\":\"deleteObject\",\"body\":{\"objectID\":null}}]}","headers":{"x-algolia-api-key":"*****","x-algolia-application-id":"GOBEQATHZY","content-type":"application/x-www-form-urlencoded"},"method":"POST","url":"https://GOBEQATHZY.algolia.net/1/indexes/sarah_people-first-cms/batch?x-algolia-agent=Algolia%20for%20JavaScript%20(4.14.2)%3B%20Node.js%20(16.16.0)","connectTimeout":2,"responseTimeout":30},"response":{"status":400,"content":"{\"message\":\"null value not allowed for objectID near line:1 column:61\",\"status\":400}","isTimedOut":false},"host":{"protocol":"https","url":"GOBEQATHZY.algolia.net","accept":2},"triesLeft":3}]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: [object Object]","    at process.<anonymous> (file:///var/runtime/index.mjs:1131:17)","    at process.emit (node:events:539:35)","    at emit (node:internal/process/promises:140:20)","    at processPromiseRejections (node:internal/process/promises:274:27)","    at processTicksAndRejections (node:internal/process/task_queues:97:32)"]}
RequestId: f5ef1d0c-7c62-4faa-9c9b-42255d31bbec Error: Runtime exited with error: signal: segmentation fault
Runtime.ExitError

The error mentions null value not allowed for objectID, but for my deleted documents I can see it has a correct object ID in the algolia dashboard (I know its a different ID but this shows objectID is being set by the code and is not null)

If I log the request I can see the object ID

But the logs in algolia show object ID as empty:

[sarah-j-lancaster]

As an update on this, I copied the src code for the indexer into my repo and removed the all the delete code, this sorted out some issues I was also having with this empty delete request firing on create and update, however for the actual delete operation the simplest solution was to create a seperate serverless fn + sanity GROQ webhook in my next app for the delete and interface directly with the algolia JS client (delete is a simple operation, this was quicker for me than debugging the package)

[sarah-j-lancaster]

More detailed information about my solution: I have two serverless functions in my api folder (i'm using Next 12) search.ts contains the same code referenced here but rather than using indexer from 'sanity-algolia' it is importing indexer from an internal util i've created which copied the source code from the package and removed the code dealing with deleting records (utils/indexer.ts)

import { SanityDocumentStub, SanityClient } from "@sanity/client";
import { SearchIndex } from "algoliasearch";

export type AlgoliaRecord = Readonly<Record<string, any>>;

export interface SerializeFunction {
  (document: SanityDocumentStub): AlgoliaRecord;
}

export interface VisiblityFunction {
  (document: SanityDocumentStub): boolean;
}

export type WebhookBody = {
  ids: {
    created: string[];
    updated: string[];
  };
};

type TypeConfig = {
  index: SearchIndex;
  projection?: string;
};

type IndexMap = {
  [key: string]: TypeConfig;
};

export const sleep = (ms: number) => {
  return new Promise((resolve) => setTimeout(resolve, ms));
};

// Properties that always should exist (only objectID is strictly needed from Algolia)
export const standardValues = (doc: SanityDocumentStub) => {
  return {
    objectID: doc._id,
    type: doc._type,
    rev: doc._rev,
  };
};

export const getAllRecords = async (index: SearchIndex) => {
  let hits: AlgoliaRecord[] = [];
  return index
    .browseObjects({
      batch: (objects) => (hits = hits.concat(objects)),
    })
    .then(() => hits);
};

export const indexMapProjection = (indexMap: IndexMap): string => {
  const types = Object.keys(indexMap);
  const res = `{
    _id,
    _type,
    _rev,
    ${types
      .map((t) => `_type == "${t}" => ${indexMap[t].projection || "{...}"}`)
      .join(",\n  ")}
  }`;
  return res;
};

export const indexer = (
  typeIndexMap: IndexMap,
  // Defines how the transformation from Sanity document to Algolia record is
  // performed. Must return an AlgoliaRecord for every input. Inputs are only
  // those Sanity document types declared as keys in `typeIndexMap`.
  serializer: SerializeFunction
  // Optionally provide logic for which documents should be visible or not.
  // Useful if your documents have a isHidden or isIndexed property or similar
) => {
  const transform = async (documents: SanityDocumentStub[]) => {
    const records: AlgoliaRecord[] = await Promise.all(
      documents.map(async (document: SanityDocumentStub) => {
        return Object.assign(
          standardValues(document),
          await serializer(document)
        );
      })
    );
    return records;
  };

  // Syncs the Sanity documents represented by the ids in the WebhookBody to
  // Algolia via the `typeIndexMap` and `serializer`
  const webhookSync = async (client: SanityClient, body: WebhookBody) => {
    // Sleep a bit to make sure Sanity query engine is caught up to mutation
    // changes we are responding to.
    await sleep(2000);

    const { created = [], updated = [] } = body.ids;

    // Query Sanity for more information
    //
    // Fetch the full objects that we are probably going to index in Algolia. Some
    // of these might get filtered out later by the optional visibility function.

    const createdAndUpdated = created.concat(updated);
    if (createdAndUpdated.length > 0) {
      const query = `* [(_id in $created || _id in $updated) && _type in $types] ${indexMapProjection(
        typeIndexMap
      )}`;
      const docs: SanityDocumentStub[] = await client.fetch(query, {
        created,
        updated,
        types: Object.keys(typeIndexMap),
      });
      console.log(docs);
      const recordsToSave = await transform(docs);

      if (recordsToSave.length > 0) {
        for (const type in typeIndexMap) {
          await typeIndexMap[type].index.saveObjects(
            recordsToSave.filter((r) => r.type === type)
          );
        }
      }
    }
  };

  return { transform, webhookSync };
};

Then inside the delete serverless (delete.ts) I am interacting directly with the algoliaSearch client, luckily deleting is easy, this is the code:

import type { NextApiRequest, NextApiResponse } from "next";
import algoliasearch from "algoliasearch";
import sanityClient from "@sanity/client";
import { isValidSignature, SIGNATURE_HEADER_NAME } from "@sanity/webhook";

const algolia = algoliasearch(
  //@ts-ignore
  process.env.ALGOLIA_APPLICATION_ID,
  process.env.ALGOLIA_ADMIN_KEY
);

const sanity = sanityClient({
  projectId: "ur-project-id",
  dataset: "production",
  // If your dataset is private you need to add a read token.
  // You can mint one at https://manage.sanity.io/,
  token: process.env.SANITY_API_TOKEN,
  apiVersion: "2021-03-25",
  useCdn: false,
});

export const config = {
  api: {
    bodyParser: false,
  },
};

async function readBody(readable: any) {
  const chunks = [];
  for await (const chunk of readable) {
    chunks.push(typeof chunk === "string" ? Buffer.from(chunk) : chunk);
  }
  return Buffer.concat(chunks).toString("utf8");
}

const handler = async (req: NextApiRequest, res: NextApiResponse) => {
  // Tip: Its good practice to include a shared secret in your webhook URLs and
  // validate it before proceeding with webhook handling. Omitted in this short
  // example.
  const secret = process.env.SANITY_WEBHOOK_DELETE_TOKEN as string;
  const signature = req.headers[SIGNATURE_HEADER_NAME] as string;
  const body = await readBody(req); // Read the body into a string
  if (!isValidSignature(body, signature, secret)) {
    res.status(401).json({ success: false, message: "Invalid signature" });
    return;
  }

  if (req.headers["content-type"] !== "application/json") {
    res.status(400);
    res.json({ message: "Bad request" });
    return;
  }
  // Configure this to match an existing Algolia index name
  const algoliaIndex = algolia.initIndex("ur-algolia-index");

  const jsonBody = JSON.parse(body);
  const deleteResponse = await algoliaIndex.deleteObject(jsonBody._id);
  return res.status(200).json({ deleteResponse });
};

export default handler;

and to trigger this i've set up a separate delete webhook on sanity:

[Mehoff]

@sarah-lancaster-springload Thank you so much with sharing more information on the topic even though I've deleted my message asking for it 😂 If I will need more info - I know who to ask :)

[CassadyBlake]

Was having the same issue on sanity-algolia version: 1.0.2.

After upgrading to 1.1.0 the webhook started working for deletions. 🎉