search

algolia / firestore-algolia-search

Apache License 2.0
108 stars 33 forks source link

Reindexing fails when updating indexable fields #214

Open jakelaws1 opened 3 weeks ago

jakelaws1 commented 3 weeks ago

Steps to reproduce:

  1. setup the firebase extension with indexable fields
    • in my example that was id,displayName,editorialSummary,formattedAddress,types,location
  2. update the extension adding new indexable fields
    • added: photoReferences,photos (example data in errors below)
    • I am not sure if the fields I am adding is making them item above 10kb and that's why it's failing? And then it can't recover when I remove it? Regardless I think the error messaging makes this impossible to tell.
    • The interesting thing here is some items do get updated with these new fields (hence my hypothesis above)
  3. After receiving the below error I try to remove the photos field as it's large and it continues to fail.
  4. This leaves the extension in a perpetual "Runtime Status: Processing" state

    Error from 3:

    "{"name":"ApiError","message":"Error: Unhandled error\n at entryFromArgs (/workspace/node_modules/firebase-functions/lib/logger/index.js:130:19)\n at Object.error (/workspace/node_modules/firebase-functions/lib/logger/index.js:116:11)\n at /workspace/node_modules/firebase-functions/lib/common/providers/tasks.js:91:24\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)","status":429,"transporterStackTrace":[{"request":{"data":"{\"requests\":[{\"action\":\"addObject\",\"body\":{\"objectID\":\"ChIJ-0huSkVgLxMRwyDaaXfRhzI\",\"path\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI\",\"id\":\"ChIJ-0huSkVgLxMRwyDaaXfRhzI\",\"displayName\":\"Da Francesco\",\"editorialSummary\":{\"text\":\"Simple, enduring eatery luring locals with its Roman pizza & pasta dishes, plus hearty main courses.\",\"languageCode\":\"en\"},\"formattedAddress\":\"Piazza del Fico, 29, 00186 Roma RM, Italy\",\"types\":[\"pizza_restaurant\",\"italian_restaurant\",\"restaurant\",\"food\",\"point_of_interest\",\"establishment\"],\"location\":{\"latitude\":41.8993349,\"longitude\":12.470391699999999},\"photos\":[{\"authorAttributions\":[{\"displayName\":\"Da Francesco\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjWI7PpDjF2vY1FXWIsabySt3ZJ1APSmDuXQ7zMHuxgdYr2S6VY=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/115896918744180276666\"}],\"widthPx\":2000,\"heightPx\":1331,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZlHhw8eBKlGYEZyTGazToGI2z6A7g_VLtE8pTM6_ssteX-4jW3nDPgkwRqHMTldlvzmb8iXikRuvK5uIX24eLtoAKoYAnn7hVo-dJpkswsySMyjuyoP_syLg6HQaO9SjPW0KLaxjkIrNfCIuigmvq0umlUuk7t6ASeA\"},{\"authorAttributions\":[{\"displayName\":\"Da Francesco\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjWI7PpDjF2vY1FXWIsabySt3ZJ1APSmDuXQ7zMHuxgdYr2S6VY=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/115896918744180276666\"}],\"widthPx\":1270,\"heightPx\":845,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZmaeeC7urkB5-WyZUnMnMfBviZceke7YknXvTrx3T317HUYULeaiggIriy7VYTNP5Jnsr_VBKq1Pqwui347fSMweBkrSSFfFoY5q-7ADflw1UqU9APXpYxWMjTt4NG_ikAgaeUfbASKzbQzxXxiQI12y7w_pfq8u3zX\"},{\"authorAttributions\":[{\"displayName\":\"Kevin Lagda\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjVk27VggxSUzklVLWRQUmwCMq8twhcALVDcTgAbYRfhJC7avRE=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/118352636820860583935\"}],\"widthPx\":4032,\"heightPx\":3024,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZlDyqTFu1B64VrAZ37yzDBZk7VWhqBoQ66V-pljgJ44iKHWKslE_A4vxm9tG9lEf47zN7wLaX8gKsk0YWUBWdgZF2d2MyYmdrdUBNV8WynpXRko793fZiI6fQyIcQpm52V7D5O_Yhm41S1hVIR2JFww4wkWBJeSyoE\"},{\"authorAttributions\":[{\"displayName\":\"Nicolle Crawford\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjUjkOcUSa3T1AigZT0Jy0tx6O_wfNW_upiU9wXp7gLny3KQd_b6=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/114560313823034183781\"}],\"widthPx\":4032,\"heightPx\":3024,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZnLHchs_68HJCRBaGGiwGitQ3fpLGS7TFO2INn1ZqoycLWlSVovb3rQ6kOy5RuODNsZqm37pfmWmedrF5s2guY3RjDPG-cTgq36fM9xrGPUrdZb1mPFwXmJKVIFjlQ2OrcdAXZx5mK0hQjar3t8GvbfZpLU8xQoyfFP\"},{\"authorAttributions\":[{\"displayName\":\"ANTONELLO ALOE VERA\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjVg9-s5mixipf4liOaR80OyzHFBZ4sQ4j8YL5mCLeYhyQuavzFbUg=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/108277862890425361715\"}],\"widthPx\":2600,\"heightPx\":4624,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZk6GG0g-4V_g3ws9oy9l93IFhXMTRfvEc_-1tDJskcY48MVkuQjLsdWGJdFMpiEXKQws3Pur4YeDUJAUUY9z3lWg0yFzTxHEo7FDwuSk3ksR-aNtbf2hfoDCifo07uQlRh1g1ANfhH7NIUcy4OIzasMsPGI4rowz3Fv\"},{\"authorAttributions\":[{\"displayName\":\"Manuel Taglienti\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjWwGbwBkcYgnVcf6bBII9PJZLfWoKYHXuL2dIQilzoGfqIrCdg=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/102574979032345837757\"}],\"widthPx\":4608,\"heightPx\":3456,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZl1YmnI4p88rkI6L2lHNc0VbVE-MUNg_ZFVONNWoRiwcrM2jYW3dpJwHUE2sWh0yLBMNVyln2ZN9pGeux37hPH4CzINPSCtMx81RzClaUTZkrdNwA5wR-ahjteaSFtthWdhXA9hyviQzGy6i20lXfthAmeuXChAMd6O\"},{\"authorAttributions\":[{\"displayName\":\"Nicki Haverstock\",\"photoUri\":\"//lh3.googleusercontent.com/a/ACg8ocJ32vuwGQv6DeNyaIYQcsvresMys_qqXQeu6kZDzqlF2yFeJA=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/105326883601952307576\"}],\"widthPx\":4032,\"heightPx\":3024,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZkY4qcfeNNpix8dH8SpvGEyVlxhlXcUoit1n-sFgSsef_e9v8mk7W7P8Mzp5qO_XWpTgwa0H13R_fdPRaD_t0I4xj_8fc72tyiXlL8RsCARI2wYp9j6WxOpbYqLnJOCkqdZeycOAXE6TEYPvVjCvXXdUepXipTxV_ow\"},{\"authorAttributions\":[{\"displayName\":\"Alberto Morales\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjWWPfb0j6AsqUAID2H8UmPsU4umsyOOCDjpxiqmlZu6BC92tklFkQ=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/106150111003024499283\"}],\"widthPx\":3000,\"heightPx\":4000,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZmWVUEA1GdZ51zfZ5c86zZTUvfk0UU8qG6GrgYxq1JU2II1FRw9x7xjt5ify85gfz9sEaq-nHXV54rU5M6kAA5DjyhK-R-GvJF1zIbLNSXzxXTMsbCVWnitbAfIUoghn21kHBflovorOlzhZ0UzcCvPtucPG-DlAX1k\"},{\"authorAttributions\":[{\"displayName\":\"Carney James Turner\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjW28fYgUmN7B5R77khnKsWzEvgUSpJ3njUh2_2JskR5kBqg0ws=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/108475095501433713028\"}],\"widthPx\":4000,\"heightPx\":3000,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZkQKmmKsULR5cNab5BxZ5ErLXHFOpeBjcp9zgA2KdPote34INqU1EvAhqpHnxj3vYR8lg4VSaeMGkRGWDZdrMJQ9TP54zL-b4wIpqbn9YFUXMZUxrwkBIgbVY48XmnWFLslhbaxlt0wE-SQUZ8DHn99SHxJ8Og4XpZ0\"},{\"authorAttributions\":[{\"displayName\":\"Carney James Turner\",\"photoUri\":\"//lh3.googleusercontent.com/a-/ALV-UjW28fYgUmN7B5R77khnKsWzEvgUSpJ3njUh2_2JskR5kBqg0ws=s100-p-k-no-mo\",\"uri\":\"//maps.google.com/maps/contrib/108475095501433713028\"}],\"widthPx\":4000,\"heightPx\":3000,\"name\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI/photos/AUGGfZlnxlRv1ZCwGPUb4fW9pvcRqcb4TieqloSrl37M8CHx_3udiYeAGc6UpaRpbK158onuQ39a1UqH7x9CLYka1hbo-D5y_KQBLg7nnKci_5ep0oXLs8neY4vqhzVV3nakBQ6A6Hs2w-5Bp0wgH1FAIAjCVwA1JO03cylS\"}],\"photoReferences\":[\"https://lh3.googleusercontent.com/places/ANXAkqHxrr4go0b1jxZG_CgXbJ7aTs6n6LGtZO6EYtSK-NtZMHm0qUVykXt4kfjTgJ0uqipZt0NYqHnnjc67oxKrgaIq7qiUMiW6tRU=s4800-w400-h400\"],\"lastmodified\":{\"_operation\":\"IncrementSet\",\"value\":1717727787016}}}

    Error from 4:

    {"name":"ApiError","message":"Error: Unhandled error\n at entryFromArgs (/workspace/node_modules/firebase-functions/lib/logger/index.js:130:19)\n at Object.error (/workspace/node_modules/firebase-functions/lib/logger/index.js:116:11)\n at /workspace/node_modules/firebase-functions/lib/common/providers/tasks.js:91:24\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)","status":429,"transporterStackTrace":[{"request":{"data":"{\"requests\":[{\"action\":\"addObject\",\"body\":{\"objectID\":\"ChIJ-0huSkVgLxMRwyDaaXfRhzI\",\"path\":\"places/ChIJ-0huSkVgLxMRwyDaaXfRhzI\",\"id\":\"ChIJ-0huSkVgLxMRwyDaaXfRhzI\",\"displayName\":\"Da Francesco\",\"editorialSummary\":{\"text\":\"Simple, enduring eatery luring locals with its Roman pizza & pasta dishes, plus hearty main courses.\",\"languageCode\":\"en\"},\"formattedAddress\":\"Piazza del Fico, 29, 00186 Roma RM, Italy\",\"types\":[\"pizza_restaurant\",\"italian_restaurant\",\"restaurant\",\"food\",\"point_of_interest\",\"establishment\"],\"location\":{\"latitude\":41.8993349,\"longitude\":12.470391699999999},\"photoReferences\":[\"https://lh3.googleusercontent.com/places/ANXAkqHxrr4go0b1jxZG_CgXbJ7aTs6n6LGtZO6EYtSK-NtZMHm0qUVykXt4kfjTgJ0uqipZt0NYqHnnjc67oxKrgaIq7qiUMiW6tRU=s4800-w400-h400\"],\"lastmodified\":{\"_operation\":\"IncrementSet\",\"value\":1717742696089}}},{\"action\":\"addObject\",\"body\":{\"objectID\":\"ChIJ-9cMAqsn0i0RmoPm81-HDy8\",\"path\":\"places/ChIJ-9cMAqsn0i0RmoPm81-HDy8\",\"id\":\"ChIJ-9cMAqsn0i0RmoPm81-HDy8\",\"displayName\":\"Jatiluwih Rice Terraces\",\"editorialSummary\":{\"text\":\"Scenic area with verdant, undulating rice terraces attracting hikers, cyclists & photographers.\",\"languageCode\":\"en\"},\"formattedAddress\":\"Village, Jl. Jatiluwih Kawan No.Desa, Jatiluwih, Kec. Penebel, Kabupaten Tabanan, Bali 82152, Indonesia\",\"types\":[\"tourist_attraction\",\"point_of_interest\",\"establishment\"],\"location\":{\"latitude\":-8.37031,\"longitude\":115.131372},\"photoReferences\":[\"https://lh3.googleusercontent.com/places/ANXAkqE5rDkHzW-QOFzmfgg1mz0tqO501e9iZNivldzZLCSPsgKtbI8GgTwqsE8yjn9T8mj4cppm-CAOI5fOd9Bt4L4YC4kiitiIZks=s4800-w400-h400\"],\"lastmodified\":{\"_operation\":\"IncrementSet\",\"value\":1717742696089}}},

This error continues on for many of the places.

The only other error I am seeing which may be relevant:

iam.googleapis.com

…e.iam.admin.v1.CreateServiceAccount

projects/roamerai-d8e29
Service account ext-firestore-algolia-search already exists within project projects/roamer-8391
smomin commented 1 day ago

Are you seeing the requests coming to Algolia? If so, can you share the logs from Algolia?