East Asian mp3s not syncing to DPUL

hackartisan commented 6 months ago

A set of MP3s were added to Figgy on Tuesday (2 days ago), under the "East Asian Library Tables of Contents" collection. See items 2 through 25 under: https://figgy.princeton.edu/?f%5Bmember_of_collection_titles_ssim%5D%5B%5D=East+Asian+Library+Tables+of+Contents&per_page=50&q=&sort=created_at_dtsi+desc

(Alternately, limit those results to audio file type)

These should be available under https://dpul.princeton.edu/ealtoc but they are not. Other items added to this collection have synced correctly.

Additional info: The access for the MP3s is set to Princeton-only.

reported by @tventimi

hackartisan commented 6 months ago

Here is the relevant error in honeybadger: https://app.honeybadger.io/projects/51733/faults/106005729

hackartisan commented 6 months ago

hackartisan commented 6 months ago

A head request to figgy for the manifest is timing out?

think about whether we should monitor this.

hackartisan commented 5 months ago

re: head request to manifest timing out. @tpendragon noted that even cached manifests can take a long time to load due to permission-checking.

tpendragon commented 5 months ago

Missing DPUL document IDS:

["808aa225-1f7a-4da9-a31c-b89483cc5c48",
 "f86ab145-f8f8-43c7-8460-b5f7400e4005",
 "56c2dc21-9478-405b-98dc-eb98e2c9c7b3",
 "22c6a6ac-de6c-486f-a848-7659af98747f",
 "7ec25438-2f8a-4a4c-b7ff-f0ec5505067e",
 "174834c7-0ba8-41a6-880c-680c52a502b5",
 "88d4d854-0084-44bd-9959-125c62d16186",
 "e541b0a2-a9a5-4156-b86e-7eaff9c427a3",
 "ffa5bf94-e588-4d90-8e61-e0728352e8b9",
 "3d955684-1b03-4a67-827c-b920802f8404",
 "8d704b8a-6cf2-4a92-9095-ec2c77ad22df",
 "1d89b2e4-81e7-41f5-821a-ddccf03f75d8",
 "5344efb6-2ae2-4f68-bc58-b27b08748846",
 "babc3840-c182-46a3-bcc3-f634ec4fbcf6",
 "ceed9ac6-8279-4483-ac43-926a1df27912",
 "1a533e65-af8c-42f5-bbcc-2e8d951fa006",
 "76e2d55a-2950-4a4d-a0cc-300eb4e8824d",
 "e4353eec-ff2f-4f8a-8a3e-9cea466cb75f",
 "a3b0e530-200b-4ea7-a3c6-561493217804",
 "9ab03dca-14bd-431c-ab3a-902d19cb4ac5",
 "eaab6aff-1a83-4694-b817-a66adf9a0f4d",
 "0e1c7500-cc38-429e-9c28-336d0b73f72d",
 "3b850ae4-08be-4b85-bbaa-2d88b00ec740",
 "5d9633bc-e4f3-40b7-ad0d-1c6f8b34c58a",
 "7f022a36-fe2a-4553-b560-acbc745e9112",
 "7c617e34-a827-413b-ada7-151b327bcc70",
 "250c76cc-36e5-46f6-86d6-72650d142136",
 "62d26ca4-afec-4660-84b8-711191e39d3f",
 "e51fd1c8-905d-473b-bc5d-b6f1f47d57b4",
 "d78a937e-d98b-492d-a8df-85bbc2199217",
 "e6f60ee2-f69f-4702-ba77-84239ef11cfe",
 "7c017e6f-4636-49c2-a483-06d4ee56f341",
 "f032c4a0-ee76-4244-91fc-474d1730c4d9",
 "0d2a86f1-026c-469f-b068-d7c6f4b666a9",
 "1bc57bcd-003a-4bbe-8ae8-c0259cfa55a5",
 "2f6548f0-65ec-4595-b131-35b283d1e3fa",
 "e768631c-ad84-4fe9-a9d9-ce8fa33c1705",
 "4e3ff8d6-32d6-4e77-91c6-6ea88a7c4009",
 "03d6518d-d46f-42ba-91ce-7ef5c2c0557f",
 "8b01e2bf-0283-4ac9-8a22-d9c80ab1a5f5",
 "422b3755-4b40-464b-a0e1-8456551e30a5",
 "6fd3598e-b485-4b2c-8dac-afb44f73b403",
 "35a064c6-3f19-4e45-80c1-6c75fb9ba652",
 "d96c6b5a-9655-4e6d-bb5d-b4a9d5033366",
 "7f8127c7-18d5-47d2-9686-f55d86898a2a",
 "f5b46fed-3a38-44df-b111-3e11857bf063",
 "2f6ba86f-f6e1-4dd2-8357-9a4786e75dd1",
 "7d06b342-7ac4-4909-9dbb-aa4363e05af7",
 "6e50a22f-af31-491f-8a1d-5d5968694ebd",
 "f8919a51-63e3-44b2-b697-176fccac594c",
 "2d6bc5c3-243b-4003-b319-f20b6625a5ef",
 "9d7405a6-54d0-4550-a504-b2ec8d1fe637",
 "64dd26ac-2baf-4d28-928c-f0305458aa62",
 "46d064f8-66ee-4345-8985-627560dae830",
 "8efc9de0-45a0-46bf-afd5-859b597d8948",
 "1d0861c0-9d9f-4509-b3da-d60276bbabe2",
 "7c8bb371-0723-4f8d-a970-61ab3ff54615",
 "f7858d9b-8ba8-4f6c-a6c6-18574981bd20",
 "2a90f782-aa93-4274-a172-f0168f53d615",
 "01091585-5006-4429-ba55-755315143b10",
 "52e917c0-1ef8-4b4e-9a5d-538149e980ce",
 "629a6ed8-cb37-408f-8afa-b445fe2a7037",
 "8098924c-3700-4617-a51e-aa97c029fb77",
 "35eb779c-6129-4693-9778-9707f6dcec29",
 "b5d95f26-eb39-4e0c-be41-3b2160701f44",
 "c587e7b4-0c56-437b-a1d1-19296b68fcae",
 "56d1113f-1c01-4a51-8340-ddaabbff2fdb",
 "59d1b26a-3cb2-48da-b929-8fe03e84f6b6",
 "514d2eed-8d64-438b-94bd-e0f895bbdd95",
 "cf0f6ed9-fddb-4d70-adca-d8379cb9132b",
 "da1ad913-3e75-420f-bb67-fc38651c9755",
 "e9ffc1f2-8771-4bc8-92a4-72e5e8f6f819",
 "c9808d6f-0d8a-4de0-b15d-9a8d229cd11b",
 "8db6c223-1930-430d-8b39-ced15f3f81ab",
 "64085a7f-408c-46ac-a219-954ecd31da60",
 "aa0c91cb-23e5-4849-a39b-072060b0840b",
 "95dad22d-f5eb-40ab-9f34-fd2954018a09",
 "8b567e60-6f95-444a-880f-014d82147a77",
 "dee3bd65-08e3-4785-9dfa-cc9aa5f3c8fa",
 "2c734d64-3ed1-4e7a-9482-7151134dada2",
 "0b2e46cd-e033-45d0-b248-aadd6ffd9acd",
 "4bb3ce68-e044-4fb5-b9bc-a5dc3fb1890f",
 "c651d7fd-1530-4ff2-ab92-40ce9e9e9527",
 "27a82baf-fafc-4a07-88a1-6d83dd4d6ced",
 "54180a3a-7bf0-4381-a8a8-42c49f5f4739",
 "5a9ef7a6-c07b-41ed-a7b8-69b654f13b59",
 "a763ca2a-37ce-4ab7-afd2-040694d5e1ca",
 "0bde5efb-785e-48d4-a5c2-039e4d63d1a0",
 "1c3d7987-b818-4108-a19e-c098116752c6",
 "6a9dc36d-e185-44ed-819b-4ac248f7f637",
 "02276389-2510-4d03-8887-f059255294c1",
 "7bd9520b-6e7e-4fbe-838e-c8f12925af0f",
 "c8e433c9-c1b1-4264-a415-db3e51b4e03b",
 "19a0b18e-b4a7-49fd-a076-2373f7cadbdb",
 "71525bf0-c6a4-4613-94d7-ded978b7b5a4",
 "07ef966d-585f-4ccb-9d2e-d751a19ffc31",
 "14fbc5c1-5200-4ed8-8aa7-a600b0b0155e",
 "8a258040-07f1-473b-9614-d97f8be2ae80",
 "3da6734b-66b1-43e5-89a7-ba2de13a3f95",
 "309d6eb4-0433-4551-b938-ae8361181015",
 "bc44399d-c27b-4add-b01a-2a820feac273",
 "2046113b-2533-43c5-8004-7372b84e1325",
 "ccfaf8e8-00d7-4a2c-81c5-84dbd8f46fa1",
 "2d294ca0-0c1f-4b84-9690-0c717fcff752",
 "8856a5d9-cf78-475b-b533-ccf6fd45067b",
 "5a4a6df3-2279-4e67-9355-cf190c35e66a",
 "b834c132-d055-46b9-a6c7-7f90fcd3da24",
 "ca488b72-1f6f-4b95-90e0-699a4143109a",
 "d99a1e55-7033-4187-b87a-097ce5660be0",
 "e8d8d336-bed2-470e-8073-07591f4070d9",
 "03abc9e7-3167-49e7-9b6f-e8141cc9c1d4",
 "d0a8ac32-dd80-4182-a278-243b07ca3baa",
 "53ccb2b6-481b-4b94-bb5e-9c927808fb7f",
 "de8cdaad-eb81-4b40-8338-f23dae42fe6e",
 "0671ecec-96dd-4a22-bd35-fc9f5bf9b97e",
 "674977f6-1509-4be0-b0de-79ab4c615522",
 "ae1caf57-fbb2-4dbf-8cda-a2ab597cc63a",
 "8c23d3d2-93d3-4263-9868-aaa33633e7a9",
 "f547bf3c-23d2-4e7d-b40a-cc967af4c004",
 "bcd21a0d-36cb-455e-83c8-69e5a92090d6",
 "59de52de-7453-4fe3-9300-fbe3cd219ee5",
 "505ec1ab-4fc4-4187-85f5-9b818cc53aff",
 "894a1146-5de6-419a-99c0-021326bf6fb2",
 "608365e6-56e9-40ce-a8f4-1cef7d373173",
 "f20f2beb-eb5b-41a8-aa94-7612b7dc345c",
 "babd36ac-71ef-4e8c-89f9-4da78a47da02",
 "865ed010-6ae3-4a95-a5b4-96142f021b70",
 "5fecc9bd-31eb-4e0a-93a6-6069f040090d",
 "47d480bd-cd31-4858-b4cd-0a2f3d3007be",
 "aa76c4e9-938c-42b7-976c-0a3ec69dd393",
 "3d54455e-c823-4d97-94c5-985f7da0b41a",
 "493d5f33-f524-464b-8d7b-5819dbfe612f",
 "e9a05c76-8de5-40e9-b378-ae9d715c1679",
 "d67a3710-72d2-4811-9a58-911719e90863",
 "c402fb7a-8987-4d76-ac25-ef735b200786",
 "be9a09de-5bbd-4064-952f-82d659a60e8b",
 "7a51a965-8e6a-4ab6-b564-0694455f3947",
 "ab84fe85-47ef-47db-a3b5-1bb0064da41f",
 "6603bca4-605b-4857-a781-6b489cddbc66",
 "b2759c28-1d0e-4238-981f-bf4c937e2031",
 "23020366-e473-4a89-935f-2124bce5f3c0",
 "6ed01634-45b9-4794-9389-2e2976d8ad47",
 "9fc73488-d766-4985-bf44-4181c1467f00",
 "6892bd22-d09e-4b56-b7d8-d7a4c0ae0023",
 "8161375f-e956-4c2d-ac08-e1e9f770ca8b",
 "f0ba317f-7021-4141-bc60-51cd5dc87f55",
 "33876d18-a805-4246-97bd-b7ca66f13b2a",
 "63d7d954-27a5-4416-a680-cbb1143c0791",
 "c518dc7c-0bfd-4870-992f-084adbc7c8cb",
 "37764ee5-c16c-493e-8dd2-d52ab4aa85fd",
 "625a5917-c244-46b8-a98e-9458a810aa86",
 "7d6049ae-9b87-48d5-979c-1944859e5a73",
 "569b4978-9c75-4464-98c8-b1ca4b3eb308",
 "e74a2270-3fed-4b72-bd75-1285faec3f03",
 "6400006d-b0e2-46cd-85a1-24d89ab66457",
 "47f3c030-0845-41f0-9531-ce611ee63250",
 "6007d21e-5781-4cfd-9fba-dc47149a5bf1",
 "6da63c94-2ccb-44f1-b097-70cf38b02759",
 "8eca94cb-d750-4443-9b8f-feab23035189",
 "0a7fa331-7aa9-44a7-8f3a-eebb33e091e8",
 "c9a134e0-f0c3-4e97-80a7-1e6a0195bb7d",
 "e916e6ed-22f0-43fd-bd1b-d736d92c7981",
 "8c3a1e10-f752-4038-b986-cc69938d8db3",
 "66b09cf9-36c3-4872-942a-7f55b5e226f2",
 "9ed57cb6-c9ab-43cd-88f7-fc4a4f898f97",
 "c37ffbfc-b584-420f-96b2-9f07c5e66fc4",
 "8d53beda-1f4c-4f6f-af3b-7f88100d1278",
 "4cba4062-6a3e-4c4a-81a9-7ec0db2b6420",
 "4ba866a0-287a-4267-9b0c-3de903dafee7",
 "dc84f4f5-1f37-418f-b0e0-3554e4463f09",
 "3c40ee4e-b47e-4ecd-8840-2e52ef7b2a7c",
 "7ffc5929-1381-47da-8e75-edd62299a4d0",
 "3a1fdcf8-a280-425c-9921-a15db31eb655",
 "1dee162b-aded-484d-ad29-3cb2e800da59",
 "30f696e4-c694-4abe-aa5f-225c0ec9ab74",
 "6476807d-e08e-4e57-8e4d-13ca7847a829",
 "a5bb9df4-3b5f-4825-bb5f-7d211dfc3309",
 "deb2f08b-4c10-4a9e-8f15-aed50e0d4dd0",
 "871c2465-8ed9-485a-8d98-b41ea96956c5",
 "f172f3a7-8dd5-4ed9-9fd1-239f95a0fb21",
 "146c013f-b344-40d7-8f23-b7a4a361f87c",
 "c6d47053-6e67-44df-b554-32ef7889502f",
 "57c503f4-fb28-47ff-a9c1-d2b77363489a",
 "aaaef793-9216-40c2-8efd-6b79d635cbd0",
 "68edeea9-b32d-4144-82cf-40bc934cc69d",
 "280619aa-6fd3-451c-a067-789bb5ff9d07",
 "62072c04-dc53-433a-8f18-90b3f6e5c4b3",
 "8a3a170d-428a-4423-88ae-fd49cb368e03",
 "53c7fe57-2be5-4478-b5b9-fb8ad5dd16b1",
 "e5603127-adbb-49af-aaa3-09a8928cd729",
 "a716f472-aec3-479e-8d0b-d46f123b7036",
 "3d75c52e-8eba-4bf6-b81d-61683da1bad2",
 "0597c5ad-0ef2-4d4c-8c5e-47276866b500",
 "bcba91be-3fc2-4a65-ba18-c03dd8952c22",
 "147d49b6-6b78-410d-a1ed-2f4a0021148d",
 "64a5e223-fec4-45f8-afbd-faeb21af43f2",
 "c56d13d5-a65d-40c8-8ed6-9ea4d677ac5a",
 "4c09654f-b477-4d69-82cd-501fd9759d65",
 "e74c2af7-571b-415f-9ba8-8921a725e864",
 "4708281f-734a-41d6-8324-c360b9fde8ac",
 "28281766-a393-4539-bcc7-84968eea0024",
 "44e26a54-168f-4eac-8d5f-bf6e0cd1b6ff",
 "70a0bce8-883a-4657-a6bb-dda2127e527e",
 "3f49f064-fa4f-47ff-b6f6-3e4d68f40af4",
 "dab78b09-1344-4e42-b8ad-827fb63abfeb",
 "627617a0-5108-494b-82a1-9fe5eea976f9",
 "6c0cb908-00ba-4ec5-8b68-84073c932a44",
 "50bf484e-5856-4957-ae19-9f615bf0b9db",
 "3b5f3f58-2a71-404d-acf5-3b8227e4c58d",
 "d14674b9-a4eb-4afc-b624-1f4dffba496d",
 "a66a0ec3-8413-48bd-ba48-44cc7bea0dd4",
 "b57f94f8-68f9-4760-b48e-5ef96d40828a",
 "525fb2ac-a383-4ccc-a5b6-8f07b7af03e8",
 "6730bf8e-5e5a-4940-a11b-281eb3025262",
 "f2440b6f-bd8f-418c-81ff-13bf146f71fd",
 "e8c1295d-715f-4581-b883-2df57bddbb7e",
 "8aa5af1f-b6c9-48d8-8222-8f9634d8896b",
 "712b052f-8b71-4a81-8dfb-07d073c134a8",
 "a3689c0c-be4a-42e0-b37a-dde2ce853426",
 "1ec31160-ca8f-4942-8658-6d7a3573255d",
 "0e2e8879-894c-4ac0-8585-ce43c852d081",
 "3c289f5b-2584-416b-90d0-9b6252897754",
 "8dc1d856-711d-4bf1-94e6-3d14188bba80",
 "ad10ce24-abf7-422a-88fa-818e9c816deb",
 "78b79f89-011c-4de2-bd0b-ff042a51ce52",
 "93926a8f-2275-4796-a81c-a08f9edd607c",
 "d6b6d22b-8447-44f4-84a2-761b4bede84f",
 "90b5a771-2514-4bd4-8882-c38b21a80b2f",
 "9231e080-2ea1-4ac7-9f5d-4db504e3c563",
 "8e47ebd4-32a0-46a9-84ba-a5360b35f91b",
 "e0c9ada3-73ca-42c2-87a3-eabb95b7fdc8",
 "c27e01c8-e0f4-4865-9182-c5a145cca052",
 "79e504c4-70e1-4e18-ad69-ecb99868f587"]

tpendragon commented 5 months ago

We think the reason the counts aren't lining up is that when you reindex the exhibit, some of the children are attached to the collection and thus in the collection manifest, so it indexes two documents in two jobs for each of those cases: one for the resource in the context of its parent, and one for the resource outside of the context of its parent.

If the resource outside the context of its parent goes last, then it ends up non-hidden in the DPUL search results. The order of those jobs are random, which is why that number changes seemingly randomly between reindexes and between staging/production.

We think we can fix this case by removing all children from the collection. We don't know what Tom's doing in his workflows to get children attached to that collection in the first place.

We'll add references to PRs that have attempted to make it impossible to add a collection to a child.

hackartisan commented 5 months ago

Once the task to remove collections from child members is run, we will close this ticket. Future work should be tracked by https://github.com/pulibrary/figgy/issues/4833

hackartisan commented 5 months ago

child members in the collection were updated and the dpul collection appears to be in line with figgy now.

pulibrary / figgy

East Asian mp3s not syncing to DPUL #6329