MuckRock's source code - Please report bugs, issues and feature requests to info@muckrock.com
GNU Affero General Public License v3.0
114
stars
22
forks
source link
Large MuckRock file uploads causing Sentry issues #1807
Open
duckduckgrayduck opened 1 year ago
Summary of the problem
https://muckrock.sentry.io/issues/2320001578/?project=2873549&referrer=weekly-email Object not found.
It appears it is trying to reference objects that do not exist on S3. Here are a few of these error events in Moesif:
[https://www.moesif.com/wrap/app/88:1445-198:915/search/events/dates/-1M?evtcf[filtersGroups][0][filters][0][id]=09b193f8&evtcf[filtersGroups][0][filters][0][field]=request.segments.2.raw&evtcf[filtersGroups][0][filters][0][op]=includes&evtcf[filtersGroups][0][filters][0][value]=23829079](https://www.moesif.com/wrap/app/88:1445-198:915/search/events/dates/-1M?evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bid%5D=09b193f8&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=request.segments.2.raw&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bop%5D=includes&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=23829079)
[https://www.moesif.com/wrap/app/88:1445-198:915/search/events/dates/-1M?evtcf[filtersGroups][0][filters][0][id]=09b193f8&evtcf[filtersGroups][0][filters][0][field]=request.segments.2.raw&evtcf[filtersGroups][0][filters][0][op]=includes&evtcf[filtersGroups][0][filters][0][value]=23829043](https://www.moesif.com/wrap/app/88:1445-198:915/search/events/dates/-1M?evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bid%5D=09b193f8&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=request.segments.2.raw&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bop%5D=includes&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=23829043)
[https://www.moesif.com/wrap/app/88:1445-198:915/search/events/dates/-1M?evtcf[filtersGroups][0][filters][0][id]=09b193f8&evtcf[filtersGroups][0][filters][0][field]=request.segments.2.raw&evtcf[filtersGroups][0][filters][0][op]=includes&evtcf[filtersGroups][0][filters][0][value]=23829082](https://www.moesif.com/wrap/app/88:1445-198:915/search/events/dates/-1M?evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bid%5D=09b193f8&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bfield%5D=request.segments.2.raw&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bop%5D=includes&evtcf%5BfiltersGroups%5D%5B0%5D%5Bfilters%5D%5B0%5D%5Bvalue%5D=23829082)
Tracing back they all seem to be linked to the MuckRock DocumentCloud account, which means it is coming from a MuckRock request.
We traced it back to this request, which includes over 2,000 files. https://www.muckrock.com/foi/sutter-county-3073/2023-sb1421sb16-request-sutter-county-district-attorney-142186/
For large responsive requests it appears we probably need to break up the uploads of large responsive sets to mitigate this error.