Images HttpException 403

[dlewis2017]

Error

When trying to load the congress images, I get the following error:

HttpExceptionWithStatus (HttpException: Invalid statusCode: 403, uri = https://www.congress.gov/img/member/m001226_200.jpg?api_key=API_KEY_SECRET

I'm able to load the images in a web browser fine, and I've been able to load them before in my flutter ios/android application. I tried adding my API key to the url above but that didn't seem to work. I was using cached network image and Image.Network. My API KEY is still valid. Any thoughts on what the issue could be?

Details

ios 17.*

flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.16.5, on macOS 14.2.1 23C71 darwin-arm64, locale en-US)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.2)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2023.1)
[✓] VS Code (version 1.85.1)
[✓] Connected device (2 available)
    ! Error: Browsing on the local area network for iPhone 13 Pro. Ensure the device is unlocked and attached with a cable or associated with the same local area
      network as this Mac.
      The device must be opted into Developer Mode to connect wirelessly. (code -27)
[✓] Network resources

• No issues found!

Repo

1. create a basic flutter application
2. add cached network image library
3. load image with any url received from congress.gov members api endpoint

   CachedNetworkImage(
                                 imageUrl:
                                     '${mageUrl}?api_key=${dotenv.env['CONGRESS_GOV_API_KEY']}',
                                 imageBuilder: (context, imageProvider) => ClipOval(
                                   child: Container(
                                     width: 50,
                                     height: 50,
                                     decoration: BoxDecoration(
                                       image: DecorationImage(
                                         image: imageProvider,
                                         fit: BoxFit.cover,
                                       ),
                                     ),
                                   ),
                                 ),
                               ),

[mnewatloc]

It looks you're like accessing the image via the Congress.gov website, not the API, so no API key is needed. The URL should end at .jpg: https://www.congress.gov/img/member/m001226_200.jpg

[dlewis2017]

I first tried without the API key and I got the same issue @mnewatloc

[mnewatloc]

We've been unable to replicate the problem. Can you perhaps provide screenshots so we can further troubleshoot?

[dlewis2017]

Sure, although, I'm not sure that'll help. The errors just say what I've indicated above:

[squealermusic]

How about this? https://stackoverflow.com/questions/62491230/how-to-handle-404-exception-with-cachednetworkimage-in-flutter

[dlewis2017]

Thanks for pointing that out @squealermusic. I did see that but unfortunately that just talks about handling the 404 gracefully and also when debugging. I do handle the 404, the issue is getting the images to load at all when they actually exist at that endpoint in my app, they once did but now they don't.

[mnewatloc]

Is this still occurring?

[squealermusic]

@dlewis2017 Two thoughts that I got after this easy-to-comprehend article: https://www.webfx.com/web-development/glossary/http-status-codes/what-is-a-304-status-code/#:~:text=In%20other%20words%2C%20a%20200,the%20resource%20in%20its%20cache.

One possibile solution is on your end: "...if a client is consistently receiving 304 status codes for a particular resource, it may indicate that the client’s caching settings need to be adjusted. For example, the client’s cache may be set to expire too frequently, causing it to repeatedly request the same resource from the server. Adjusting the cache expiration settings can help reduce the number of requests and improve performance."

Also keep in mind that we have rate limits: https://github.com/LibraryOfCongress/api.congress.gov/tree/main "Rate Limit The rate limit is set to 1,000 requests per hour.

Limit and Offset By default, the API returns 20 results starting with the first record. The 20 results limit can be adjusted up to 250 results. If the limit is adjusted to be greater than 250 results, only 250 results will be returned. The offset, or the starting record, can also be adjusted to be greater than 0."

I'm probably not telling you anything you don't already know, but it doesn't hurt to lay it out just in case.

We are returning 304 codes immediately after I refresh the page. It's possible that our server settings in regards to caching can use some tweaking, so I'll see if I can find somebody with knowledge in that area to look at the results I'm seeing.

[dlewis2017]

@squealermusic thanks for looking into this and the explanation. I don't seem to be receiving 304 status codes so I don't think it's a cacheing issue, furthermore, the package I'm using is supposed to handle caching well. There is a way to override the default behavior but I'm not sure it's necessary.

And I seem to be getting the issue consistently while developing locally which definitely shouldn't be getting close to 1000 calls per hour.

[mnewatloc]

Has this been resolved?

[petejodo]

If you take the html from the 403 response and render it, you'll see that it's attempting to verify if you're human or not. This is why you can view it on the web while you can't programmatically. The question is if there's a way to pull these images programmatically? Seems like the "human verification" is too broadly applied to every route when it should skip on certain files like congressional photos

[petejodo]

Looking closer at the response, it's a cloudflare challenge via the cf-mitigation header. This would require configuring settings via WAF exceptions

[dlewis2017]

mnewatloc No I switched to using a difference API

petejodo Good catch! Thanks for looking into this