replaysMike / Binner

Open source parts inventory system for makers, electronics hobby, and professional engineers
https://binner.io
GNU General Public License v3.0
261 stars 41 forks source link

[Bug]: Mouser Order Import only included first 30 items out of 48 #244

Closed dubajj closed 5 months ago

dubajj commented 8 months ago

Binner version

v2.6.0

Operating System

Windows

Describe the bug and the steps to reproduce it

I imported my most recent order from mouser which has 48 line items. Only the first 30 show up in the order import view

Would you like to attach your appsetings.json configuration?

No response

Screenshots or Videos (Optional, but they help!)

No response

Are you able to contribute a PR? (No is ok!)

None

dubajj commented 8 months ago

built from source this evening and in public async Task<IApiResponse> GetOrderAsync(string orderId, Dictionary<string, string>? additionalOptions = null) i can see that all 48 items are returned from mouser, and parsed correctly and passed to return new ApiResponse(results, nameof(MouserApi));

I guess that means its a problem with the front end or dB? Something else is surely going wrong as the response from mousers servers takes about 1/3 a second but the page hangs (in production and debug) for a full minute after that response before it shows anything at all, and then it only shows 30 out of the 48 items

replaysMike commented 8 months ago

Hi @dubajj - I'll look today to see if there's an issue. Sounds to me like a frontend issue, but we haven't pushed any releases in a while as I've been away. I'll take a look at your other tickets as well.

dubajj commented 8 months ago

i joined the discord let me know if i can help out.

shaun-leach commented 7 months ago

Ran into this issue myself. Looks like the problem is that Mouser will occasionally return an error code of Forbidden when calling GetProductDetailsAsync here:

https://github.com/replaysMike/Binner/blob/bed3ea6d7519d5adea05c08f2b848351f348ac46/Binner/Library/Binner.Common/Services/PartService.cs#L356C41-L356C41

shaun-leach commented 7 months ago

It looks like if you retry once or twice it will return without an error code.

replaysMike commented 5 months ago

thanks for the tips, I'm looking into this now and hopefully I can repro.

replaysMike commented 5 months ago

ok I think I understand what the issue is, haven't looked at this code in a long time. Initially I thought Mouser's api may be paginating the data, but there are no such options in their api.

The Mouser api for grabbing order history details doesn't return much data as part of the response, so Binner requests more information from their product info api for each line item in the order. For larger orders they may be throttling the requests after so many are made, and Binner isn't handling that properly (because it's difficult to trigger during testing). This is also likely slow depending on how long each product request takes from their api.

I've asked for more information from Mouser on this behavior, meanwhile I'll try to harden the code to handle this better and refamiliarize myself with the orders api again.

replaysMike commented 5 months ago

This is now fixed and will appear in the v2.6.1 release.