search

YourITGroup / MemberListView

Back-Office Member Management for Umbraco 7 & 8
27 stars 18 forks source link

Export timing out #26

Open OwainJ opened 2 years ago

OwainJ commented 2 years ago

Hello, we're experiencing an issue where the export (both excel and csv) is timing out after 3.8 minutes.

image The site has 18,949 members,

I'm getting this with Umbraco 8.18.3 and MemberListView 2.1.0 Unfortunately, there's no errors in the umbraco log.

Do you have any advice?

Thanks,

Owain

robertjf commented 2 years ago

Hi @OwainJ, that's a lot of members to export... are you able to apply a filter to reduce down the number of members you are trying to export perhaps?

I'm wondering whether we need to process the export as a background task and then send an email to download the resulting file to the user...

OwainJ commented 2 years ago

Hey @robertjf, unfortunately, no, they want to be able to export the entire members list.

I noticed that in MemberExtendedService.cs:L83-L97, it either gets the members for the export via Examine or via the Database if there are no filters applied. Forgive me if I'm being naive here, but what's the point of ever hitting the database? Wouldn't that always be slower than using examine, especially when making a full export with a large amount of members?

Although, I did try adding a filter (isApproved) to my export to force it to use the examine index, which also lowered the member number down to 18,067, but unfortunately it still timed out.

I'm wondering whether we need to process the export as a background task and then send an email to download the resulting file to the user...

That does sound like it would be the best solution to this issue, is that a feature that you would be looking to add to the package at some point in the future?

robertjf commented 2 years ago

hey @OwainJ the sql access is a fallback - in the past, the Examine index has been unreliable - if a member is updated, the update doesn't always get straight into the Examine index straight away (and some fields don't seem to make it in there at all).

processing the export as a background task is something I'd like to do, but I can't say when at the moment due to other commitments - if someone wanted to sponsor it I could put a resource on it but everyone's tied up at the moment.

I'll create a new task for that though.

robertjf commented 2 years ago

ok, task #27 has been created to track this feature enhancement

robertjf commented 1 year ago

The database query has been disabled to resolve #22 in the latest release for Umbraco 10+ - this isn't a complete fix, but may help with this scenario. As the list is compiled in memory, it could also be an issue of server resources.

ScottYourITTeam commented 9 months ago

Hi Owain,

I'll setup a testing environment with the 20 000 generated users and replicating this issue. While doing that I'll look at making it a background process. I'll keep you informed.