search

Informatievlaanderen / VSDS-LDESServer4J

Linked Data Event Streams Server for Java
https://informatievlaanderen.github.io/VSDS-LDESServer4J
European Union Public License 1.2
9 stars 8 forks source link

Performance issues with deleting views #721

Closed Tomvbe closed 4 months ago

Tomvbe commented 1 year ago

Describe the bug Deletion of a rather small view takes more than 10 min. Needs a deep dive into performance.

To Reproduce Steps to reproduce the behavior:

Have a view with around 13k members
Performadmin/api/v1/eventstreams/{collection}/views/{view}
See how deletion takes more than 10 min
Tomvbe commented 1 year ago

Fixed in https://github.com/Informatievlaanderen/VSDS-LDESServer4J/pull/731

Tomvbe commented 1 year ago

Reopened this issue as the deletion of views is slow again after splitting the retention logic to a separate module. In DeleteEventHandler we have the following listener: 

@EventListener
    public void handleViewDeletedEvent(ViewDeletedEvent event) {
        String viewName = event.getViewName().asString();
        memberPropertiesRepository.getMemberPropertiesWithViewReference(viewName)
                .forEach(memberProperties -> memberRemover.removeMemberFromView(memberProperties, viewName));
    }

This iterates all members of a view one by one when a view is deleted. When a view has 1 mio members, this takes ages..

jobulcke commented 4 months ago

Fixed in https://github.com/Informatievlaanderen/VSDS-LDESServer4J/pull/1232