WebCuratorTool / webcurator

The root of the webcurator tool project, containing all modules needed to run a fully functional webcurator tool.
Apache License 2.0
1 stars 1 forks source link

3.1.5/bug fix vis #101

Closed leefrank9527 closed 4 months ago

leefrank9527 commented 4 months ago

Fixed the issues and improved the performance for visualization:

  1. Removed the max rows limitation from the grid view of visualization. Related to issue Remove the limitaion of the maximum number of rows from inspection view. #96

  2. Improved the performance of UI:

    • Did not valid each URLs when put them to the pruning/recrawling candidate page.
    • Fixed the issue of filter for tree views: for the lazy loading tree views, the nodes will be recursively loaded and expanded when filtering from the views. Disabled the auto expanded properties to figure out this issue.
    • Optimized the timer for progress updating: the users can open the page to inspect the progress of modification of havest results. The UI will polling the status of progress periodically. For huge modification, it will take a long time to get the progress data. Sometimes, the previous polling request wasn't finished then the next polling request was triggered. Optimized the process to start the next polling request only after the previous is finished.
    • Cleared the filter of the inspect table after reloadeding the data of the inspect table.
obrienben commented 4 months ago

@hannakoppelaar ready for you to review/merge

hannakoppelaar commented 4 months ago

@leefrank9527 It looks good! There's just a minor thing I noticed with respect to the removal of the max rows limitation. The method searchUrlDTOs in NetworkMapClientLocal now has a boolean parameter to disable this limitation. However, in other methods (searchUrl, queryChildrenRecursivelyCrawl and queryChildrenRecursivelyFolder) the code dealing with this limitation is commented out. I think it would make the code easier to understand if we simply got rid of the limitation completely: removing the parameter allowBigDatasets from searchUrlDTOs and deleting the commented out sections.

leefrank9527 commented 4 months ago

searchUrlDTOs

@hannakoppelaar Thanks for the feedback. Accepted the advices and cleared the comments and redundant argument. I've commited the changes to this PR.