Incorrect results intermittently when sorting

[naleeha]

Bug Report

Steps to Reproduce:

1. Run the code with the basket constituent sort fix https://github.com/finos/vuu/issues/1027
2. If you sort by description more than once - sometimes the results are out of order, sometimes they are correct
3. If you had sort on before loading the VP, it always creates VP with correct sort

When it correctly sorts When it sorts incorrectly

[naleeha]

• Confirmed that the UI sends CHANGE_VP correctly and the TABLE_ROW that is returned from server has incorrect data. Incorrect index for certain rows sometimes, sometimes multiple results for the same index
• Confirmed sort logic always returned the expected order
• Seems to have multiple thread publishing updates at the same time - which seems to be causing the confusion in results

Open questions

• How does the UI deals with it if duplicate record with same index in a table?

[naleeha]

Description of what is happening Can see nioEventLoopGroup is publishing updates for each rows with the old sorted values viewPortExecutorRunner does the sort viewPortExecutorRunner also publish updates for each rows with new sorted values TABLE_ROW update is generated as aggregates of these update row events but now has mix of two results

server logs.txt websocket messages.txt

Relevant Logs 17:21:21.563 [nioEventLoopGroup-3-2] INFO org.finos.vuu.core.sort.GenericSort2 - Sorting by List(description) and List(A) 17:21:21.563 [nioEventLoopGroup-3-2] INFO o.f.vuu.viewport.ViewPortContainer - [VP] default else condition in change() call 17:21:21.564 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - changeStructure(..) onlySortOrFilterChange=false 17:21:21.564 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - Sending updates on 283 inrangeKeys 17:21:21.564 [viewPortExecutorRunner[4]5] INFO o.f.vuu.viewport.ViewPortContainer - refreshing view port to apply sortGenericSort2(SortSpec(List(SortDef(description,A))),List(SimpleColumn(description,8,class java.lang.String))) requestId VP(user:nha1,table:basketConstituent,size: 782,id:nha1-VP-00000000) @SESS-5be9ea2c-c5ca-41be-981f-d851c4c5a87b 17:21:21.564 [viewPortExecutorRunner[4]5] INFO o.f.vuu.viewport.ViewPortContainer - refreshing view port to apply filter org.finos.vuu.core.filter.NoFilter$@ba3121c requestId VP(user:nha1,table:basketConstituent,size: 782,id:nha1-VP-00000000) @SESS-5be9ea2c-c5ca-41be-981f-d851c4c5a87b 17:21:21.564 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.core.sort.GenericSort2 - Starting map 17:21:21.565 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[0] = III.L..FTSE100 for request id 4 17:21:21.565 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[1] = MMM..SP500 for request id 4 17:21:21.565 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.core.sort.GenericSort2 - Starting sort ....

17:21:21.569 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.core.sort.GenericSort2 - [SORT]: Table Size: 782 DataToArray: 1ms, Sort: 2ms, ImmutArr: 2ms ...

17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[171] = 01209.HK..HSI for request id 4 17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[172] = 00836.HK..HSI for request id 4 17:21:21.570 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[0] = ZS..NASDAQ100 for request id 4 17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[173] = 01088.HK..HSI for request id 4 17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[174] = 00762.HK..HSI for request id 4 17:21:21.570 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[1] = ZM..NASDAQ100 for request id 4 17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[175] = 01378.HK..HSI for request id 4 17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[176] = CMG..SP500 for request id 4 17:21:21.570 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[2] = ZTS..SP500 for request id 4 17:21:21.571 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[177] = 01929.HK..HSI for request id 4 17:21:21.571 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[3] = ZION..SP500 for request id 4 17:21:21.571 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[178] = CB..SP500 for request id 4 17:21:21.571 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[4] = ZBH..SP500 for request id 4

[naleeha]

17:21:21.563 [nioEventLoopGroup-3-2] INFO org.finos.vuu.core.sort.GenericSort2 - Sorting by List(description) and List(A) 17:21:21.563 [nioEventLoopGroup-3-2] INFO o.f.vuu.viewport.ViewPortContainer - [VP] default else condition in change() call 17:21:21.564 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - changeStructure(..) onlySortOrFilterChange=false 17:21:21.564 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - Sending updates on 283 inrangeKeys

The onlySortOrFilterChange is expected to be true in this case and if so, it won't be publishing updates for the keys on nioEventLoopGroup

Currently onlySortOrFilterChange returns false even when it only has sort because the columns being compared here (ViewPortColumns) are not case class and the equality check returns false currently.

  private def onlyFilterOrSortChanged(newStructuralFields: ViewPortStructuralFields, current: ViewPortStructuralFields): Boolean = {
    newStructuralFields.table.asTable.name == current.table.asTable.name && newStructuralFields.groupBy == current.groupBy && newStructuralFields.columns == current.columns
  }