scala / scala-dev

Scala 2 team issues. Not for user-facing bugs or directly actionable user-facing improvements. For build/test/infra and for longer-term planning and idea tracking. Our bug tracker is at https://github.com/scala/bug/issues
Apache License 2.0
130 stars 15 forks source link

AllocationTest is flaky on JDK 19 #848

Open som-snytt opened 1 year ago

som-snytt commented 1 year ago

Reproduction steps

2.13.10

junit/testOnly scala.collection.immutable.ListAllocationTest

after moving the allocation tests out of ListTest.

Problem

Under JDK 19, when the test tries to verify an exact allocation for a list of N strings, the calculated cost is too low.

Some debug showing "cost per loop" while probing object allocation:

[info] Test run scala.collection.immutable.ListAllocationTest started
[info] Test scala.collection.immutable.ListAllocationTest.smallListAllocation started
TAG scala.collection.immutable.List
[][cost=0][warmup=10000][loops=10000][counts=Array(16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 13696, 16, *** , 16, 16, 0, 0, 0, 0, ***

The large number appears with regularity (an allocation after 64, perhaps, and 13696 is a multiple of 64). There are other smaller allocations, and finally the "cost" goes to zero. That is with 10000 loops of warmup and execution.

With current 1000, the behavior falls at random during one of the tests, and it fails:

*** 120, 104, *** // per loop alloc drops
[error] Test scala.collection.immutable.ListAllocationTest.smallListAllocation failed: java.lang.AssertionError: allocating min = 104 allowed = 120 -- list  size 5
[error]  result = List(0, 1, 2, 3, 4) (class scala.collection.immutable.$colon$colon)
[error]  allocation 104 (905 times)
[error]  allocation 104 (95 times)
[error] , took 0.113 sec

The docs say that the metrics are not intended to be precise; also tried putting generated references in an array to avoid collection.