The JNA implementation seems unbarably slow on Windows (7)

spangaer commented 6 years ago

When running things like sbt eclipse SBT munches through the local Ivy repository very aggressively. On Windows it used the JNA code for this

at sbt.internal.io.WinMilli$.getModifiedTimeNative(Milli.scala:271)

and it seems to be unbearably slow. Many of the file accesses take multiple seconds twice, once to figure out a symlink that needs to be resolved (repo is relocated via symlink) and once to access the actual file. (observed via procmon) Because these slow cases run in the hundreds (for certain projects; iterates multiple times too), it can take up to 15 minutes to munch through.

When enabling -Dsbt.io.jdktimestamps=true the same file access becomes sub millisecond. (I will re-iterate on Monday with a cold(er) system to verify).

This is a:

SBT 1.1.4
Intel i7 4th gen
Win 7
Spectre/Meltdown fixes enabled (given the milisecond story that might be of relevance)
Trend Micro AV and more anti-nasty protection

eed3si9n commented 6 years ago

Thanks for the report @spangaer

spangaer commented 6 years ago

Confirming improvement by system property flag.

post weekend run
post reboot run (force cold FS cache)
post eclipse start run

All have consistent improvement. It's not blazingly fast, because it generates 300k file related procmon entries for a single run. But only between 0 and 5 (across runs) are over 1 second instead of hundreds, the remainder is in the millisecond or sub-millisecond range and at least the CPU is doing something else then waiting for timestamp info.

sbt / io

The JNA implementation seems unbarably slow on Windows (7) #143