leventov / Koloboke

Java Collections till the last breadcrumb of memory and performance
https://koloboke.com/
1.01k stars 139 forks source link

Cannot build with Java 1.8.0_11 #24

Closed llogiq closed 10 years ago

llogiq commented 10 years ago

Hi there,

I'm currently using trove for a few projects, so naturally I wanted to benchmark hftc in comparison. However, the build buildMain always stops at > Building 8% > :lib:impl:compileJava.

My setup is Java 1.8.0_11 (oracle) on LUbuntu 13.10 (64bit), gradle 2.0, groovy 2.3.3, ant 1.9.3.

PS. It would be great if you could release binaries.

leventov commented 10 years ago

HFTC build is very time & memory consuming. On my not so week dev machine (8 GB RAM, SSD) the full build takes more than half an hour. I have never tried to run it on HDD. :lib:impl:compileJava takes the most memory among all tasks, more than 3 GB.

If you build yourself, it seems practical to generate less specializations, excluding which you don't need. for example, add these lines after https://github.com/OpenHFT/hftc/blob/e64f5e34bc7d02738cff143dcab35cca3e759033/lib/build.gradle#L31:

    never 'byte|char|short'
    exclude 'key=float|double,value=float|double|int|long|obj'
    exclude 'key=int,value=obj|float'

Also, why you can't get these binaries: https://oss.sonatype.org/content/groups/public/net/openhft/hftc-api-jdk8/0.5/hftc-api-jdk8-0.5.jar and https://oss.sonatype.org/content/groups/public/net/openhft/hftc-impl-jdk8/0.5/hftc-impl-jdk8-0.5.jar?

llogiq commented 10 years ago

Thanks - I wouldn't have found those binaries without your link; you may consider linking them from README.md for us Sonatype-impaired folks?

I took the binaries and ran a small test for IntHashSet (writing 50k random numbers, checking 100k random numbers, some of which are probably not within the set).

I may have too puny a machine (only 4GB of ram, sluggish HDD), but hftc ran roughly 20-40× slower than a comparable trove TIntHashSet. I figure I must be doing something wrong, though.

leventov commented 10 years ago

Could you show benchmarking code, please?

llogiq commented 10 years ago

Here you are.

Yes, I know I should have used JMH instead of caliper, but somehow my mvn got borked, thus JMH doesn't run ATM. Still, the results are so far off that I think there must be something amiss & it should be easy enough to reproduce.

Regards, Andre

2014-07-28 13:57 GMT+02:00 Roman Leventov notifications@github.com:

Could you show benchmarking code, please?

— Reply to this email directly or view it on GitHub https://github.com/OpenHFT/hftc/issues/24#issuecomment-50329307.

leventov commented 10 years ago

Sorry, I don't see the code or the link to.

I'm sure Caliper is good enough for this type of benchmark. But it is anyway interesting to explore the case.

llogiq commented 10 years ago

Ok, I've put the code into a public gist

leventov commented 10 years ago

@llogiq, that isn't a problem of your bench, that was an awful bug in HFTC implementation :( That's surprising how could it live in the repo for so long time.

Now results are (I rewritten in JMH without changes, because there are problems with running Caliper on Windows):

Benchmark                             Mode   Samples         Mean   Mean error    Units
t.HftcVsTroveIntSet.timeHftcHash      avgt        10       29,173        0,632    ns/op
t.HftcVsTroveIntSet.timeTroveHash     avgt        10       56,952        0,121    ns/op

I have released new binaries with a fix: https://oss.sonatype.org/content/groups/public/net/openhft/hftc-api-jdk8/0.5.1/hftc-api-jdk8-0.5.1.jar and https://oss.sonatype.org/content/groups/public/net/openhft/hftc-impl-jdk8/0.5.1/hftc-impl-jdk8-0.5.1.jar

leventov commented 10 years ago

@llogiq, thank you very much.

llogiq commented 10 years ago

You're welcome. Thanks for the timely fix and keep up the good work!