kniren
commented
8 years ago I ran some benchmarks using some of the base methods that can be used for operating on DataFrames. The dataset used was nycflights13 that consists on 336776 rows and 20 columns. The benchmark code was the following:
f, _ := os.Open("data/nycflights13.csv")
flights := dataframe.ReadCSV(f)
b.Run("Filter_1", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Filter(dataframe.F{"month", "==", 1})
}
})
b.Run("Filter_2", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Filter(dataframe.F{"month", "==", 1}).
Filter(dataframe.F{"day", "==", 1})
}
})
b.Run("Filter_3", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Filter(dataframe.F{"month", "==", 1}).
Filter(dataframe.F{"day", "==", 1}).
Filter(dataframe.F{"hour", "<=", 12})
}
})
b.Run("Arrange_1", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Arrange(
dataframe.Order{"arr_delay", true},
)
}
})
b.Run("Arrange_2", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Arrange(
dataframe.Order{"year", false},
dataframe.Order{"month", false},
)
}
})
b.Run("Arrange_3", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Arrange(
dataframe.Order{"year", false},
dataframe.Order{"month", false},
dataframe.Order{"day", false},
)
}
})
b.Run("Select_1", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Select([]string{"year"})
}
})
b.Run("Select_2", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Select([]string{"year", "month"})
}
})
b.Run("Select_3", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Select([]string{"year", "month", "day"})
}
})
var index10 []int
for i := 0; i < 10; i++ {
index10 = append(index10, i)
}
var index100 []int
for i := 0; i < 100; i++ {
index100 = append(index100, i)
}
var index1000 []int
for i := 0; i < 1000; i++ {
index1000 = append(index1000, i)
}
sub10 := flights.Subset(index10)
sub100 := flights.Subset(index100)
sub1000 := flights.Subset(index1000)
b.Run("Subset_10", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Subset(index10)
}
})
b.Run("Subset_100", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Subset(index100)
}
})
b.Run("Subset_1000", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Subset(index1000)
}
})
b.Run("Set_10", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Set(index10, sub10)
}
})
b.Run("Set_100", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Set(index100, sub100)
}
})
b.Run("Set_1000", func(b *testing.B) {
for i := 0; i < b.N; i++ {
flights.Set(index1000, sub1000)
}
})The two most obvious improvements that could be extracted from the cpu profile were that it seemed that a lot of small memory allocations were taking place and thus the GC was being impacted very heavily and also a lot of reflection was being made when new Series where copied or created. In order to address the first issue, slices pre-allocation has taken over growing slices, and for the second one, type assertion to check the given values is added prior to reflection and if the given types are not recognised then it defaults to use reflection.
With these two changes, already present on ffcfe13, we could observe a significant speedup with the same dataset:
benchmark old ns/op new ns/op delta
BenchmarkAll/Filter_1-4 278527790 119058640 -57.25%
BenchmarkAll/Filter_2-4 272421441 125048692 -54.10%
BenchmarkAll/Filter_3-4 280313297 126590535 -54.84%
BenchmarkAll/Arrange_1-4 3044940770 1342395569 -55.91%
BenchmarkAll/Arrange_2-4 4140007971 1527648847 -63.10%
BenchmarkAll/Arrange_3-4 6168188016 2275641266 -63.11%
BenchmarkAll/Select_1-4 48459199 14510337 -70.06%
BenchmarkAll/Select_2-4 92780819 28908778 -68.84%
BenchmarkAll/Select_3-4 132650059 44706709 -66.30%
BenchmarkAll/Subset_10-4 75320 30194 -59.91%
BenchmarkAll/Subset_100-4 419068 220374 -47.41%
BenchmarkAll/Subset_1000-4 3323734 2029517 -38.94%
BenchmarkAll/Set_10-4 1955485629 656503486 -66.43%
BenchmarkAll/Set_100-4 1889303947 638403492 -66.21%
BenchmarkAll/Set_1000-4 1979352270 678069517 -65.74%Further optimization is still on the cards by improving the algorithms, reducing memory allocations or using concurrency for some processes. For the latter, I've done some experiments on the dataframe.New function by using goroutines to perform the copy of the given Series. This results in a performance increase all over the board, but I haven't included it on the current dev branch just yet, since I wan't to make sure that using this kind of pattern won't affect users trying to made their code concurrent on their side.
So far there has not been an study on the performance of this library in terms of speed and memory consumption. I'm prioritising now those features that impact the users directly, since the API design is still on flux, but this should be addressed on the near future.