OSX performance

[stuta]

OSX, latest Luajit 2.1.

Totals: { cjson = 0, dkjson = 0, rapidjson = 0.07904502, simdjson = 0.57141524 }

[FourierTransformer]

Thanks for trying it out! Huh, those are pretty interesting outcomes. What CPU do you have (including generation) and what version of OSX?

[stuta]

OSX Mojave 10.14.6 (18G4032)

CPU Intel(R) Core(TM) i5-3570K CPU @ 3.40GHz, Ivy Bridge (Core i5)

• 4 cores and 4 logical processors, 32K L1 cache and 256K L2 cache
• 3403 MHz (according to your OS) CPU features: vender:GenuineIntel idmax:13 step:9 model:58 family:6 type:0 fpu:1 mmx:1 sse:1 sse2:1 htt:1 ia64:0 sse3:1 vmx:1 ssse3:1 fma:0 sse4.1:1 sse4.2:1 popcnt:1 aes:1 avx:1 hypervisor:0 bmi1:0 avx2:0 bmi2:0 avx512f:0 avx512dq:0 avx512ifma:0 avx512pf:0 avx512er:0 avx512cd:0 sha:0 avx512bw:0 avx512vl:0 avx512vbmi:0

I have sse4.2 enabled version of rapidjson.

CMakeLists.txt:

add_definitions(-DRAPIDJSON_SSE2=1 -DRAPIDJSON_SSE42=1 -DRAPIDJSON_SIMD=1) set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -msse2 -msse4.2") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -msse2 -msse4.2")

[FourierTransformer]

I'll do some testing on my end to see if OSX builds are causing any weirdness...

According to the CPU arch implementations, it looks like only the SSE 4.2 instructions should be used. I wonder if they aren't being enabled properly on OSX or if without AVX2 it actually ends up being a lot slower...

It looks like the active implementation can be queried, I'll try to get that added so we can inspect this a little better.

[FourierTransformer]

I just added a simdjson.activeImplementation(). It should be in master/luarocks dev install now. If you want to let me know what that returns, that would be great!

[stuta]

I had to clean makefile to make it compile:

LUA_INCDIR = /usr/local/include LIBFLAG = -undefined dynamic_lookup SRC = src/luasimdjson.cpp src/simdjson.cpp INCLUDE = -I$(LUA_INCDIR) LIBS_PATH = # -L$(LUA_LIBDIR) LIBS = -lpthread FLAGS = -std=c++11 -Wall $(LIBFLAG) $(CFLAGS)

all: simdjson.so

simdjson.so: $(CXX) $(SRC) $(FLAGS) $(INCLUDE) $(LIBS_PATH) $(LIBS) -o $@

clean: rm simdjson.so

install: simdjson.so cp simdjson.so $(INST_LIBDIR)

---

jsonexamples/apache_builds.json Bytes: 127275 simd 0.00453523 rapidjson 0.00060687

jsonexamples/canada.json Bytes: 2251051 simd 0.0888311 rapidjson 0.01558697

jsonexamples/citm_catalog.json Bytes: 1727204 simd 0.04902592 rapidjson 0.0067769999999998

jsonexamples/github_events.json Bytes: 65132 simd 0.0021981700000001 rapidjson 0.00027338000000007

jsonexamples/google_maps_api_compact_response.json Bytes: 11812 simd 0.00072353999999994 rapidjson 0.00012911000000003

jsonexamples/google_maps_api_response.json Bytes: 26102 simd 0.00097348999999983 rapidjson 0.00013881000000005

jsonexamples/gsoc-2018.json Bytes: 3327831 simd 0.08846701 rapidjson 0.00557199

jsonexamples/instruments.json Bytes: 220346 simd 0.0073841899999999 rapidjson 0.0011489799999999

jsonexamples/marine_ik.json Bytes: 2983466 simd 0.12173004 rapidjson 0.02108867

jsonexamples/mesh.json Bytes: 723597 simd 0.02761947 rapidjson 0.0030319900000002

jsonexamples/mesh.pretty.json Bytes: 1577353 simd 0.04321352 rapidjson 0.0037987600000002

jsonexamples/numbers.json Bytes: 150124 simd 0.0050755499999997 rapidjson 0.00044210999999947

jsonexamples/random.json Bytes: 510476 simd 0.04103568 rapidjson 0.0074857599999999

jsonexamples/repeat.json Bytes: 11356 simd 0.0007519300000002 rapidjson 4.101000000027e-05

jsonexamples/twitter_api_compact_response.json Bytes: 10075 simd 0.00042121000000016 rapidjson 5.7139999999904e-05

jsonexamples/twitter_api_response.json Bytes: 15253 simd 0.00053880999999969 rapidjson 7.6809999999696e-05

jsonexamples/twitterescaped.json Bytes: 562408 simd 0.02475531 rapidjson 0.00331927

jsonexamples/twitter.json Bytes: 631515 simd 0.026094730000001 rapidjson 0.0030136799999995

jsonexamples/twitter_timeline.json Bytes: 42233 simd 0.0017324000000001 rapidjson 0.00028170999999993

jsonexamples/update-center.json Bytes: 533178 simd 0.02229718 rapidjson 0.0040785100000002

jsonexamples/small/adversarial.json Bytes: 80 simd 8.4799999999774e-06 rapidjson 1.7300000003218e-06

jsonexamples/small/demo.json Bytes: 387 simd 1.8069999999852e-05 rapidjson 3.0100000002165e-06

jsonexamples/small/flatadversarial.json Bytes: 64 simd 7.5099999997263e-06 rapidjson 1.9900000000916e-06

jsonexamples/small/smalldemo.json Bytes: 204 simd 1.4739999999946e-05 rapidjson 3.1500000000051e-06

jsonexamples/small/truenull.json Bytes: 12000 simd 0.00044737999999988 rapidjson 7.1430000000205e-05

Totals: { cjson = 0, dkjson = 0, rapidjson = 0.07702984, simdjson = 0.55790066 }

[FourierTransformer]

Ooh, i'll try to update the rockspec so it cleans and builds. You should now have the new method, what's the result of running print(simdjson.activeImplementation()) ?

[stuta]

I prefer plain makefiles or cmake, I don't use luarocks.

print(simdjson.activeImplementation()): westmere (Intel/AMD SSE4.2)

[FourierTransformer]

I upgraded to the latest simdjson version (0.5.0) and did some testing on various platforms. And on some, rapidjson still beat out simdjson. I think it ends up being architecture specific. I'll put in a note in the benchmark section.

Thanks!