llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28k stars 11.56k forks source link

[libc++] LLVM libc++'s fstream is very slow compared to GNU libstdc++ #102768

Open trcrsired opened 1 month ago

trcrsired commented 1 month ago
D:\msys64\home\unlvs\projects\fast_io\benchmark\0002.file_io_tsc\time>clang++ -o fstream fstream.cc -O3 -std=c++26 -I../../include -lntdll -s -flto=thin -fuse-ld=lld --sysroot=D:\toolchains\llvm\x86_64-windows-gnu\x86_64-windows-gnu -rtlib=compiler-rt --unwindlib=libunwind --stdlib=libc++ -lntdll -lc++abi -I../../../include

D:\msys64\home\unlvs\projects\fast_io\benchmark\0002.file_io_tsc\time>fstream
output:9.5154552s
input:2.6723663s

D:\msys64\home\unlvs\projects\fast_io\benchmark\0002.file_io_tsc\time>clang++ -o fstream fstream.cc -O3 -std=c++26 -I../../include -lntdll -s -flto=thin -fuse-ld=lld -lntdll -I../../../include

D:\msys64\home\unlvs\projects\fast_io\benchmark\0002.file_io_tsc\time>fstream
output:0.3279629s
input:0.4409194s

D:\msys64\home\unlvs\projects\fast_io\benchmark\0002.file_io_tsc\time>clang++ -o iobuf_file iobuf_file.cc -O3 -std=c++26 -I../../include -lntdll -s -flto=thin -fuse-ld=lld --sysroot=D:\toolchains\llvm\x86_64-windows-gnu\x86_64-windows-gnu -rtlib=compiler-rt --unwindlib=libunwind --stdlib=libc++ -lntdll -lc++abi -I../../../include

D:\msys64\home\unlvs\projects\fast_io\benchmark\0002.file_io_tsc\time>iobuf_file.exe
output:0.0587952s
input:0.1234146s

D:\msys64\home\unlvs\projects\fast_io\benchmark\0002.file_io_tsc\time>clang++ -o filebuf_file filebuf_file.cc -O3 -std=c++26 -I../../include -lntdll -s -flto=thin -fuse-ld=lld --sysroot=D:\toolchains\llvm\x86_64-windows-gnu\x86_64-windows-gnu -rtlib=compiler-rt --unwindlib=libunwind --stdlib=libc++ -lntdll -lc++abi -I../../../include

D:\msys64\home\unlvs\projects\fast_io\benchmark\0002.file_io_tsc\time>filebuf_file
output:0.3089046s
input:0.1379314s

image

This is at least 20x slower than libstdc++, and I am sure it is not the buffering part but the formatting part that is slow since the fast_io::filebuf_file isn't slow.

https://github.com/cppfastio/fast_io/blob/next/benchmark/0002.file_io_tsc/time/fstream.cc

Benchmark is here.

filebuf_file implementation (wrapper around std::basic_filebuf): https://github.com/cppfastio/fast_io/blob/next/include/fast_io_legacy_impl/filebuf/filebuf_file.h https://github.com/cppfastio/fast_io/blob/next/include/fast_io_legacy_impl/filebuf/fp_hack/libc%2B%2B.h https://github.com/cppfastio/fast_io/blob/next/include/fast_io_legacy_impl/filebuf/fp_hack/libstdc%2B%2B.h https://github.com/cppfastio/fast_io/blob/next/include/fast_io_legacy_impl/filebuf/fp_hack/msvc_stl.h

trcrsired commented 1 month ago

LLVM and GCC are here: https://github.com/trcrsired/gcc-releases/releases https://github.com/trcrsired/llvm-releases/releases

GCC and LLVM build scripts: https://github.com/trcrsired/toolchainbuildscripts/blob/main/llvm/x86_64-windows-gnu.sh

harshn05 commented 3 weeks ago

I think its a known issue for many years.

Any solution or workaround?

trcrsired commented 3 weeks ago

I think its a known issue for many years.

Any solution or workaround?

i guess the only solution is to use my fast_io instead? i guess no one wants to get into trouble with iostream

Does not seem to be fixable based on what jwakely said.