search

shssoichiro / sqlformat-rs

SQL Formatter written in Rust
Apache License 2.0
89 stars 26 forks source link

Tokenizer O(n²) complecity #23

Closed darksidecat closed 1 year ago

darksidecat commented 1 year ago

With large queryes, for example batch insert of 2000 rows 6 colums, there is significant performance problems, formatting such query takes 13 seconds on my machine

O(n²) complecity: https://github.com/shssoichiro/sqlformat-rs/blob/66bdb0fadf9e750600df0c778cff88e45277d7c1/src/tokenizer.rs#L17-L24

Related comment https://github.com/launchbadge/sqlx/issues/633#issuecomment-1687904140

darksidecat commented 1 year ago

I think that we can just save last TokenKind::Reserved position instead searching for it. But it is necessary to investigate and test

https://github.com/shssoichiro/sqlformat-rs/blob/66bdb0fadf9e750600df0c778cff88e45277d7c1/src/tokenizer.rs#L22