search

go-jet / jet

Type safe SQL builder with code generation and automatic query result data mapping
Apache License 2.0
2.23k stars 110 forks source link

10x mysql db info querying speedup #274

Closed josephbuchma closed 9 months ago

josephbuchma commented 9 months ago

Optimisation consists of two parts:

Sample run:

$ jet -source=mysql -dsn="$MYSQL_USER:$MYSQL_PASS@tcp(localhost:$MYSQL_PORT)/facility" -path=./gen
Connecting to MySQL database...
Retrieving database information...
        FOUND 148 table(s), 162 view(s), 13 enum(s)
Destination directory: gen/facility
Cleaning up destination directory...
Generating table model files...
Generating view model files...
Generating enum model files...
Generating table sql builder files
Generating view sql builder files
Generating enum sql builder files

Results:

Tested with MySQL 5.7 running locally in Docker.

josephbuchma commented 9 months ago

I'm wondering why am I the first to be annoyed by slow codegen? Could not find any related issues or PRs.

houten11 commented 9 months ago

Hmm, 4m9s that's weird. I routinely run the generator with around 50 entities(tables and views) and it usually doesn't take more than a second.
How many columns on average your tables/views have?

josephbuchma commented 9 months ago

12 columns on average. Another factor that makes it slow is that it's docker and I'm on mac with m1 chip (although I'm using mysql 5.7 image build for arm)

josephbuchma commented 9 months ago

LOL, just tried it on mysql/mysql-server:latest and it took 1.5s on master and 550ms with this changes. Maybe this is not a problem after all :D

josephbuchma commented 9 months ago

Tried to run official mysql 5.7 image with x86 emulation in docker, and it's very slow. We're planning to upgrade mysql, but right now I need to be able to work with 5.7 locally, and I prefer running it in docker, so would appreciate getting this PR merged.

go-jet commented 9 months ago

A bit more speed is always welcomed. :+1:

codecov-commenter commented 9 months ago

Codecov Report

Merging #274 (f472bec) into master (e2fe799) will increase coverage by 0.06%. The diff coverage is 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##           master     #274      +/-   ##
==========================================
+ Coverage   90.55%   90.62%   +0.06%     
==========================================
  Files         124      124              
  Lines        7064     7080      +16     
==========================================
+ Hits         6397     6416      +19     
+ Misses        515      513       -2     
+ Partials      152      151       -1
Files Changed Coverage Δ
generator/mysql/query_set.go 90.21% <100.00%> (+6.00%) :arrow_up:
josephbuchma commented 9 months ago

@go-jet I addressed all your comments, thanks for quick feedback!

go-jet commented 9 months ago

LGTM. :+1: