Closed drakkan closed 2 years ago
jackc
commented
4 years ago The imports you mention are dependencies of the pgx module as a whole, but not of the core pgx package. i.e. They will not be included in your binary unless necessary. e.g. pgx includes adapters for several popular logging packages in the log directory. But if you don't include those packages they will not be included.
You can use go tool nm PROGRAM to confirm what is or is not included in the final binary.
drakkan
commented
4 years ago Hi,
you are right, all package imported inside subdirs are not included inside the final binary, however please try to build this sample program:
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/lib/pq"
)
func main() {
connStr := "user=postgres password=postgres dbname=postgres host=127.0.0.1 sslmode=disable"
driver := "postgres"
db, err := sql.Open(driver, connStr)
if err != nil {
log.Fatal(err)
}
defer db.Close()
err = db.Ping()
fmt.Printf("ping err %v\n", err)
}its size is 6018288 bytes, if you replace lib/pq with pgx this way:
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/jackc/pgx/v4/stdlib"
)
func main() {
connStr := "user=postgres password=postgres dbname=postgres host=127.0.0.1 sslmode=disable"
driver := "pgx"
db, err := sql.Open(driver, connStr)
if err != nil {
log.Fatal(err)
}
defer db.Close()
err = db.Ping()
fmt.Printf("ping err %v\n", err)
}the size is now 8809625 bytes.
Do you think the size could be reduced somehow? Thank you
jackc
commented
4 years ago I don't know of any ways to trim the core library down. I suspect it is mostly the extensive type support. Just as an experiment I removed the hstore type and array types from pgtype. The size of the final binary decreased by 700k. However, array support could not be separated into another package without breaking backwards compatibility.
drakkan
commented
4 years ago Thank you for your test.
Just to give you an idea I report here the size cost for other SQL driver.
In SFTPGo I can disable SQL drivers using build tags
If I compile all the features using Go 1.15.1 on Linux with the following command:
go build -ldflags "-s -w -X github.com/drakkan/sftpgo/version.commit=`git describe --always --dirty` -X github.com/drakkan/sftpgo/version.date=`date -u +%FT%TZ`" -o sftpgothe binary size is 26632144 bytes
lib/pq) support (-tags nopgsql) the size is 26361296 bytes so lib/pq adds 264 KiB-tags nomysql) the size is 26398320 bytes so go-sql-driver/mysql adds 228 KiB-tags nosqlite) the size is 25346048 bytes so the CGO package go-sqlite3 adds 1.2MBpgx adds 2.2 MB, so it seems the most feature rich driver :)
drakkan
commented
3 years ago Hi,
what about adding a build tag? If someone wants to use pgx only as a database/sql driver, they could enable a build tag which excludes all additional features.
This would be backward compatibile too, thank you
jackc
commented
3 years ago what about adding a build tag? If someone wants to use pgx only as a database/sql driver, they could enable a build tag which excludes all additional features.
It's might be possible, but I think the maintenance burden of wiring and testing this among the multiple packages that would need to be changed would be too high.
drakkan
commented
2 years ago Hi,
I just tested pgx/v5, the binary size increase is now about 600k compared to lib/pq, much better!
Are you planning other improvements to reduce imports when the library is used as a database/sql driver or is this the expected end result? Thanks
jackc
commented
2 years ago This is probably as much size reduction as we will get. I don't know of anything else that would be practical to trim off.
bfontaine
commented
2 years ago I believe we can close this issue now. Feel free to reopen it if you think it’s a mistake.
drakkan
commented
2 years ago This is not really fixed, pgx v5 increases the binary size in the same way as v4 if we add the nopgxregisterdefaulttypes flag and about 2 MB more than v4 if we omit that flag.
I switched to pgx anyway because it is much better maintained and it has more features compared to lib/pq though I only use it as database/sql compatible driver. SFTPGo uses pgx from v2.4.0 released a few days ago, let's see if I get any complaints other than the increased binary size. @jackc thanks for your work
Hi,
I'm evaluating to replace
lib/pqwithpgxin SFTPGo usingpgxas database/sql compatible driver. The conversion is quite simple I just need to replace the import and then usesql.Open("pgx",....instead ofsql.Open("postgres",...All my test cases pass, however my binary size increased of about 2MB. This is not a big deal however I'm wondering if it is possible to avoid uneeded imports and so reduce the binary size, for example in my
go.modI have now logrus, zap etc.. are all these deps required? If they are not required and you avoid to import them ingithub.com/jackc/pgx/v4/stdlibthey will not end up inside the final binary.To give you an idea:
lib/pg, sftpgo size is 26632144 bytespgx, sftpgo size is 28641456 bytesand here are the diffs for
go.modandgo.sum:thank you