Shopify / ghostferry

The swiss army knife of live data migrations
https://shopify.github.io/ghostferry
MIT License
693 stars 65 forks source link

Add platform to docker compose #324

Closed itsWill closed 2 years ago

itsWill commented 2 years ago

Specifiying the platform as x86_64 is needed for the m1 laptops, this allows the development environment to boot up with rosetta.

I was able to run the: make copydb && ghostferry-copydb -verbose examples/copydb/conf.json command with no issue.

I suspect this should still work on a regular intel mac but I have not tested it.

ghostferry git:(m1_compatibility) make copydb && ghostferry-copydb -verbose examples/copydb/conf.json

go build -ldflags "-X github.com/Shopify/ghostferry.VersionString=1.1.0+20211209173642+5f483a4" -o /Users/grpm/bin/ghostferry-copydb ./copydb/cmd
INFO[0000] hello world from 1.1.0+20211209173642+5f483a4  tag=ferry
INFO[0000] connecting to database                        dbname=source dsn="root:<masked>@tcp(127.0.0.1:29291)/?checkConnLiveness=false&collation=utf8mb4_unicode_ci&multiStatements=true&maxAllowedPacket=0&charset=utf8mb4&sql_mode=%27STRICT_ALL_TABLES%2CNO_BACKSLASH_ESCAPES%27&time_zone=%27%2B00%3A00%27" tag=ferry
INFO[0000] connected to source                           dbname=source hasSSL=false ssl_cipher= tag=ferry
INFO[0000] connecting to database                        dbname=target dsn="root:<masked>@tcp(127.0.0.1:29292)/?checkConnLiveness=false&collation=utf8mb4_unicode_ci&multiStatements=true&maxAllowedPacket=0&charset=utf8mb4&sql_mode=%27STRICT_ALL_TABLES%2CNO_BACKSLASH_ESCAPES%27&time_zone=%27%2B00%3A00%27" tag=ferry
INFO[0000] connected to target                           dbname=target hasSSL=false ssl_cipher= tag=ferry
INFO[0000] table schemas cached                          tables="[]" tag=table_schema_cache
INFO[0000] initializing                                  tag=control_server
INFO[0000] ferry initialized                             tag=ferry
INFO[0000] starting binlog streaming                     file=mysql-bin.000004 host=127.0.0.1 port=29291 position=194 tag=source_binlog_streamer
INFO[0000] creating databases and tables on target
INFO[0000] starting ferry run                            tag=ferry
INFO[0000] starting data iterator run                    tablesCount=0 tag=data_iterator
INFO[0000] running on 0.0.0.0:8000                       tag=control_server
INFO[0000] starting binlog streamer                      tag=source_binlog_streamer
INFO[0000] starting periodic reverifier                  tag=inline-verifier
DEBU[0000] reached position                              file=mysql-bin.000004 lastStreamedBinlogPosition="(mysql-bin.000004, 194)" position=0 tag=source_binlog_streamer type="*replication.RotateEvent"
INFO[0000] binlog file rotated                           last_filename=mysql-bin.000004 last_position=194 new_filename=mysql-bin.000004 new_position=194 tag=source_binlog_streamer
DEBU[0000] reached position                              file=mysql-bin.000004 lastStreamedBinlogPosition="(mysql-bin.000004, 194)" position=0 tag=source_binlog_streamer type="*replication.FormatDescriptionEvent"
INFO[0000] done queueing tables to be iterated, closing table channel  tag=data_iterator
INFO[0000] data copy is complete, waiting for cutover    tag=ferry
DEBU[0001] reverified                                    remainingRowCount=0 tag=inline-verifier
DEBU[0001] waiting for AutomaticCutover to become true before signaling for row copy complete  tag=ferry
DEBU[0002] reverified                                    remainingRowCount=0 tag=inline-verifier
# ..... Truncated DEBU lines
DEBU[0132] waiting for AutomaticCutover to become true before signaling for row copy complete  tag=ferry
INFO[0132] served http request                           fields.time=1.556459ms method=GET path=/ tag=control_server
INFO[0132] served http request                           fields.time=5.412958ms method=GET path=/static/css/app.css tag=control_server
INFO[0132] served http request                           fields.time=6.887417ms method=GET path=/static/css/normalize.css tag=control_server
INFO[0132] served http request                           fields.time=5.977792ms method=GET path=/static/css/skeleton.css tag=control_server
INFO[0132] served http request                           fields.time="59.458µs" method=GET path=/favicon.ico tag=control_server
DEBU[0133] reverified                                    remainingRowCount=0 tag=inline-verifier
DEBU[0133] waiting for AutomaticCutover to become true before signaling for row copy complete  tag=ferry
DEBU[0134] reverified                                    remainingRowCount=0 tag=inline-verifier
DEBU[0134] waiting for AutomaticCutover to become true before signaling for row copy complete  tag=ferry
DEBU[0135] reverified                                    remainingRowCount=0 tag=inline-verifier
DEBU[0135] waiting for AutomaticCutover to become true before signaling for row copy complete  tag=ferry
DEBU[0136] reverified                                    remainingRowCount=0 tag=inline-verifier
DEBU[0136] waiting for AutomaticCutover to become true before signaling for row copy complete  tag=ferry
INFO[0136] served http request                           fields.time="130.875µs" method=POST path=/api/cutover tag=control_server
INFO[0136] served http request                           fields.time="819.875µs" method=GET path=/ tag=control_server
DEBU[0137] reverified                                    remainingRowCount=0 tag=inline-verifier
DEBU[0137] waiting for AutomaticCutover to become true before signaling for row copy complete  tag=ferry
INFO[0137] shutdown periodic reverifier                  tag=inline-verifier
INFO[0137] calling VerifyBeforeCutover                   tag=ferry
INFO[0137] completed re-verification iteration 0         iteration=0 store_size_after=0 store_size_before=0 tag=inline-verifier
INFO[0137] entering cutover phase, notifying caller that row copy is complete  tag=ferry
INFO[0137] requesting binlog streamer to stop            tag=source_binlog_streamer
INFO[0137] current stop binlog position was recorded     stop_at_position="(mysql-bin.000004, 194)" tag=source_binlog_streamer
INFO[0137] exiting binlog streamer                       lastStreamedBinlogPosition="(mysql-bin.000004, 194)" stopAtBinlogPosition="(mysql-bin.000004, 194)" tag=source_binlog_streamer
INFO[0137] ghostferry run is complete, shutting down auxiliary services  tag=ferry
INFO[0137] ghostferry main operations has terminated but the control server remains online
INFO[0137] press CTRL+C or send an interrupt to stop the control server and end this process
floriecai commented 2 years ago

I think we need to change the docker-compose version here. To something liek 1.27.0 (but it's in pre-release)