planetscale / cli

The CLI for PlanetScale Database
https://planetscale.com/cli
Apache License 2.0
603 stars 51 forks source link

[Bug] pscale dump command with --replica creates filenames with @replica #863

Closed FancyFane closed 2 months ago

FancyFane commented 5 months ago

Problem Description

When larger tables are dumped using the pscale dump command files are sometimes produced with an @replica designation. This causes issues when trying to import data, as this filename will target replicas for the import, producing an error:

[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables
Starting to restore database fane_test from folder ./
⠋ Restoring database ...2024-05-06T17:05:42-05:00   error   error restoring {"error": "supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)"}
Error: failed to restore database: supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)

Reproduction Steps

  1. Create a database with some reasonable amount of data in it
  2. Run a pscale dump command to export the data including the --replica flag.
    pscale database dump --keyspace fane_test_sharded fane_test main --replica
    Starting to dump all tables from database fane_test to folder /home/jason/pscale_dump_fane_test_main_fane_test_sharded_20240506_170354
    Dumping is finished! (elapsed time: 6.636205788s)
  3. Change directory into the dump location and do a listing note the @replica within the filenames of some files.
    [jason@blackbox ~]$ cd pscale_dump_fane_test_main_fane_test_sharded_20240506_170354/
    [jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ ls -lah
    total 266M
    drwxr-xr-x  2 jason jason 4.0K May  6 17:04 .
    drwxr-xr-x 45 jason jason  12K May  6 17:03 ..
    -rw-r--r--  1 jason jason 130M May  6 17:03 fane_test_sharded@replica.sbtest1.00001.sql
    -rw-r--r--  1 jason jason 130M May  6 17:04 fane_test_sharded@replica.sbtest1.00002.sql
    -rw-r--r--  1 jason jason 7.2M May  6 17:04 fane_test_sharded.sbtest1.00003.sql
    -rw-r--r--  1 jason jason  234 May  6 17:03 fane_test_sharded.sbtest1-schema.sql
    -rw-r--r--  1 jason jason  207 May  6 17:03 fane_test_sharded.test-schema.sql
    -rw-r--r--  1 jason jason    0 May  6 17:03 metadata
  4. Attempt to import into a test branch:
    
    [jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables
    Starting to restore database fane_test from folder ./
    ⠋ Restoring database ...2024-05-06T17:05:42-05:00   error   error restoring {"error": "supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)"}
    Error: failed to restore database: supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)

### Work Around
There is a work-a-round if this occurs, you can simply rename the files removing the `@replica` from the filename and the import runs without issues. 

[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ mv fane_test_sharded@replica.sbtest1.00001.sql fane_test_sharded.sbtest1.00001.sql [jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ mv fane_test_sharded@replica.sbtest1.00002.sql fane_test_sharded.sbtest1.00002.sql [jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables Starting to restore database fane_test from folder ./ Restore is finished! (elapsed time: 1m17.473330261s)


## Version

[jason@blackbox ~]$ pscale version pscale version 0.197.0 (build date: 2024-04-23T15:25:04Z commit: d8b065d)