When larger tables are dumped using the pscale dump command files are sometimes produced with an @replica designation. This causes issues when trying to import data, as this filename will target replicas for the import, producing an error:
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables
Starting to restore database fane_test from folder ./
⠋ Restoring database ...2024-05-06T17:05:42-05:00 error error restoring {"error": "supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)"}
Error: failed to restore database: supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)
Reproduction Steps
Create a database with some reasonable amount of data in it
Run a pscale dump command to export the data including the --replica flag.
pscale database dump --keyspace fane_test_sharded fane_test main --replica
Starting to dump all tables from database fane_test to folder /home/jason/pscale_dump_fane_test_main_fane_test_sharded_20240506_170354
Dumping is finished! (elapsed time: 6.636205788s)
Change directory into the dump location and do a listing note the @replica within the filenames of some files.
[jason@blackbox ~]$ cd pscale_dump_fane_test_main_fane_test_sharded_20240506_170354/
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ ls -lah
total 266M
drwxr-xr-x 2 jason jason 4.0K May 6 17:04 .
drwxr-xr-x 45 jason jason 12K May 6 17:03 ..
-rw-r--r-- 1 jason jason 130M May 6 17:03 fane_test_sharded@replica.sbtest1.00001.sql
-rw-r--r-- 1 jason jason 130M May 6 17:04 fane_test_sharded@replica.sbtest1.00002.sql
-rw-r--r-- 1 jason jason 7.2M May 6 17:04 fane_test_sharded.sbtest1.00003.sql
-rw-r--r-- 1 jason jason 234 May 6 17:03 fane_test_sharded.sbtest1-schema.sql
-rw-r--r-- 1 jason jason 207 May 6 17:03 fane_test_sharded.test-schema.sql
-rw-r--r-- 1 jason jason 0 May 6 17:03 metadata
Attempt to import into a test branch:
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables
Starting to restore database fane_test from folder ./
⠋ Restoring database ...2024-05-06T17:05:42-05:00 error error restoring {"error": "supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)"}
Error: failed to restore database: supported only for primary tablet type, current type: replica (errno 1105) (sqlstate HY000)
### Work Around
There is a work-a-round if this occurs, you can simply rename the files removing the `@replica` from the filename and the import runs without issues.
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ mv fane_test_sharded@replica.sbtest1.00001.sql fane_test_sharded.sbtest1.00001.sql
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ mv fane_test_sharded@replica.sbtest1.00002.sql fane_test_sharded.sbtest1.00002.sql
[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables
Starting to restore database fane_test from folder ./
Restore is finished! (elapsed time: 1m17.473330261s)
## Version
[jason@blackbox ~]$ pscale version
pscale version 0.197.0 (build date: 2024-04-23T15:25:04Z commit: d8b065d)
Problem Description
When larger tables are dumped using the
pscale dump
command files are sometimes produced with an@replica
designation. This causes issues when trying to import data, as this filename will target replicas for the import, producing an error:Reproduction Steps
pscale dump
command to export the data including the--replica
flag.@replica
within the filenames of some files.[jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ mv fane_test_sharded@replica.sbtest1.00001.sql fane_test_sharded.sbtest1.00001.sql [jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ mv fane_test_sharded@replica.sbtest1.00002.sql fane_test_sharded.sbtest1.00002.sql [jason@blackbox pscale_dump_fane_test_main_fane_test_sharded_20240506_170354]$ pscale database restore-dump --dir ./ fane_test test --overwrite-tables Starting to restore database fane_test from folder ./ Restore is finished! (elapsed time: 1m17.473330261s)
[jason@blackbox ~]$ pscale version pscale version 0.197.0 (build date: 2024-04-23T15:25:04Z commit: d8b065d)