Test about sharding-scaling

avalon566 / sharding-scaling

Apache License 2.0

10 stars 4 forks source link

Test about sharding-scaling #27

Open ssxlulu opened 5 years ago

ssxlulu commented 5 years ago

I am doing a simple test about the sharding-scaling and I will write a test document after the test.

avalon566 commented 5 years ago

Great! Here is some test scripts for shrading-scaling sharding-scaling-test, I think is useful for you.

ssxlulu commented 5 years ago

@avalon566 I am being testing.And I found the speed is too slow.I migrate one database with one table with 10,000 thousands data items to another database. And only 11w data items have been migrated after 1 hour. The table has no primary key and no indexes.

avalon566 commented 5 years ago

Tuning performance work will start after the feature and code is stable and no serious bugs.

avalon566 commented 5 years ago

@ssxlulu I've fixed a bug, you can test the speed again.

ssxlulu commented 5 years ago

@avalon566 I found it is too slow to use... It takes me about 3d 17h to migrate about 960w data items, and these data's size only less than 4GB.

avalon566 commented 5 years ago

Do you pull the newest code? I test with ssd disk, 100w data cost 10 minute.

ssxlulu commented 5 years ago

Do you pull the newest code? I test with ssd disk, 100w data cost 10 minute.

I use the Last Saturday's code, and the code may not change a lot after that?

avalon566 commented 5 years ago

Yes, you can checkout this commit 13b1570, and test it.

ssxlulu commented 5 years ago

Following is the test figure.

And the following is the statistics of HOST B:

In conclusion, The disk I/O is idle, and the network traffic is idle too. I want to increse the batch size of data items it migrates each time, but i can not find the parameter.We may make it can be configured.

avalon566 commented 5 years ago

Great! I will finish this feature.

avalon566 commented 5 years ago

And Welcome to contribute this feature.