Altinity / clickhouse-mysql-data-reader

utility to read mysql data
MIT License
368 stars 94 forks source link

crach for mysql syncing #139

Closed xiaods closed 6 years ago

xiaods commented 6 years ago
1. /usr/bin/clickhouse-server(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x22) [0x3019952]            
2. /usr/bin/clickhouse-server(DB::Context::getTableImpl(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, DB::Exception*) const+0x1f3) [0x708b523]                                                                                   
3. /usr/bin/clickhouse-server(DB::Context::getTable(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x73) [0x708b9f3]
4. /usr/bin/clickhouse-server(DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery const&)+0x191) [0x70edae1]                                                                 
5. /usr/bin/clickhouse-server(DB::InterpreterInsertQuery::execute()+0x63) [0x70ee8c3]
6. /usr/bin/clickhouse-server() [0x71a0727]
7. /usr/bin/clickhouse-server(DB::executeQuery(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum)+0x8a) [0x71a245a]
8. /usr/bin/clickhouse-server(DB::TCPHandler::runImpl()+0x4d9) [0x3027179]
9. /usr/bin/clickhouse-server(DB::TCPHandler::run()+0x2b) [0x302836b]
10. /usr/bin/clickhouse-server(Poco::Net::TCPServerConnection::start()+0xf) [0x96107df]
11. /usr/bin/clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0x16a) [0x9610bba]
12. /usr/bin/clickhouse-server(Poco::PooledThread::run()+0x77) [0x972d6b7]
13. /usr/bin/clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x38) [0x9729228]
14. /usr/bin/clickhouse-server() [0x9e659ef]
15. /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f0dbe3286db]
16. /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f0dbd8a788f]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/clickhouse-mysql-data-reader/clickhouse_mysql/tablemigrator.py", line 194, in migrate_one_table_data                                                               
    self.chwriter.insert(event)
  File "/root/clickhouse-mysql-data-reader/clickhouse_mysql/writer/chwriter.py", line 88, in insert                                                                              
    sys.exit(0)
SystemExit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/clickhouse-mysql-data-reader/clickhouse_mysql/main.py", line 140, in run
    migrator.migrate_all_tables_data()
  File "/root/clickhouse-mysql-data-reader/clickhouse_mysql/tablemigrator.py", line 160, in migrate_all_tables_data                                                              
    self.migrate_one_table_data(db=db, table=table)
  File "/root/clickhouse-mysql-data-reader/clickhouse_mysql/tablemigrator.py", line 200, in migrate_one_table_data                                                               
    self.host,
AttributeError: 'TableMigrator' object has no attribute 'host'
=============
'TableMigrator' object has no attribute 'host'

sql:

python3 -m clickhouse_mysql.main \
    --src-server-id=1 \
    --src-wait \
    --nice-pause=1 \
    --src-host=172.31.x.x \
    --src-user=reader \
    --src-password=xx \
    --src-tables=x.x \
    --dst-host=192.168.x.x \
    --dst-schema=chain \
    --dst-table=chain_tx.chain_tx \
    --migrate-table \
    --pump-data \
    --csvpool
xiaods commented 6 years ago

sometimes it raised below error msg:

clickhouse-3-0.clickhouse-3.clickhouse.svc.cluster.local :) use chain_tx;

USE chain_tx      

Received exception from server (version 18.12.17):
Code: 81. DB::Exception: Received from 192.168.3.105:9000. DB::Exception: Database chain_tx doesn't exist.                                                                       

0 rows in set. Elapsed: 0.002 sec.        

i don't know why the database is missing, i have already create by manual step.

xiaods commented 6 years ago

hi team, i carefully read README again, and retry again the steps. it works like a charm. sorry for the naive question. Clickhouse is cool db.

justlooks commented 5 years ago

i have same problem ,can you post the solution?@xiaods

xiaods commented 5 years ago

@justlooks, i have forget it. so please read README, it will help you.