search

Altinity / clickhouse-mysql-data-reader

utility to read mysql data
MIT License
368 stars 94 forks source link

Special characters in clickhouse password causes CSV dump to fail #168

Closed moonlightelite closed 4 years ago

moonlightelite commented 4 years ago

If one uses a clickhouse account password with special characters and tries to use pump-data + CSV, during the tail | clickhouse-client call (thru Bash), the password would be escaped by Bash and causes authentication failure.

2020-01-24 10:38:10,209/1579891090.209181:INFO:starting clickhouse-client process 2020-01-24 10:38:10,209/1579891090.209221:DEBUG:starting tail -n +2 'yyyyyyy_1579891090.202464_12a560ba-7819-4d01-a114-053757d98264.csv' | clickhouse-client --host=127.0.0.1 --port=9000 --user=default --password=xxxxxxxx --query='INSERT INTO ....... FORMAT CSV' Code: 193. DB::Exception: Received from 127.0.0.1:9000. DB::Exception: Wrong password for user default.

tail: write error: Broken pipe 2020-01-24 10:38:10,230/1579891090.230426:DEBUG:class:<class 'clickhouse_mysql.writer.processwriter.ProcessWriter'> process() done 2020-01-24 10:38:11,225/1579891091.225788:DEBUG:Check events in binlog stream

A propose fix would be to single quote the password during argument construction (https://github.com/Altinity/clickhouse-mysql-data-reader/blob/c8fdf1b9f83e35694f92b2c1b8e15d13ba7063cb/clickhouse_mysql/writer/chcsvwriter.py#L89)

            if self.password:
                choptions += " --password='" + self.password + "'"
            bash = "tail -n +2 '{0}' | clickhouse-client {1} --query='{2}'".format(
                event.filename,
                choptions,
                sql,
            )

Thanks

sunsingerus commented 4 years ago

The more generic way is to use quote/escape values in bash commands Please check