search

Altinity / clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.
https://altinity.com
Other
1.21k stars 218 forks source link

error restore when table and database have the same name #331

Closed vanyasvl closed 2 years ago

vanyasvl commented 2 years ago

Hello. clickhouse-backup 1.2.2 has a bug: 2021/12/20 14:18:57 debug CREATE TABLE flows.flows.flows UUID '6f89828f-433f-44b3-af89-828f433f04b3' (`Type` Enum8('FLOWUNKNOWN' = 0, 'SFLOW_5' = 1, 'NETFLOW_V5' = 2, 'NETFLOW_V9' = 3, 'IPFIX' = 4), `TimeReceived` DateTime, `SequenceNum` UInt32, `SamplingRate` UInt64, `SamplerAddressIPv4` IPv4, `TimeFlowStart` DateTime, `TimeFlowEnd` DateTime, `Bytes` UInt64, `Packets` UInt64, `SrcAddrIPv4` IPv4, `DstAddrIPv4` IPv4, `SrcAddrIPv6` IPv6, `DstAddrIPv6` IPv6, `EType` UInt32, `Proto` UInt32, `SrcPort` UInt32, `DstPort` UInt32, `InIf` UInt32, `OutIf` UInt32, `SrcMac` UInt64, `DstMac` UInt64, `SrcVlan` UInt32, `DstVlan` UInt32, `VlanId` UInt32, `IPTos` UInt32, `IPTTL` UInt32, `TCPFlags` UInt32, `IcmpType` UInt32, `IcmpCode` UInt32, `IPv6FlowLabel` UInt32, `FragmentId` UInt32, `FragmentOffset` UInt32, `SrcAS` UInt32, `DstAS` UInt32, `NextHopIPv4` IPv4, `NextHopIPv6` IPv6, `NextHopAS` UInt32, `SrcNet` UInt32, `DstNet` UInt32, `HasMPLS` UInt8, `MPLSCount` UInt32, `MPLS1TTL` UInt32, `MPLS1Label` UInt32, `MPLS2TTL` UInt32, `MPLS2Label` UInt32, `MPLS3TTL` UInt32, `MPLS3Label` UInt32, `MPLSLastTTL` UInt32, `MPLSLastLabel` UInt32, `OwnSrcIp` UInt8, `OwnDstIp` UInt8, `DstNetwork` String, `SrcNetwork` String, `DstSubnetworkId` UInt32, `SrcCountry` UInt32, `DstCountry` UInt32, `RouterId` UInt32, `RouterName` String, `LocationId` UInt32, `LocationName` String) ENGINE = MergeTree PARTITION BY toDate(TimeReceived) ORDER BY (TimeFlowStart, SamplerAddressIPv4, NextHopIPv4, NextHopIPv6, SrcAddrIPv4, DstAddrIPv4, SrcAddrIPv6, DstAddrIPv6) SETTINGS storage_policy = 'hybrid_storage', index_granularity = 8192 2021/12/20 14:18:57 error can't create table `flows`.`flows`: code: 62, message: Syntax error: failed at position 32 ('UUID'): UUID '6f89828f-433f-44b3-af89-828f433f04b3' (`Type` Enum8('FLOWUNKNOWN' = 0, 'SFLOW_5' = 1, 'NETFLOW_V5' = 2, 'NETFLOW_V9' = 3, 'IPFIX' = 4), `TimeReceived` Dat. Expected one of: Dot, token after 1 times, please check your schema dependencies

As you can see, long query truncated to CREATE TABLE flows.flows.flows UUID '6f89828f-433f-44b3-af89-828f433f04b3' (`Type` Enum8('FLOWUNKNOWN' = 0, 'SFLOW_5' = 1, 'NETFLOW_V5' = 2, 'NETFLOW_V9' = 3, 'IPFIX' = 4), `TimeReceived` DateTime, `SequenceNum` UInt32, `SamplingRate` UInt64, `SamplerAddressIPv4` IPv4, `TimeFlowStart` Dat and restore desn't work.

Version 1.2.0 doesn't have this bug and works good with the same backup

Slach commented 2 years ago

Could you share:

cat /var/lib/clickhouse/backup/<backup_name>/metadata/flows/flows%2Eflows.json

?

vanyasvl commented 2 years ago

flows.json.txt

Slach commented 2 years ago

ok. it not "truncate", it's a just error message

look like error happens when database and table have the same name and related to fix which was applied for fix https://github.com/AlexAkulov/clickhouse-backup/issues/297

thanks a lot for reporting