search

guotong1988 / NL2SQL-RULE

Content Enhanced BERT-based Text-to-SQL Generation https://arxiv.org/abs/1910.07179
188 stars 48 forks source link

Why no use of BertTokenizer? #24

Closed BoCiFOM closed 1 year ago

BoCiFOM commented 1 year ago

Hello, why there is no use of BertTokenizer and what is the advantage of the custom BasicTokenizer?

Best Regards!

guotong1988 commented 1 year ago

Tokenizer is not the key factor here.