Open AdamWP opened 6 years ago
How does this project handle special characters such as currency symbols, apostrophes, percent symbols, etc (¢, $, %, ')? Can I just adjust the vocabulary and include these symbols in my dataset?
TARGET_VOCAB_SIZE = 26+10+3 # 0: PADDING, 1: GO, 2: EOS, >2: 0-9, a-z)
Please help me on including the special characters
How does this project handle special characters such as currency symbols, apostrophes, percent symbols, etc (¢, $, %, ')? Can I just adjust the vocabulary and include these symbols in my dataset?
TARGET_VOCAB_SIZE = 26+10+3 # 0: PADDING, 1: GO, 2: EOS, >2: 0-9, a-z)