Closed zhujiem closed 6 months ago
streaming: False should be used in dataset_config
Fix issue in loading pretrained_emb in npz format in tokenzier.py
elif pretrain_path.endswith(".npz"): npz = np.load(pretrain_path) keys = npz["key"]
streaming: False should be used in dataset_config