Closed tom1997 closed 1 year ago
manifests = read_manifests_if_cached(
dataset_parts=dataset_parts,
output_dir=args.src_dir,
prefix=args.prefix,
suffix=args.suffix,
types=["recordings", "supervisions", "cuts"],
)
maybe you can check your manifests first
manifests = read_manifests_if_cached( dataset_parts=dataset_parts, output_dir=args.src_dir, prefix=args.prefix, suffix=args.suffix, types=["recordings", "supervisions", "cuts"], )
maybe you can check your manifests first
ok, I just print it
{'train': {'recordings': CutSet(len=0) [underlying data type: <class 'dict'>], 'supervisions': CutSet(len=0) [underlying data type: <class 'dict'>]}, 'dev': {'recordings': CutSet(len=0) [underlying data type: <class 'dict'>], 'supervisions': CutSet(len=0) [underlying data type: <class 'dict'>]}, 'test': {'recordings': CutSet(len=0) [underlying data type: <class 'dict'>], 'supervisions': CutSet(len=0) [underlying data type: <class 'dict'>]}
I just test aishell1, and here is the manifests
-rw-r--r-- 1 root root 54 Jul 13 03:29 aishell_recordings_dev.jsonl.gz
-rw-r--r-- 1 root root 55 Jul 13 03:29 aishell_recordings_test.jsonl.gz
-rw-r--r-- 1 root root 56 Jul 13 03:29 aishell_recordings_train.jsonl.gz
-rw-r--r-- 1 root root 56 Jul 13 03:29 aishell_supervisions_dev.jsonl.gz
-rw-r--r-- 1 root root 57 Jul 13 03:29 aishell_supervisions_test.jsonl.gz
-rw-r--r-- 1 root root 58 Jul 13 03:29 aishell_supervisions_train.jsonl.gz
CutSet is null. And I try re-generate again by prepare.sh, it remain the same.
I solved it by setting the right path of dataset, refer to lhotse
I solved it by setting the right path of dataset, refer to lhotse
can you explain it more detail
I tried to trained on aishell4, after prepare the dataset manifests I run
main()
File "bin/tokenizer.py", line 227, in main
for c in tqdm(cut_set):
File "/zhangyingxian/miniconda3/envs/valle/lib/python3.8/site-packages/tqdm/std.py", line 1178, in iter
for obj in iterable:
TypeError: 'NoneType' object is not iterable
python3 bin/tokenizer.py --dataset-parts "train_L train_S train_M test" --text-extractor "pypinyin_initials_finals" --audio-extractor Encodec --batch-duration 400 --prefix aishell4 --src-dir ../../../dataset/aishell4/manifects/ --output-dir data/tokenized
And I got the following log Computing features in batches: 0%| | 0/30 [00:00<?, ?it/s]0it [00:00, ?it/s] Traceback (most recent call last): File "bin/tokenizer.py", line 263, inI already re-install the least lhotse version, how could I solve it?