Closed wanghua-lei closed 1 year ago
Hi, you cannot use the tar command provided by the system. You need to make an order-preserving tar (that is, the corresponding text and json for each sample need to be nearby each other in the tar). You can refer to our code to make the tar: https://github.com/LAION-AI/audio-dataset/blob/main/utils/make_tar_utils.py, https://github.com/LAION-AI/audio-dataset/blob/main/utils/make_tar.py
tar
thanks for ans I use command tar -cvf 1.tar fsx/* can solve the problem.
Hi, you cannot use the
tar
command provided by the system. You need to make an order-preserving tar (that is, the corresponding text and json for each sample need to be nearby each other in the tar). You can refer to our code to make the tar: https://github.com/LAION-AI/audio-dataset/blob/main/utils/make_tar_utils.py, https://github.com/LAION-AI/audio-dataset/blob/main/utils/make_tar.py