Error in generate TF Records hourglass

HI Mr. **Ethan Yanjia Li

when i run this code tfrecords_mpii.py to generate tf record , but i got this error

tensorflow.python.framework.errors_impl.NotFoundError: ./tfrecords/train_0001_of_0064.tfrecords; No such file or directory

thanks for sharing your knowledge in pose estimation the references on this top is much less I appreciate your helping

i'm thinking if you have a tfrecords directory in the same level of this tfrecords_mpii.py script. basically, what this script does is to take annotations and images from one location and convert them into a TFRecords serialized format at another location.

not working

i am use : Ubuntu 18.04 gpu: RTX2080 TI cuda:10.1 python 3.6 tensorflow==2.1.0 Pillow==7.0.0 ray==0.8.0 loguru==0.4.0 tensorflow-addons==0.8.2 google-cloud-storage==1.14.0

i download anootation from >> https://onedrive.live.com/?authkey=%21AMdfUcJgrZBwTRU&id=93774C670BD4F835%211101&cid=93774C670BD4F835

and download mpii images from >> https://datasets.d2.mpi-inf.mpg.de/andriluka14cvpr/mpii_human_pose_v1.tar.gz.

i make directory dataset and it contain :

dataset |-- tfrecords_mpii.py |--tfrecords_mpii |--mpii_human_pose_v1_u12_2 -- |-- train.jsom -- |-- validation.json -- |-- test.json -- |-- trainval.json -- |-- valid.json |--mpii -- |-- images ----------|-- 000001163.jpg ----------|-- 000003072.jpg

i got this error message below i want to know where is my fault , and thank you for replying

(saad) saad@Z390:~/HG/dataset$ python tfrecords_mpii.py

2020-05-22 14:17:51,499 INFO resource_spec.py:216 -- Starting Ray with 6.45 GiB memory available for workers and up to 3.24 GiB for objects. You can adjust these settings with ray.init(memory=, object_store_memory=). Start to parse annotations.

First train annotation: {'filename': '015601864.jpg', 'filepath': './mpii/images/015601864.jpg', 'joints_visibility': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'joints': [[620.0, 394.0], [616.0, 269.0], [573.0, 185.0], [647.0, 188.0], [661.0, 221.0], [656.0, 231.0], [610.0, 187.0], [647.0, 176.0], [637.0201, 189.8183], [695.9799, 108.1817], [606.0, 217.0], [553.0, 161.0], [601.0, 167.0], [692.0, 185.0], [693.0, 240.0], [688.0, 313.0]]} First val annotation: {'filename': '005808361.jpg', 'filepath': './mpii/images/005808361.jpg', 'joints_visibility': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'joints': [[804.0, 711.0], [816.0, 510.0], [908.0, 438.0], [1040.0, 454.0], [906.0, 528.0], [883.0, 707.0], [974.0, 446.0], [985.0, 253.0], [982.7591, 235.9694], [962.2409, 80.0306], [869.0, 214.0], [798.0, 340.0], [902.0, 253.0], [1067.0, 253.0], [1167.0, 353.0], [1142.0, 478.0]]} Start to build TF Records. (pid=6537) start to build tf records for ./tfrecords/train_0008_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0009_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0010_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0011_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0012_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0013_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0014_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0015_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0017_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0019_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0021_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0023_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0007_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0039_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0005_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0043_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0006_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0016_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0022_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0024_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0028_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0030_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0035_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0040_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0046_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0004_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0018_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0020_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0025_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0026_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0029_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0031_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0032_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0037_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0042_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0002_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0033_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0036_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0041_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0001_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0034_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0038_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0044_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0047_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0048_of_0064.tfrecords Traceback (most recent call last): File "tfrecords_mpii.py", line 157, in main() File "tfrecords_mpii.py", line 149, in main build_tf_records(train_annotations, num_train_shards, 'train') File "tfrecords_mpii.py", line 110, in build_tf_records ray.get(futures) File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/ray/worker.py", line 1457, in get raise value.as_instanceof_cause() ray.exceptions.RayTaskError(NotFoundError): ray::__main__.build_single_tfrecord() (pid=6535, ip=10.20.30.237) File "python/ray/_raylet.pyx", line 636, in ray._raylet.execute_task File "tfrecords_mpii.py", line 91, in build_single_tfrecord with tf.io.TFRecordWriter(path) as writer: File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/tensorflow_core/python/lib/io/tf_record.py", line 218, in __init__ compat.as_bytes(path), options._as_record_writer_options(), status) File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/tensorflow_core/python/framework/errors_impl.py", line 554, in __exit__ c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.NotFoundError: ./tfrecords/train_0001_of_0064.tfrecords; No such file or directory (pid=6537) start to build tf records for ./tfrecords/train_0027_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0051_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0058_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0064_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0049_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0056_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0063_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0052_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0057_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0062_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0045_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0054_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0059_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0053_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0060_of_0064.tfrecords (pid=6540) start to build tf records for ./tfrecords/train_0003_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0050_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0055_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0061_of_0064.tfrecords`

so the script actually create tfrecords in a directory called tfrecords i made a fix here: https://github.com/ethanyanjiali/deep-vision/pull/10 or you can manually create it like:

dataset
|-- tfrecords_mpii.py
|--tfrecords
--|--train_0055_of_0064.tfrecords
--|--...
|--mpii_human_pose_v1_u12_2
-- |-- train.jsom
-- |-- validation.json
-- |-- test.json
-- |-- trainval.json
-- |-- valid.json
|--mpii
-- |-- images
----------|-- 000001163.jpg
----------|-- 000003072.jpg

On Fri, May 22, 2020 at 4:24 AM Saad Al-Sheekh notifications@github.com wrote:

not working

i am use : Ubuntu 18.04 gpu: RTX2080 TI cuda:10.1 python 3.6 tensorflow==2.1.0 Pillow==7.0.0 ray==0.8.0 loguru==0.4.0 tensorflow-addons==0.8.2 google-cloud-storage==1.14.0

i download anootation from >> https://onedrive.live.com/?authkey=%21AMdfUcJgrZBwTRU&id=93774C670BD4F835%211101&cid=93774C670BD4F835

and download mpii images from >> https://datasets.d2.mpi-inf.mpg.de/andriluka14cvpr/mpii_human_pose_v1.tar.gz .

i make directory dataset and it contain :

dataset |-- tfrecords_mpii.py |--tfrecords_mpii |--mpii_human_pose_v1_u12_2 -- |-- train.jsom -- |-- validation.json -- |-- test.json -- |-- trainval.json -- |-- valid.json |--mpii -- |-- images |-- 000001163.jpg |-- 000003072.jpg

i got this error message below i want to know where is my fault , and thank you for replying

`(saad) saad@Z390:~/HG/dataset$ python tfrecords_mpii.py

2020-05-22 14:17:51,499 INFO resource_spec.py:216 -- Starting Ray with 6.45 GiB memory available for workers and up to 3.24 GiB for objects. You can adjust these settings with ray.init(memory=, object_store_memory=). Start to parse annotations. First train annotation: {'filename': '015601864.jpg', 'filepath': './mpii/images/015601864.jpg', 'joints_visibility': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'joints': [[620.0, 394.0], [616.0, 269.0], [573.0, 185.0], [647.0, 188.0], [661.0, 221.0], [656.0, 231.0], [610.0, 187.0], [647.0, 176.0], [637.0201, 189.8183], [695.9799, 108.1817], [606.0, 217.0], [553.0, 161.0], [601.0, 167.0], [692.0, 185.0], [693.0, 240.0], [688.0, 313.0]]} First val annotation: {'filename': '005808361.jpg', 'filepath': './mpii/images/005808361.jpg', 'joints_visibility': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'joints': [[804.0, 711.0], [816.0, 510.0], [908.0, 438.0], [1040.0, 454.0], [906.0, 528.0], [883.0, 707.0], [974.0, 446.0], [985.0, 253.0], [982.7591, 235.9694], [962.2409, 80.0306], [869.0, 214.0], [798.0, 340.0], [902.0, 253.0], [1067.0, 253.0], [1167.0, 353.0], [1142.0, 478.0]]} Start to build TF Records. (pid=6537) start to build tf records for ./tfrecords/train_0008_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0009_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0010_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0011_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0012_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0013_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0014_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0015_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0017_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0019_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0021_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0023_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0007_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0039_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0005_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0043_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0006_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0016_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0022_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0024_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0028_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0030_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0035_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0040_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0046_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0004_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0018_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0020_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0025_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0026_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0029_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0031_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0032_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0037_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0042_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0002_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0033_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0036_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0041_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0001_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0034_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0038_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0044_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0047_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0048_of_0064.tfrecords Traceback (most recent call last): File "tfrecords_mpii.py", line 157, in main() File "tfrecords_mpii.py", line 149, in main build_tf_records(train_annotations, num_train_shards, 'train') File "tfrecords_mpii.py", line 110, in build_tf_records ray.get(futures) File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/ray/worker.py", line 1457, in get raise value.as_instanceof_cause() ray.exceptions.RayTaskError(NotFoundError): ray::main.build_single_tfrecord() (pid=6535, ip=10.20.30.237) File "python/ray/_raylet.pyx", line 636, in ray._raylet.execute_task File "tfrecords_mpii.py", line 91, in build_single_tfrecord with tf.io.TFRecordWriter(path) as writer: File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/tensorflow_core/python/lib/io/tf_record.py", line 218, in init compat.as_bytes(path), options._as_record_writer_options(), status) File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/tensorflow_core/python/framework/errors_impl.py", line 554, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.NotFoundError: ./tfrecords/train_0001_of_0064.tfrecords; No such file or directory (pid=6537) start to build tf records for ./tfrecords/train_0027_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0051_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0058_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0064_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0049_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0056_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0063_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0052_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0057_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0062_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0045_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0054_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0059_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0053_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0060_of_0064.tfrecords (pid=6540) start to build tf records for ./tfrecords/train_0003_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0050_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0055_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0061_of_0064.tfrecords

`

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ethanyanjiali/deep-vision/issues/9#issuecomment-632642074, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCHB75S3PKDGARIWAYIXE3RSZOG3ANCNFSM4NG7RC5A .

ethanyanjiali / deep-vision

Error in generate TF Records hourglass #9

HI Mr. **Ethan Yanjia Li