Open hisoka418 opened 4 years ago
i'm thinking if you have a tfrecords
directory in the same level of this tfrecords_mpii.py
script. basically, what this script does is to take annotations and images from one location and convert them into a TFRecords serialized format at another location.
not working
i am use : Ubuntu 18.04 gpu: RTX2080 TI cuda:10.1 python 3.6 tensorflow==2.1.0 Pillow==7.0.0 ray==0.8.0 loguru==0.4.0 tensorflow-addons==0.8.2 google-cloud-storage==1.14.0
i download anootation from >> https://onedrive.live.com/?authkey=%21AMdfUcJgrZBwTRU&id=93774C670BD4F835%211101&cid=93774C670BD4F835
and download mpii images from >> https://datasets.d2.mpi-inf.mpg.de/andriluka14cvpr/mpii_human_pose_v1.tar.gz.
i make directory dataset and it contain :
dataset |-- tfrecords_mpii.py |--tfrecords_mpii |--mpii_human_pose_v1_u12_2 -- |-- train.jsom -- |-- validation.json -- |-- test.json -- |-- trainval.json -- |-- valid.json |--mpii -- |-- images ----------|-- 000001163.jpg ----------|-- 000003072.jpg
i got this error message below i want to know where is my fault , and thank you for replying
(saad) saad@Z390:~/HG/dataset$ python tfrecords_mpii.py
2020-05-22 14:17:51,499 INFO resource_spec.py:216 -- Starting Ray with 6.45 GiB memory available for workers and up to 3.24 GiB for objects. You can adjust these settings with ray.init(memory=
First train annotation: {'filename': '015601864.jpg', 'filepath': './mpii/images/015601864.jpg', 'joints_visibility': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'joints': [[620.0, 394.0], [616.0, 269.0], [573.0, 185.0], [647.0, 188.0], [661.0, 221.0], [656.0, 231.0], [610.0, 187.0], [647.0, 176.0], [637.0201, 189.8183], [695.9799, 108.1817], [606.0, 217.0], [553.0, 161.0], [601.0, 167.0], [692.0, 185.0], [693.0, 240.0], [688.0, 313.0]]}
First val annotation: {'filename': '005808361.jpg', 'filepath': './mpii/images/005808361.jpg', 'joints_visibility': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'joints': [[804.0, 711.0], [816.0, 510.0], [908.0, 438.0], [1040.0, 454.0], [906.0, 528.0], [883.0, 707.0], [974.0, 446.0], [985.0, 253.0], [982.7591, 235.9694], [962.2409, 80.0306], [869.0, 214.0], [798.0, 340.0], [902.0, 253.0], [1067.0, 253.0], [1167.0, 353.0], [1142.0, 478.0]]}
Start to build TF Records.
(pid=6537) start to build tf records for ./tfrecords/train_0008_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0009_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0010_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0011_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0012_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0013_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0014_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0015_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0017_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0019_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0021_of_0064.tfrecords
(pid=6537) start to build tf records for ./tfrecords/train_0023_of_0064.tfrecords
(pid=6542) start to build tf records for ./tfrecords/train_0007_of_0064.tfrecords
(pid=6542) start to build tf records for ./tfrecords/train_0039_of_0064.tfrecords
(pid=6536) start to build tf records for ./tfrecords/train_0005_of_0064.tfrecords
(pid=6536) start to build tf records for ./tfrecords/train_0043_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0006_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0016_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0022_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0024_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0028_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0030_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0035_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0040_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0046_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0004_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0018_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0020_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0025_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0026_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0029_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0031_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0032_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0037_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0042_of_0064.tfrecords
(pid=6538) start to build tf records for ./tfrecords/train_0002_of_0064.tfrecords
(pid=6538) start to build tf records for ./tfrecords/train_0033_of_0064.tfrecords
(pid=6538) start to build tf records for ./tfrecords/train_0036_of_0064.tfrecords
(pid=6538) start to build tf records for ./tfrecords/train_0041_of_0064.tfrecords
(pid=6535) start to build tf records for ./tfrecords/train_0001_of_0064.tfrecords
(pid=6535) start to build tf records for ./tfrecords/train_0034_of_0064.tfrecords
(pid=6535) start to build tf records for ./tfrecords/train_0038_of_0064.tfrecords
(pid=6535) start to build tf records for ./tfrecords/train_0044_of_0064.tfrecords
(pid=6542) start to build tf records for ./tfrecords/train_0047_of_0064.tfrecords
(pid=6538) start to build tf records for ./tfrecords/train_0048_of_0064.tfrecords
Traceback (most recent call last):
File "tfrecords_mpii.py", line 157, in File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/ray/worker.py", line 1457, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(NotFoundError): ray::__main__.build_single_tfrecord() (pid=6535, ip=10.20.30.237)
File "python/ray/_raylet.pyx", line 636, in ray._raylet.execute_task
File "tfrecords_mpii.py", line 91, in build_single_tfrecord
with tf.io.TFRecordWriter(path) as writer:
File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/tensorflow_core/python/lib/io/tf_record.py", line 218, in __init__
compat.as_bytes(path), options._as_record_writer_options(), status)
File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/tensorflow_core/python/framework/errors_impl.py", line 554, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: ./tfrecords/train_0001_of_0064.tfrecords; No such file or directory
(pid=6537) start to build tf records for ./tfrecords/train_0027_of_0064.tfrecords
(pid=6542) start to build tf records for ./tfrecords/train_0051_of_0064.tfrecords
(pid=6542) start to build tf records for ./tfrecords/train_0058_of_0064.tfrecords
(pid=6542) start to build tf records for ./tfrecords/train_0064_of_0064.tfrecords
(pid=6536) start to build tf records for ./tfrecords/train_0049_of_0064.tfrecords
(pid=6536) start to build tf records for ./tfrecords/train_0056_of_0064.tfrecords
(pid=6536) start to build tf records for ./tfrecords/train_0063_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0052_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0057_of_0064.tfrecords
(pid=6539) start to build tf records for ./tfrecords/train_0062_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0045_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0054_of_0064.tfrecords
(pid=6541) start to build tf records for ./tfrecords/train_0059_of_0064.tfrecords
(pid=6538) start to build tf records for ./tfrecords/train_0053_of_0064.tfrecords
(pid=6538) start to build tf records for ./tfrecords/train_0060_of_0064.tfrecords
(pid=6540) start to build tf records for ./tfrecords/train_0003_of_0064.tfrecords
(pid=6535) start to build tf records for ./tfrecords/train_0050_of_0064.tfrecords
(pid=6535) start to build tf records for ./tfrecords/train_0055_of_0064.tfrecords
(pid=6535)
start to build tf records for ./tfrecords/train_0061_of_0064.tfrecords`
so the script actually create tfrecords in a directory called tfrecords
i made a fix here: https://github.com/ethanyanjiali/deep-vision/pull/10
or you can manually create it like:
dataset
|-- tfrecords_mpii.py
|--tfrecords
--|--train_0055_of_0064.tfrecords
--|--...
|--mpii_human_pose_v1_u12_2
-- |-- train.jsom
-- |-- validation.json
-- |-- test.json
-- |-- trainval.json
-- |-- valid.json
|--mpii
-- |-- images
----------|-- 000001163.jpg
----------|-- 000003072.jpg
On Fri, May 22, 2020 at 4:24 AM Saad Al-Sheekh notifications@github.com wrote:
not working
i am use : Ubuntu 18.04 gpu: RTX2080 TI cuda:10.1 python 3.6 tensorflow==2.1.0 Pillow==7.0.0 ray==0.8.0 loguru==0.4.0 tensorflow-addons==0.8.2 google-cloud-storage==1.14.0
i download anootation from >> https://onedrive.live.com/?authkey=%21AMdfUcJgrZBwTRU&id=93774C670BD4F835%211101&cid=93774C670BD4F835
and download mpii images from >> https://datasets.d2.mpi-inf.mpg.de/andriluka14cvpr/mpii_human_pose_v1.tar.gz .
i make directory dataset and it contain :
dataset |-- tfrecords_mpii.py |--tfrecords_mpii |--mpii_human_pose_v1_u12_2 -- |-- train.jsom -- |-- validation.json -- |-- test.json -- |-- trainval.json -- |-- valid.json |--mpii -- |-- images |-- 000001163.jpg |-- 000003072.jpg
i got this error message below i want to know where is my fault , and thank you for replying
`(saad) saad@Z390:~/HG/dataset$ python tfrecords_mpii.py
2020-05-22 14:17:51,499 INFO resource_spec.py:216 -- Starting Ray with 6.45 GiB memory available for workers and up to 3.24 GiB for objects. You can adjust these settings with ray.init(memory=, object_store_memory=). Start to parse annotations. First train annotation: {'filename': '015601864.jpg', 'filepath': './mpii/images/015601864.jpg', 'joints_visibility': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'joints': [[620.0, 394.0], [616.0, 269.0], [573.0, 185.0], [647.0, 188.0], [661.0, 221.0], [656.0, 231.0], [610.0, 187.0], [647.0, 176.0], [637.0201, 189.8183], [695.9799, 108.1817], [606.0, 217.0], [553.0, 161.0], [601.0, 167.0], [692.0, 185.0], [693.0, 240.0], [688.0, 313.0]]} First val annotation: {'filename': '005808361.jpg', 'filepath': './mpii/images/005808361.jpg', 'joints_visibility': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'joints': [[804.0, 711.0], [816.0, 510.0], [908.0, 438.0], [1040.0, 454.0], [906.0, 528.0], [883.0, 707.0], [974.0, 446.0], [985.0, 253.0], [982.7591, 235.9694], [962.2409, 80.0306], [869.0, 214.0], [798.0, 340.0], [902.0, 253.0], [1067.0, 253.0], [1167.0, 353.0], [1142.0, 478.0]]} Start to build TF Records. (pid=6537) start to build tf records for ./tfrecords/train_0008_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0009_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0010_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0011_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0012_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0013_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0014_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0015_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0017_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0019_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0021_of_0064.tfrecords (pid=6537) start to build tf records for ./tfrecords/train_0023_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0007_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0039_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0005_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0043_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0006_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0016_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0022_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0024_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0028_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0030_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0035_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0040_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0046_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0004_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0018_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0020_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0025_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0026_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0029_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0031_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0032_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0037_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0042_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0002_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0033_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0036_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0041_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0001_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0034_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0038_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0044_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0047_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0048_of_0064.tfrecords Traceback (most recent call last): File "tfrecords_mpii.py", line 157, in main() File "tfrecords_mpii.py", line 149, in main build_tf_records(train_annotations, num_train_shards, 'train') File "tfrecords_mpii.py", line 110, in build_tf_records ray.get(futures) File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/ray/worker.py", line 1457, in get raise value.as_instanceof_cause() ray.exceptions.RayTaskError(NotFoundError): ray::main.build_single_tfrecord() (pid=6535, ip=10.20.30.237) File "python/ray/_raylet.pyx", line 636, in ray._raylet.execute_task File "tfrecords_mpii.py", line 91, in build_single_tfrecord with tf.io.TFRecordWriter(path) as writer: File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/tensorflow_core/python/lib/io/tf_record.py", line 218, in init compat.as_bytes(path), options._as_record_writer_options(), status) File "/home/saad/anaconda3/envs/saad/lib/python3.7/site-packages/tensorflow_core/python/framework/errors_impl.py", line 554, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.NotFoundError: ./tfrecords/train_0001_of_0064.tfrecords; No such file or directory (pid=6537) start to build tf records for ./tfrecords/train_0027_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0051_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0058_of_0064.tfrecords (pid=6542) start to build tf records for ./tfrecords/train_0064_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0049_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0056_of_0064.tfrecords (pid=6536) start to build tf records for ./tfrecords/train_0063_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0052_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0057_of_0064.tfrecords (pid=6539) start to build tf records for ./tfrecords/train_0062_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0045_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0054_of_0064.tfrecords (pid=6541) start to build tf records for ./tfrecords/train_0059_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0053_of_0064.tfrecords (pid=6538) start to build tf records for ./tfrecords/train_0060_of_0064.tfrecords (pid=6540) start to build tf records for ./tfrecords/train_0003_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0050_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0055_of_0064.tfrecords (pid=6535) start to build tf records for ./tfrecords/train_0061_of_0064.tfrecords
`
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ethanyanjiali/deep-vision/issues/9#issuecomment-632642074, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCHB75S3PKDGARIWAYIXE3RSZOG3ANCNFSM4NG7RC5A .
HI Mr. **Ethan Yanjia Li
when i run this code
tfrecords_mpii.py
to generate tf record , but i got this errorthanks for sharing your knowledge in pose estimation the references on this top is much less I appreciate your helping