search

paarthneekhara / text-to-image

Text to image synthesis using thought vectors
MIT License
2.16k stars 394 forks source link

training error for MS-COCO #9

Closed nitish11 closed 7 years ago

nitish11 commented 7 years ago

When I tried to run training on MS-COCO dataset,

python data_loader.py --data_set='MS-COCO' --data_dir='MSCOCO-data'

I downloaded the images and captions from [http://mscoco.org/dataset/#download](MS-COCO dataset) The contents of MSCOCO-data/ folder are as follows: MSCOCO-data/ |-- annotations | |-- captions_train2014.json | |-- captions_val2014.json | |-- instances_train2014.json | |-- instances_val2014.json | |-- person_keypoints_train2014.json | |-- person_keypoints_val2014.json |-- meta_train.pkl |-- train2014 | |-- COCO_train2014_000000000009.jpg | |-- COCO_train2014_000000000025.jpg

I got the below error while running the training code:

Traceback (most recent call last):
  File "data_loader.py", line 111, in <module>
    main()
  File "data_loader.py", line 108, in main
    save_caption_vectors_ms_coco(args.data_dir, args.split, args.batch_size)
  File "data_loader.py", line 39, in save_caption_vectors_ms_coco
    h5f_tv_batch = h5py.File( join(data_dir, 'tvs/'+split + '_tvs_' + str(batch_no)), 'w')
  File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 272, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 98, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2684)
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2642)
  File "h5py/h5f.pyx", line 96, in h5py.h5f.create (/tmp/pip-4rPeHA-build/h5py/h5f.c:2097)
IOError: Unable to create file (Unable to open file: name = '/home/nitish/mscoco-data/tvs/train_tvs_0', errno = 2, error message = 'no such file or directory', flags = 13, o_flags = 242)
nitish11 commented 7 years ago

I fixed the issue by creating the folder 'tvs' in folder MSCOCO-data.

mkdir -p MSCOCO-data/tvs

There was another error at line, which shold be replaced with following line

for i in range(batch_no*batch_size, (batch_no+1)*batch_size):

paarthneekhara commented 7 years ago

Thanks for pointing that out! I haven't tried training the model MS-COCO. Let me know if you face further issues.

dheeru999 commented 7 years ago

I have done with pre-processing using MS-COCO dataset. But When i'am trying to Run (Train .py) By using MS-COCO dataset . I'am facing the below errors.

please provide suggestions to solve those errors.

ERRORS: Traceback (most recent call last): File "/home/mcis-lap-40/raviwork/Tensorflow_Models/text-to-image-generation/train.py", line 227, in main() File "/home/mcis-lap-40/raviwork/Tensorflow_Models/text-to-image-generation/train.py", line 105, in main loaded_data) TypeError: 'NoneType' object is not iterable

Process finished with exit code 1

dheeru999 commented 7 years ago

|-- train2014 | |-- COCO_train2014_000000000009.jpg | |-- COCO_train2014_000000000025.jpg

can you please tell me what is "| |-- COCO_train2014_000000000009.jpg | |-- COCO_train2014_000000000025.jpg" in tain2014 folder

nitish11 commented 7 years ago

@dheeru999 : Can you post the below two lines : 

File "/home/mcis-lap-40/raviwork/Tensorflow_Models/text-to-image-generation/train.py", line 227 
File "/home/mcis-lap-40/raviwork/Tensorflow_Models/text-to-image-generation/train.py", line 105

The error (TypeError: 'NoneType' object is not iterable) implies that your are unable to load the data. So, just after loading the data, just print the size of the variable. Also, check the folder structure and folder names.

I don't have the data now. So, I can't tell you about the below images. | |-- COCO_train2014_000000000009.jpg | |-- COCO_train2014_000000000025.jpg

dheeru999 commented 7 years ago

I don't have the data now. So, I can't tell you about the below images. | |-- COCO_train2014_000000000009.jpg | |-- COCO_train2014_000000000025.jpg

From where u downloaded the data , Can u provide the link. "In place of this can we load train 2014 images "

dheeru999 commented 7 years ago

Have u trained on MS-COCO data set,did u got any result.

Can u provide us some suggestions that what have to be modified in the original code** for MS-COCO dataset.

nitish11 commented 7 years ago

I have trained on MS-COCO dataset and I was able to run it successfully. I don't have the result form now. Download the data from MS-COCO official page as http://mscoco.org/dataset/#download And then, put data, annotations as mentioned before. The contents of MSCOCO-data/ folder are as follows: MSCOCO-data/ |--tvs |-- annotations | |-- captions_train2014.json | |-- captions_val2014.json | |-- instances_train2014.json | |-- instances_val2014.json | |-- person_keypoints_train2014.json | |-- person_keypoints_val2014.json |-- meta_train.pkl |-- train2014 | |-- COCO_train2014_000000000009.jpg | |-- COCO_train2014_000000000025.jpg

dheeru999 commented 7 years ago

Thank U so much for rersponding. Data paths are perfectly set,but still not able to run .

Do u have the modified code. If possible can u please share with us.

anushaGundapaneni commented 7 years ago

If you download the dataset properly and put them in a folders,you will not get those image loaded errors.

anushaGundapaneni commented 7 years ago

After doing all the steps which you have discussed above, I'am facing the error called "ImageList"

Process has been started but stopped for 1st iteration. Please provide some suggestions.

d1 1.63985 d2 0.371909 d3 0.157891 D 2.16965 Traceback (most recent call last): File "/home/Tensorflow_Models/text-to-image-generation/train.py", line 250, in main() File "home/Tensorflow_Models/text-to-image-generation/train.py", line 139, in main print "LOSSES", d_loss, g_loss, batch_no, i, len(loaded_data['image_list']) / args.batch_size KeyError: 'image_list' LOSSES 2.16965 0.000277596 0 0

Please respond to this.

madmalar commented 5 years ago

I have done with pre-processing using MS-COCO dataset. But When i'am trying to Run (Train .py) By using MS-COCO dataset . I'am facing the below errors.

please provide suggestions to solve those errors.

ERRORS: Traceback (most recent call last): File "/home/mcis-lap-40/raviwork/Tensorflow_Models/text-to-image-generation/train.py", line 227, in main() File "/home/mcis-lap-40/raviwork/Tensorflow_Models/text-to-image-generation/train.py", line 105, in main loaded_data) TypeError: 'NoneType' object is not iterable

Process finished with exit code 1

I faced the same issue.Solved by using the below command for training: python train.py --data_set="mscoco" --data_dir=Data/MSCOCO-data

madmalar commented 5 years ago

After doing all the steps which you have discussed above, I'am facing the error called "ImageList"

Process has been started but stopped for 1st iteration. Please provide some suggestions.

d1 1.63985 d2 0.371909 d3 0.157891 D 2.16965 Traceback (most recent call last): File "/home/Tensorflow_Models/text-to-image-generation/train.py", line 250, in main() File "home/Tensorflow_Models/text-to-image-generation/train.py", line 139, in main print "LOSSES", d_loss, g_loss, batch_no, i, len(loaded_data['image_list']) / args.batch_size KeyError: 'image_list' LOSSES 2.16965 0.000277596 0 0

Please respond to this.

I faced the same issue. Solved by replacing the line print "LOSSES", d_loss, g_loss, batch_no, i, len(loaded_data['image_list']) / args.batch_size with *print "LOSSES", d_loss, g_loss, batch_no, i