bshillingford / python-torchfile

Deserialize Lua torch-serialized objects from Python
BSD 3-Clause "New" or "Revised" License
216 stars 25 forks source link

Converting text file to .t7 file #14

Open Shubham-kale opened 6 years ago

Shubham-kale commented 6 years ago

While trying to run a code which uses torchfile I encountered the following error :-

T7ReaderException                         Traceback (most recent call last)
<ipython-input-11-89caedbea733> in <module>()
      5 else :
      6     cap_path = "Data/flowers/example_captions"
----> 7 t_file = torchfile.load(cap_path)
      8 #torch.save(cap_path,'ascii')# ///// 'output.t7',
      9 captions_list = t_file.raw_txt

~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/torchfile.py in load(filename, **kwargs)
    422     with open(filename, 'rb') as f:
    423         reader = T7Reader(f, **kwargs)
--> 424         return reader.read_obj()

~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/torchfile.py in read_obj(self)
    412         else:
    413             raise T7ReaderException(
--> 414                 "unknown object type / typeidx: {}".format(typeidx))
    415 
    416 

T7ReaderException: unknown object type / typeidx: 1936287828

I gather that .t7 is not a file format but just an extension , is that true ? I tried renaming the file under consideration with the extension .t7 , but things didn't workout so I want to know how to convert a text file to .t7 file ?

bshillingford commented 6 years ago

Do you have the most recent version from GitHub? That error means the file contains an instance of an unknown class, not anything to do with the file format itself. The latest version will forcefully try to deserialize all types. You can get the same behaviour in old versions with the kwarg force_deserialize_classes (see the docstrings for details).

Shubham-kale commented 6 years ago

Previous error is now changed to :- AttributeError Traceback (most recent call last)

in () 8 t_file = torchfile.load(cap_path) 9 print("type is :-",type(t_file)) ---> 10 captions_list = t_file.raw_txt 11 embeddings = np.concatenate(t_file.fea_txt, axis=0) 12 num_embeddings = len(captions_list) AttributeError: 'bytes' object has no attribute 'raw_txt'
bshillingford commented 6 years ago

I misread your original email. You're trying to save and load the torch text-based file format? That's not supported, please resave it as binary.

Shubham-kale commented 6 years ago

Respected Sir, I am currently working on StackGANhttps://github.com/hanzhanggit/StackGAN and one of the primary scripts in the code demands loading a text-file with the help of function " t_file = torchfile.load(cap_path) " where ' cap_path ' is the path of the file I wanted to know what is the type of variable 't_file' in the statement of code , because the error states that it is a ' byte object ' and has no attribute 'raw_txt' I need to know is there any other object type which has such an attribute if yes,then how can I convert this byte object to that form.

The code snippet I am trying to run is :-

**# Load text embeddings generated from the encoder t=input("Enter Your Choice to train Model On :- \n 1.Birds \n 2.Flowers\n") if t==1 : cap_path = "/home/oslab/Data/birds/example_captions.t7" else :
cap_path ="/home/oslab/Data/flowers/example_captions.t7"

torch.save(cap_path,'ascii')# ///// 'output.t7',

t_file = torchfile.load(cap_path)

print("type is :-",type(t_file))

captions_list = t_file.raw_txt embeddings = np.concatenate(t_file.fea_txt, axis=0) num_embeddings = len(captions_list) print('Successfully load sentences from: ', cap_path) print('Total number of sentences:', num_embeddings) print('num_embeddings:', num_embeddings, embeddings.shape) ''' path to save generated samples''' save_dir = cap_path[:cap_path.find('.t7')] if num_embeddings > 0: batch_size = np.minimum(num_embeddings, cfg.TEST.BATCH_SIZE)**

AnwarUllahKhan commented 5 years ago

@Shubham-kale Dear have you solved this issue because I am also facing such kind of error? If yes then please give some suggestions.

thank you very much.

Shubham-kale commented 5 years ago

I Apologize for replying so late But there is issue with torch code used in the project I did the project using skip-thought encoding model

On Thu, 22 Nov, 2018, 12:11 PM AnwarUllahKhan <notifications@github.com wrote:

@Shubham-kale https://github.com/Shubham-kale Dear have you solved this issue because I am also facing such kind of error? If yes then please give some suggestions.

thank you very much.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bshillingford/python-torchfile/issues/14#issuecomment-440928521, or mute the thread https://github.com/notifications/unsubscribe-auth/AVzjHZeLgCYEbeLXIdyiJbsxqhq_cPJxks5uxkcqgaJpZM4SmTh3 .

cbadjatya commented 3 years ago

I Apologize for replying so late But there is issue with torch code used in the project I did the project using skip-thought encoding model On Thu, 22 Nov, 2018, 12:11 PM AnwarUllahKhan @.*** wrote: @Shubham-kale https://github.com/Shubham-kale Dear have you solved this issue because I am also facing such kind of error? If yes then please give some suggestions. thank you very much. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#14 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AVzjHZeLgCYEbeLXIdyiJbsxqhq_cPJxks5uxkcqgaJpZM4SmTh3 .

Hey Shubham, I'm trying to use the same repo for text to image generation on birds dataset. I'm trying to get images on new captions, but can't work around the .t7 file. Can you please hint on how to do it? Thanks

Shubham-kale commented 3 years ago

@Chinmay4400 , refer - https://github.com/Shubham-kale/Project I used the skip-thoughts pre-trained model, and its been a long time I don't really remember the details.