search

boheumd / A2Summ

The official implementation of 'Align and Attend: Multimodal Summarization with Dual Contrastive Losses' (CVPR 2023)
https://boheumd.github.io/A2Summ/
62 stars 10 forks source link

new dataset #14

Open anas2908 opened 6 months ago

anas2908 commented 6 months ago

running a new dataset for your model getting error "h5py objects cannot be pickled" All requirements are satisfied and are taken special care

boheumd commented 6 months ago

Hi, it seems that you need to set the num_workers=0 in this case. https://stackoverflow.com/questions/60684061/typeerror-h5py-objects-cannot-be-pickled

anas2908 commented 6 months ago

The Training time becomes significantly slow it will take weeks, is there any other way to just directly dump .npy features in your model?

boheumd commented 6 months ago

Hi, did you store the features of the custom dataset in the h5py format? If it is in this case, maybe you can change the h5py file into numpy format, and then you can directly read from the numpy file when loading the dataset features.