pender / chatbot-rnn

A toy chatbot powered by deep learning and trained on data from Reddit
MIT License
900 stars 371 forks source link

IndexError: too many indices for array #26

Closed kpratik2015 closed 6 years ago

kpratik2015 commented 7 years ago

Traceback (most recent call last): File "train.py", line 172, in main() File "train.py", line 41, in main train(args) File "train.py", line 93, in train data_loader.cue_batch_pointer_to_epoch_fraction(global_epoch_fraction) File "D:\chatbot-rnn-master\utils.py", line 233, in cue_batch_pointer_to_epoch_fraction self._cue_batch_pointer_to_step_count(step_target) File "D:\chatbot-rnn-master\utils.py", line 242, in _cue_batch_pointer_to_step_count self._load_preprocessed(i) File "D:\chatbot-rnn-master\utils.py", line 172, in _load_preprocessed self.tensor = self.tensor[:self.num_batches * self.batch_size] IndexError: too many indices for array

AaronTan120 commented 7 years ago

I am having the same problem as well. May I know which version of Python you are using?

kpratik2015 commented 7 years ago

Python 35.5.2

pender commented 7 years ago

Hi everyone, the repo was developed with Tensorflow 0.12 and Python 2.7. Unfortunately Tensorflow 1.0 broke many of the APIs that the model relied on. I have a build on my machine that updates the code to TF >1.0 and Python 3.5, but then the pretrained model no longer loads since the variable names have changed. At some point I will get around to either fixing the variable names in the pretrained model or retraining a new model... but for now it is broken, and retraining a model takes months of GPU time (at least with the best hyperparameters that I was able to find).

I'm sorry!

nichcuta commented 7 years ago

@pender I am willing to give it a try and train against my GPU, what are the best hyper parameters that you found if i may ask (IE: Learning rate, decay rate, num layers, etc..)?

Thanks!

VineethRaghavan commented 7 years ago

@nichcuta Hey did you try training the model? If it worked, could you maybe upload it?

jeshocarmel commented 6 years ago

@pender Do you mind sharing the updated code which you have with Tensorflow > 1.0 and Python 3.5. I would like to train a model of movie subtitles from opensubtitles in my GPU. Also do share the hyperparameters which would make our job easier. Thanks

birnbaum commented 6 years ago

As @pender stated there are a few things you have to change to get the project running on tensorflow 1.x and python 3. This specific bug can be fixed by changing: https://github.com/pender/chatbot-rnn/blob/a60a56eeed8521145ce9395e5c4a9cca9516bb6a/utils.py#L148 to self.tensor = np.array(list(map(self.vocab.get, data)))

Otherwise you feed np.array() a generator (in python 3, map() returns a generator). I may issue a PR with all the changes you have to do to the project soon.

ButterscotchV commented 6 years ago

I've worked off of @zenixls2's update to TensorFlow 1.2 to update the project to Python 3.5. I have put the code here, https://github.com/Dankrushen/chatbot-rnn, and you should be able to do everything except parse the Reddit comments (I'm open if anyone can fix that, but you should just be able to run it on Python 2.7).

pender commented 6 years ago

Hi, I pushed a significant update to TF 1.4 and Python3, so I am closing out old issues -- feel free to reopen if this is still a problem, and I will take a look. (reddit-parse.py has also been updated to Python 3 now.)