search

NervanaSystems / neon

Intel® Nervana™ reference deep learning framework committed to best performance on all hardware
http://neon.nervanasys.com/docs/latest
Apache License 2.0
3.87k stars 811 forks source link

Skip thought vectors #436

Open scigeek72 opened 6 years ago

scigeek72 commented 6 years ago

I would like to use the skip thought vectors implementation in neon. The documentation provided is not enough. Is there a tutorial available on how to obtain skip thought vectors from scratch on my own documentation ? I have created a virtual environment and downloaded neon. I am not sure how to train the skip thought model. Thanks.

baojun-nervana commented 6 years ago

@scigeek72 Which backend do you use? Below is an example I used to run skip-thought model. It used a small subset of data to demo it can train. (--subset_pct 0.1 means 0.1% of the dataset)

python examples/skip-thought/train.py -e 2 -b mkl --serialize 1 --no_progress_bar -v -s examples/skip-thought/train.prm -l tmpfile.log -w /dataset --subset_pct 0.1

scigeek72 commented 6 years ago

Thanks @baojun-nervana for our quick response. I am using MacOSX and no GPUs. But I can fire up a compute node on AWS to use GPU.

I don't have any experience with neon. Do I need to use any other ? I use keras with tensorflow as a backend for deep learning related experiments. Does skip thought vector module ship with neon or do we have to separately download them from github? Also, what's the average training time for the skip thought model?

Thanks a lot.

baojun-nervana commented 6 years ago

@scigeek72 The example can run on either cpu or gpu.

With gpu backend, it takes about 3 hrs per epoch. For cpu mkl it will takes about 30hrs per epoch.