search

franciscovargas / TeviotDataScienceGame

This repository is for the edinburgh university data science game entry
MIT License
0 stars 0 forks source link

Baseline Unsupervised Pretraining #5

Open franciscovargas opened 8 years ago

franciscovargas commented 8 years ago

Look in to implementing one or the other . Autoencoders in keras seem straightforward.

evgkanias commented 8 years ago

I trained the autoencoders, but the result was not good.. about 72 validation.. now I try to build them layer by layer and uses different training approach! I hope the results will be better

evgkanias commented 8 years ago

UPDATE

Denoising autoencoders done... achieved 74.2 The corresponding script is 'rbm2.py'

Here is some key points of the log:

root: DEBUG: adding noise... root: DEBUG: generating data... root: DEBUG: Start pretraining... Epoch 30/30 21999/21999 [==============================] - 497s - loss: 0.6441 - acc: 0.0268
root: DEBUG: SAVING WEIGHTS in file: rbm_1_weights_top.h5... root: DEBUG: Predicting next input...
Epoch 30/30 21999/21999 [==============================] - 482s - loss: 0.0042 - acc: 0.1540
root: DEBUG: SAVING WEIGHTS in file: rbm_2_weights_top.h5... root: DEBUG: Predicting next input... Epoch 30/30 21999/21999 [==============================] - 646s - loss: 0.6138 - acc: 0.2226
root: DEBUG: SAVING WEIGHTS in file: rbm_3_weights_top.h5... root: DEBUG: Predicting next input... root: DEBUG: Done preprocessing. root: DEBUG: Start training... Epoch 28/30 8000/8000 [==============================] - 113s - loss: 0.7185 - acc: 0.7556
Epoch 29/30 8000/8000 [==============================] - 116s - loss: 0.7068 - acc: 0.7619
Epoch 30/30 8000/8000 [==============================] - 112s - loss: 0.6978 - acc: 0.7601
root: DEBUG: Done training.

franciscovargas commented 8 years ago

Use laplacian as prepro layer for the autoencoders williams phd student said raw images might be too much for them and recommended hog the nice thing about log is that it performs a smooth derivative and log kernels in layer one may allow the autoencoders to find a better hidden representation think of it as a little prod / push in the right direction On 26 Jun 2016 10:31 pm, "Evripidis Gkanias" notifications@github.com wrote:

I trained the autoencoders, but the result was not good.. about 72 validation.. now I try to build them layer by layer and uses different training approach! I hope the results will be better

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/franciscovargas/TeviotDataScienceGame/issues/5#issuecomment-228624221, or mute the thread https://github.com/notifications/unsubscribe/AFSJTJy3sK2PkaioaX06qd-1FCYm4zXhks5qPu-kgaJpZM4I6nSW .

franciscovargas commented 8 years ago

How was test accuracy??? On 27 Jun 2016 10:36 pm, "Evripidis Gkanias" notifications@github.com wrote:

UPDATE

Denoising autoencoders done... achieved 74.2 The corresponding script is 'rbm2.py' Here is some key points of the log:

root: DEBUG: adding noise... root: DEBUG: generating data... root: DEBUG: Start pretraining... Epoch 30/30 21999/21999 [==============================] - 497s - loss: 0.6441 - acc: 0.0268

root: DEBUG: SAVING WEIGHTS in file: rbm_1_weights_top.h5... root: DEBUG: Predicting next input...

Epoch 30/30 21999/21999 [==============================] - 482s - loss: 0.0042 - acc: 0.1540

root: DEBUG: SAVING WEIGHTS in file: rbm_2_weights_top.h5... root: DEBUG: Predicting next input... Epoch 30/30 21999/21999 [==============================] - 646s - loss: 0.6138 - acc: 0.2226

root: DEBUG: SAVING WEIGHTS in file: rbm_3_weights_top.h5... root: DEBUG: Predicting next input... root: DEBUG: Done preprocessing. root: DEBUG: Start training... Epoch 28/30 8000/8000 [==============================] - 113s - loss: 0.7185 - acc: 0.7556

Epoch 29/30 8000/8000 [==============================] - 116s - loss: 0.7068 - acc: 0.7619

Epoch 30/30 8000/8000 [==============================] - 112s - loss: 0.6978 - acc: 0.7601

root: DEBUG: Done training.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/franciscovargas/TeviotDataScienceGame/issues/5#issuecomment-228883152, or mute the thread https://github.com/notifications/unsubscribe/AFSJTBQao3lXiceKR1drbfwJWo0fQIEGks5qQEJQgaJpZM4I6nSW .

evgkanias commented 8 years ago

How was test accuracy???

74.2 is the accuracy of the submission (test), and the training was 76

franciscovargas commented 8 years ago

Good stuff can always try both training and pretraining for longer I also think log pretraining is worth a shot On 28 Jun 2016 2:15 am, "Evripidis Gkanias" notifications@github.com wrote:

How was test accuracy???

74.2 is the accuracy of the submission (test), and the training was 76

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/franciscovargas/TeviotDataScienceGame/issues/5#issuecomment-228922139, or mute the thread https://github.com/notifications/unsubscribe/AFSJTE1j68Pj4CEld856cFmmiLJrVW_uks5qQHWvgaJpZM4I6nSW .

evgkanias commented 8 years ago

I do pretrain now a new network, which will take a couple (or triple) of days to be trained. I hope this will make the difference.

franciscovargas commented 8 years ago

Liase with jonathan (check slack) he is running stuff on gpus and speeding things up he said he wanted to catch you in the lab.

On Tue, Jun 28, 2016 at 11:48 AM, Evripidis Gkanias < notifications@github.com> wrote:

I do pretrain now a new network, which will take a couple (or triple) of days to be trained. I hope this will make the difference.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/franciscovargas/TeviotDataScienceGame/issues/5#issuecomment-229015603, or mute the thread https://github.com/notifications/unsubscribe/AFSJTKjJkzZ84goP1kIuwbq0UlYJg_BIks5qQPwHgaJpZM4I6nSW .