Post/db sequence training poutyne

dot-layer / blog

https://dotlayer.org/

Other

14 stars 6 forks source link

Post/db sequence training poutyne #133

Closed davebulaval closed 3 years ago

davebulaval commented 3 years ago

A post about the training of RNN (in Python) using Poutyne.

The used case uses addresses dataset that hare now hosted on dot-layer/external-assets-blog. The post tries to introduce different ML concept such as

RNN and LSTM (with external links to explain more)
- Effect on performance of bi-LSTM and stacked layers.
- Padding and packing trick (with external links to explain more)
- Seed effect (with an external link to explain more)
- Exploration of zero shoot evaluation

freud14 commented 3 years ago

Also, maybe we should link a jupyter notebook with the post?

davebulaval commented 3 years ago

Also, maybe we should link a jupyter notebook with the post?

I've added a Jupyter notebook with the article and I've loaded it into Google Colab with a link to it in the article (since one need a GPU).

I will, later on, update the link from the one in the master branch.

davebulaval commented 3 years ago

@chblw done a second pass of Grammarly.

samperochkin commented 3 years ago

Sorry pour l'attente @davebulaval! Je passe là-dessus d'ici mercredi prochain.

davebulaval commented 3 years ago

Il ne faudra pas oublier de synchroniser les corrections dans le notebook.

For sure @freud14, the last commit will be the date update and the notebook sync.

davebulaval commented 3 years ago

@TortillasAlfred allright, so I've updated the google colab and now everything should be ok.

davebulaval commented 3 years ago

Alright, so, for a reason I don't understand, Magnitude doesn't work well local. I've tested it on three different computers and I've always got a sqlite3.DatabaseError: database disk image is malformed. Since I manage to use the embedding in a different case, my hypothesis is that the multi threads use with the dataloader is not working as expected (since I manage to make it work in a single thread use).

So, I've decided to propose an in-memory fasttext solution in the article and in a notebook and a Magnitude version in a Google Colab (which for a strange reason work don't ask me why).

Should be the last modification of the code.

samperochkin commented 3 years ago

Est-ce qu'on est rendu à l'étape re-lecture finale et choix d'une date pour poster?

davebulaval commented 3 years ago

Est-ce qu'on est rendu à l'étape re-lecture finale et choix d'une date pour poster?

Yes it's!