Open varisd opened 6 years ago
Note, that currently this branch removes the default l1, l2 logging into summaries
Oh, I didn't even notice. I'd like to keep the L2 plot in TensorBoard.
The L2 plot is back.
Právě kvůli ty sémantice bych to nedělal.. regularizers jsou regularizers a new regularizers and then some..
Dne pá 3. 8. 2018 21:48 uživatel Dušan Variš notifications@github.com napsal:
@varisd commented on this pull request.
In neuralmonkey/trainers/generic_trainer.py https://github.com/ufal/neuralmonkey/pull/742#discussion_r207649319:
unweighted losses for fetching
- self.losses = [o.loss for o in objectives] + [l1_value, l2_value]
- tf.summary.scalar("train_l1", l1_value,
- collections=["summary_train"])
- tf.summary.scalar("train_l2", l2_value,
- collections=["summary_train"])
- self.losses = [o.loss for o in objectives] + reg_values
we always want to include l2 values in the summary
- if L2Regularizer not in [type(r) for r in self.regularizers]:
- reg_values.append(L2Regularizer().value(regularizable))
Ten L2Regularizer by se mel pridat i do self.regularizers.
reg_values se potom, co se pridaji do summaries, dale uz nikde nepouzivaji, takze neni problem tam pridat tenhle "default" regularizer
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ufal/neuralmonkey/pull/742#discussion_r207649319, or mute the thread https://github.com/notifications/unsubscribe-auth/ABwcs4sS68v6cTEfJRcyTu-TgJa6xpDFks5uNKkLgaJpZM4VkjNe .
S/new/ne/
Dne pá 3. 8. 2018 21:54 uživatel Jindra Helcl jindra.helcl@gmail.com napsal:
Právě kvůli ty sémantice bych to nedělal.. regularizers jsou regularizers a new regularizers and then some..
Dne pá 3. 8. 2018 21:48 uživatel Dušan Variš notifications@github.com napsal:
@varisd commented on this pull request.
In neuralmonkey/trainers/generic_trainer.py https://github.com/ufal/neuralmonkey/pull/742#discussion_r207649319:
unweighted losses for fetching
- self.losses = [o.loss for o in objectives] + [l1_value, l2_value]
- tf.summary.scalar("train_l1", l1_value,
- collections=["summary_train"])
- tf.summary.scalar("train_l2", l2_value,
- collections=["summary_train"])
- self.losses = [o.loss for o in objectives] + reg_values
we always want to include l2 values in the summary
- if L2Regularizer not in [type(r) for r in self.regularizers]:
- reg_values.append(L2Regularizer().value(regularizable))
Ten L2Regularizer by se mel pridat i do self.regularizers.
reg_values se potom, co se pridaji do summaries, dale uz nikde nepouzivaji, takze neni problem tam pridat tenhle "default" regularizer
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ufal/neuralmonkey/pull/742#discussion_r207649319, or mute the thread https://github.com/notifications/unsubscribe-auth/ABwcs4sS68v6cTEfJRcyTu-TgJa6xpDFks5uNKkLgaJpZM4VkjNe .
Jo, testy jsem jeste neresil.
Ten gradient runner pouzivam, abych si vyprintil gradienty z celeho datasetu (vetsinou validacniho, protoze neni tak velky). Z toho si pak bokem udelam prumer gradientu pro kazdou vahu modelu; ty pak muzu pouzit v EWC.
Tak kdyby se ti podařilo tuhle pipelinu nasimulovat v tests/tests_run.sh
tak by to bylo úplně nejlepší.
ping, uz jsem tuhle vetev rebasoval minimalne trikrat... pokud ji nemate v planu mergovat, dejte mi vedet
Mno já jsem myslel, že se to zamerguje, až bude hotovej refaktoříček s tf datasetem (stejně jako hromada commitů, který si syslim ve štelářku a nedělám z toho PR).
NOTE: currently is not working with DelayedUpdateTrainer due to a problem with trainer variable restoration.