calamari-train: warmstart not working without also giving network spec

[bertsky]

Maybe just a documentation issue:

When using --warmstart with a (ckpt.json) model that uses non-default network architecture, I end up with incompatible shapes:

INFO     2022-04-27 18:46:19,061 tfaip.trainer.warmstart.warmst: Source model successfully loaded for warmstart.
ERROR    2022-04-27 18:46:19,074 tfaip.trainer.warmstart.warmst: Cannot assign to variable root/root/CalamariGraph/lstm_0/bidirectional/forward_lstm/lstm_cell_1/kernel:0 due to variable shape (720, 800) and valu
e shape (3, 3, 60, 120) are incompatible
Traceback (most recent call last):
  File "/home/h1/rosa992c/my-kernel/powerai-kernel2/lib/python3.7/site-packages/tfaip/trainer/warmstart/warmstarter.py", line 144, in warmstart
    target_model, [np.asarray(w) for w in src_model.weights if w.name not in names_to_ignore]
  File "/home/h1/rosa992c/my-kernel/powerai-kernel2/lib/python3.7/site-packages/calamari_ocr/ocr/training/warmstart.py", line 33, in apply_weights
    self.copy_weights_from_model(target_model, new_weights, *self.codec_changes)
  File "/home/h1/rosa992c/my-kernel/powerai-kernel2/lib/python3.7/site-packages/calamari_ocr/ocr/training/warmstart.py", line 38, in copy_weights_from_model
    target_weight.assign(source_weight)
  File "/sw/installed/TensorFlow/2.4.1-fosscuda-2019b-Python-3.7.4/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 891, in assign
    (tensor_name, self._shape, value_tensor.shape))
ValueError: Cannot assign to variable root/root/CalamariGraph/lstm_0/bidirectional/forward_lstm/lstm_cell_1/kernel:0 due to variable shape (720, 800) and value shape (3, 3, 60, 120) are incompatible

Also passing the matching --network deep3 made the problem disappear.

This is somewhat unexpected, since I would assume that the architecture can be derived from the warmstart.