Training with large system size is hard

[DesmondYuan]

[Essential] system size - data size relationship analysis

• why

  • question is indeed too hard
  • too many/few time points for each trajectory
  • only config level limit, better fine-tuning can help

• solution

  • increase sparsity (less params)
  • add more information (longer ntotal)
  • try different network structure

[DesmondYuan]

Adding tests for Beeline networks 47e8652a3d38d7189fca12b763ab6818f310c586 and a2b0c0b6e0c5b7530cd50c5494afafd5e772b1f2

Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj, A. & Murali, T. M. Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nat. Methods 17, 147–154 (2020). https://www.nature.com/articles/s41592-019-0690-6

[jiweiqi]

A good demo for L1

network: "beeline_networks/Synthetic_LI.csv"
ns: 7

tfinal: 20.0
ntotal: 20
batch_size: 16
epoch_size: -1

lr: 1.e-3
lr_new: -1  #use -1 otehrwise
weight_decay: 1.e-5

n_mu: 3

n_exp_train: 20
n_exp_val: 5
n_exp_test: 5
noise: 0.01

n_iter_max: 100000
n_plot: 20 # frequency of callback

n_iter_buffer: 5000
n_iter_burnin: 100
n_iter_tol: 10000
convergence_tol: 1e-8

drop_range:
   lb: -0.1
   ub: 0.1

[jiweiqi]

The program takes about 7 minutes. about 2.1 it/s.

One testing condition

[jiweiqi]

A simple criterion to judge if the data is sufficient is whether there is a big gap between training loss and validation loss. If there is, we shall increase the number of conditions.

[judyueshen]

Another good training example for LI network, with 10 training conditions and 10 ntotal

is_restart: false
network: "beeline_networks/Synthetic_LI.csv"
ns: 7

tfinal: 10.0
ntotal: 10
batch_size: 8
epoch_size: -1

lr: 1.e-3
weight_decay: 1.e-6

n_mu: 3

n_exp_train: 10
n_exp_val: 5
n_exp_test: 5
noise: 0.01

n_iter_max: 10000
n_plot: 20 # frequency of callback

n_iter_buffer: 50
n_iter_burnin: 100
n_iter_tol: 500
convergence_tol: 1e-8

drop_range:
   lb: -0.1
   ub: 0.1

[DesmondYuan]

Curated model