Open csolorio opened 6 years ago
Hi @csolorio
Is the one active branch always the same? Or it change from MB to MB?
The branch would be randomly selected between minibatches. Is that possible?
Yes, it is possible, but a little complicated. The idea is to have 2 version of each branch one with frozen parameters and one without. Then you will have 3 networks, each network has 2 branches frozen and on not-frozen. Assuming the 3 networks are z1, z2 and z3.
Now you need to create 3 trainers, one for each network and switch between trainers at random after each MB.
Another idea without using cloning or creating multiple trainer. Is to add an 3 extra float inputs to your network, those extra inputs will always be set to zeros except for one will be set to 1.0, and use those as coefficient to the 3 networks in your loss function.
Ok, great ideas! I'll try them. Can one learner be used in the 3 trainers or would separate learner instances be required?
Hi,
I'm trying to train a model that have several branches but just having one active branch during each minibatch training. How can I achieve this? Let me explain a little bit the model: ..............|-- branch1_1 - |.........................................|-- branch1_2 - output1 Input -|-- branch2_1 - | -- common_path -- |-- branch2_2 -output2 ..............|-- branch3_1 - |..........................................|-- branch3_2 - output3
Depending on the input data, I would only have to use one branch path while keeping the rest fixed and just having the corresponding output active. Could I achieve this by using the clone and freeze function on the branches that are inactive during an update? Some tutorial that could help me?