Closed mullerhai closed 1 year ago
SequentialImpl can push_back AnyModule obj ,but when put anymodule obj meet error
/** Adds a new {@code Module} to the {@code Sequential} container, moving or copying it
* into a {@code shared_ptr} internally. This method allows passing value types,
* and letting the container deal with the boxing. This means you can write
* {@code Sequential(Module(3, 4))} instead of
* {@code Sequential(std::make_shared<Module>(3, 4))}. */
/** Adds a new named {@code Module} to the {@code Sequential} container, moving or copying
* it into a {@code shared_ptr} internally. This method allows passing value types,
* and letting the container deal with the boxing. */
/** Unwraps the contained module of a {@code ModuleHolder} and adds it to the
* {@code Sequential}. */
/** Unwraps the contained named module of a {@code ModuleHolder} and adds it to the
* {@code Sequential}. */
/** Iterates over the container and calls {@code push_back()} on each value. */
/** Adds a type-erased {@code AnyModule} to the {@code Sequential}. */
public native void push_back(@ByVal AnyModule any_module);
public native void push_back(@StdString BytePointer name, @ByVal AnyModule any_module);
public native void push_back(@StdString String name, @ByVal AnyModule any_module);
··· class SequentialAnyModuleNow() extends Module{
val anyNet = new AnyNet()
println(s"anyNet . ${anyNet.is_empty()}")
···
console log
anyNet . true
why anyModule object is empty is true ? I has fill with three linnear layer
AnyModule
is not meant to be subclassed.
It's not a solution for adding modules defined in Java to a Sequential
.
AnyModule
is the C++ trick that allows Sequential
to work: it is a container for a unknown module and you can call forward
with any type and number of arguments and it will dynamically call the forward function of the contained module.
But it cannot contain a module with a Java forward method.
If you have custom modules in Java/Scala you must call their forward method from Java/Scala.
What are you trying to do exacly ?
Why you don't use something like this, instead of trying to use Sequential ?
(in Java here, I don't know Scala)
abstract class AnyNet extends Module {
public abstract Tensor forward(Tensor x);
}
class MyNet extends AnyNet {
final LinearImpl fc1, fc2, fc3;
MyNet() {
fc1 = new LinearImpl(784, 64)
register_module("fc1", fc1)
fc2 = new LinearImpl(64, 32)
register_module("fc2", fc2)
fc3 = new LinearImpl(32, 10)
register_module("fc3", fc3)
}
@Override
public Tensor forward(Tensor x) {
x = relu(fc1.forward(x.reshape(x.size(0), 784)))
x = dropout(x, 0.5, true)
x = relu(fc2.forward(x))
x = log_softmax(fc3.forward(x), 1)
return x;
}
}
class AnyModuleNow extends Module {
final AnyNet net;
AnyModuleNow(AnyNet net) {
this.net = net;
register_module("net", net);
}
Tensor forward(Tensor x) {
x = x.reshape(x.size(0), 784);
return net.forward(x);
}
}
AnyModule
is not meant to be subclassed. It's not a solution for adding modules defined in Java to aSequential
.AnyModule
is the C++ trick that allowsSequential
to work: it is a container for a unknown module and you can callforward
with any type and number of arguments and it will dynamically call the forward function of the contained module. But it cannot contain a module with a Java forward method. If you have custom modules in Java/Scala you must call their forward method from Java/Scala. What are you trying to do exacly ? Why you don't use something like this, instead of trying to use Sequential ? (in Java here, I don't know Scala)abstract class AnyNet extends Module { public abstract Tensor forward(Tensor x); } class MyNet extends AnyNet { final LinearImpl fc1, fc2, fc3; MyNet() { fc1 = new LinearImpl(784, 64) register_module("fc1", fc1) fc2 = new LinearImpl(64, 32) register_module("fc2", fc2) fc3 = new LinearImpl(32, 10) register_module("fc3", fc3) } @Override public Tensor forward(Tensor x) { x = relu(fc1.forward(x.reshape(x.size(0), 784))) x = dropout(x, 0.5, true) x = relu(fc2.forward(x)) x = log_softmax(fc3.forward(x), 1) return x; } } class AnyModuleNow extends Module { final AnyNet net; AnyModuleNow(AnyNet net) { this.net = net; register_module("net", net); } Tensor forward(Tensor x) { x = x.reshape(x.size(0), 784); return net.forward(x); } }
I want to embed some layer blocks in Sequential , but Sequential can not receive the block who extend with nn.Module ,only can receive AnyModule instance, so I create AnyModule class obj ,then fill it into Sequential ,then meet error
so as you know how to embeded layer blocks in Sequential?
Please tell me why you want to use Sequential instead of the other solutions I suggested several times.
Sequential
Sequential is most popular layer and block container in pytorch python version, You has successfully implement SequentialImpl in new version ,so I must try the Sequential if could normal use or not in javacpp, you tell me another solution need to convert the layer organize style, not the best choice for me。I just want to same organize layer block them as Sequential like python pytorch
do you remember you tell me create the block ConvBnReluBlock, I need ten ConvBnReluBlocks in one container to train,use Sequential is the best container in python pytorch ,but Sequential not receive the block which extend nn.Module.
import org.bytedeco.javacpp._
import org.bytedeco.pytorch.global.torch
import org.bytedeco.pytorch.global.torch.ScalarType
import org.bytedeco.pytorch.{BatchNorm2dImpl, BatchNormOptions, Conv1dImpl, Conv1dOptions, Conv2dImpl, Conv2dOptions, Module, ReLUImpl, ReLUOptions, Tensor, _}
import scala.collection.mutable.{ArrayBuffer, ListBuffer}
import scala.language.postfixOps
/** *
*
* @param inChannels
* @param outChannels
* @param kernelSize
* @param stride
*/
class ConvBnReluBlock(val inChannels: Int, val outChannels: Int, val kernelSize: Int, val stride: Int) extends Module {
val convOpt = new Conv2dOptions(inChannels, outChannels, new LongPointer(kernelSize))
convOpt.stride.put(Array[Long](stride, stride): _*)
convOpt.padding.put(new LongPointer(kernelSize / 2))
convOpt.bias.put(false)
var conv = new Conv2dImpl(convOpt)
register_module("conv", conv)
val bnOpt = new BatchNormOptions(outChannels)
var bn = new BatchNorm2dImpl(bnOpt)
register_module("bn", bn)
val reluOpt = new ReLUOptions
reluOpt.inplace.put(true)
var relu = new ReLUImpl(reluOpt)
register_module("relu", relu)
def forward(x: Tensor): Tensor = relu.forward(bn.forward(conv.forward(x)))
}
You still can use Sequential for blocks like ConvBnRelu, since they only contain C++ modules.
I need ten ConvBnReluBlocks in one container to train
Do you mean 10 conv-bn-relu blocks or 10 blocks with different type of layers each time ?
If you need 10 conv-bn-relu block, then define your ConvBnRelu
class once and re-instantiate it 10 times. I find this more practical than instantiating 10 Sequential
and filling them 10 times.
I remind you my other suggestion: define your own Sequential
in Java:
package your.package;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import org.bytedeco.pytorch.Module;
public class Sequential<O> extends Module {
private final Module[] modules;
private final Method[] forwardMethods;
public Sequential(Module... modules) {
this.modules = modules;
forwardMethods = new Method[modules.length];
MODULE:
for (int i = 0; i < modules.length; i++) {
for (Method method : modules[i].getClass().getMethods()) {
if (method.getName().equals("forward")) {
forwardMethods[i] = method;
register_module(Integer.toString(i), modules[i]);
continue MODULE;
}
}
throw new IllegalArgumentException("No forward method found for module " + i);
}
}
public O forward(java.lang.Object... inputs) {
java.lang.Object output;
for (int i = 0; ; i++) {
try {
output = forwardMethods[i].invoke(modules[i], inputs);
} catch (IllegalAccessException | InvocationTargetException e) {
throw new RuntimeException(e);
}
if (i == modules.length - 1) break;
inputs = new java.lang.Object[]{output};
}
return (O) output;
}
}
that you can use like this
Conv2dOptions convOpt = new Conv2dOptions(8, 16, new LongPointer(3, 3));
convOpt.stride().put(new long[]{2, 2});
convOpt.padding().put(new LongPointer(1, 1));
convOpt.bias().put(false);
Conv2dImpl conv = new Conv2dImpl(convOpt);
BatchNormOptions bnOpt = new BatchNormOptions(16);
BatchNorm2dImpl bn = new BatchNorm2dImpl(bnOpt);
ReLUOptions reluOpt = new ReLUOptions();
reluOpt.inplace().put(true);
ReLUImpl relu = new ReLUImpl(reluOpt);
Sequential<Tensor> seq = new Sequential<>(conv, bn, relu);
Tensor out = seq.forward(in);
For using a module in the C++ sequential, we need this module to be defined at C++ compile time. So the only solution to add Java modules to C++ sequential would be to define in the presets a subclass of Module
with a virtual forward
function. But we would need one such class for every possible signature of forward
. And the number of possible forward signature is infinite. We could only add those for some classical signatures, like taking 1 tensor and returning 1 tensor, and another taking 2 tensors and returning 1 tensor, and a couple of others... but you'll eventually find a not covered case. So I'm not sure it's a good idea, since there are alternatives like discussed above.
You still can use Sequential for blocks like ConvBnRelu, since they only contain C++ modules.
I need ten ConvBnReluBlocks in one container to train
Do you mean 10 conv-bn-relu blocks or 10 blocks with different type of layers each time ? If you need 10 conv-bn-relu block, then define your
ConvBnRelu
class once and re-instantiate it 10 times. I find this more practical than instantiating 10Sequential
and filling them 10 times.I remind you my other suggestion: define your own
Sequential
in Java:package your.package; import java.lang.reflect.InvocationTargetException; import java.lang.reflect.Method; import org.bytedeco.pytorch.Module; public class Sequential<O> extends Module { private final Module[] modules; private final Method[] forwardMethods; public Sequential(Module... modules) { this.modules = modules; forwardMethods = new Method[modules.length]; MODULE: for (int i = 0; i < modules.length; i++) { for (Method method : modules[i].getClass().getMethods()) { if (method.getName().equals("forward")) { forwardMethods[i] = method; register_module(Integer.toString(i), modules[i]); continue MODULE; } } throw new IllegalArgumentException("No forward method found for module " + i); } } public O forward(java.lang.Object... inputs) { java.lang.Object output; for (int i = 0; ; i++) { try { output = forwardMethods[i].invoke(modules[i], inputs); } catch (IllegalAccessException | InvocationTargetException e) { throw new RuntimeException(e); } if (i == modules.length - 1) break; inputs = new java.lang.Object[]{output}; } return (O) output; } }
that you can use like this
Conv2dOptions convOpt = new Conv2dOptions(8, 16, new LongPointer(3, 3)); convOpt.stride().put(new long[]{2, 2}); convOpt.padding().put(new LongPointer(1, 1)); convOpt.bias().put(false); Conv2dImpl conv = new Conv2dImpl(convOpt); BatchNormOptions bnOpt = new BatchNormOptions(16); BatchNorm2dImpl bn = new BatchNorm2dImpl(bnOpt); ReLUOptions reluOpt = new ReLUOptions(); reluOpt.inplace().put(true); ReLUImpl relu = new ReLUImpl(reluOpt); Sequential<Tensor> seq = new Sequential<>(conv, bn, relu); Tensor out = seq.forward(in);
For using a module in the C++ sequential, we need this module to be defined at C++ compile time. So the only solution to add Java modules to C++ sequential would be to define in the presets a subclass of
Module
with a virtualforward
function. But we would need one such class for every possible signature offorward
. And the number of possible forward signature is infinite. We could only add those for some classical signatures, like taking 1 tensor and returning 1 tensor, and another taking 2 tensors and returning 1 tensor, and a couple of others... but you'll eventually find a not covered case. So I'm not sure it's a good idea, since there are alternatives like discussed above.
I think ,this solution maybe good in javacpp,the SequentialImpl in javacpp is not the same function as Python,this way only todo like your coding style,thanks
HI, now I try to use AnyModule in new pytorch-javacpp version,but I found it can not smooth use, maybe it need some ptr param to create it , @HGuillemet @saudet
console error
similar issue
https://github.com/pytorch/pytorch/pull/34208 https://github.com/pytorch/pytorch/pull/17552 https://discuss.pytorch.org/t/modify-nn-anymodule-embedded-in-nn-sequential/100002