How to create mulity neuralnetworks at the same time?

[xinsuinizhuan]

I need to new multy neuralnetworks at the same time，and when i destroy my app, i also need to delete them? How should i do?

[josephjaspers]

They will automatically be destroyed at the end of the scope. You do not need to do anything to destroy them.

I am not sure what you mean by "needing to make multiple neural networks." If you simply need to copy your network you can do so like this:

    auto network = neuralnetwork(
        feedforward(system_tag, 784, 256),
        tanh(system_tag, 256),
        feedforward(system_tag, 256, 10),
        softmax(system_tag, 10),
        logging_output_layer(system_tag, 10, BC::nn::RMSE).skip_every(100)
    );
         //if you only need a few copies 
    auto copy1 = network;
    auto copy2 = network;
    auto copy3 = network;

        //if you need many copies
    std::vector<decltype(network)> list_of_networks;
    for (int i = 0; i  < 10; ++i) {
        list_of_networks.push_back(network);
    }

Let me know if that answers your question.

[xinsuinizhuan]

I will create many instances(one company forecast data one instance), and each instance corresponds to its own network, so each instance creates its own networks. In addition, The data is growing, if the network cannot continue training, I need to destroy the original network and recreate the new network for training for every training cycle.

[xinsuinizhuan]

My original work is that, network is a class instances, so i create the network class instance for every instance, and i destroy the original class instance network and recreate the new network for training for every training cycle.

[xinsuinizhuan]

whether i can new the network for my every preditc instance, then delete it when i retrain or destroy the instance?

[josephjaspers]

Do you just need to re-randomize the weights?

I am unsure what you mean by deleting the neural network. When it goes out of scope it will be deleted. (The destructor will destroy the memory automatically for you)

[xinsuinizhuan]

Do you just need to re-randomize the weights?

I am unsure what you mean by deleting the neural network. When it goes out of scope it will be deleted. (The destructor will destroy the memory automatically for you)

No, i need not to re-randomize the weights，i only need to continue to train, becouse my data increasing day by day, so i need use the new data set that increased to retrain the modle. But the neural network don’t support to continue to train, so i must destroy the neural network, then recreate it to retrain my data, because i should hold on the neural network for forecast request to forecast before retrain. My idea is: 1、new my instance, new the neural network at the same time (there are many instances, so the more instance the more neural network) 2、I train the current data set by current instance's neural network, i also save the train model(restart my app, i only load the model and do not retrain before need to retrain) 3、hold on the current instance's neural network for forecast（the request can be predicted at any time, so i need hold on the neural network） 4、when arrived at the cycle that data increaing, new data will add to the data set, so i must retrain my model. But the neural network not support the continue train, so i must delete the current instance's neural network, then create the same neural network as old ,then retrain the model, and save, and hold on.

[xinsuinizhuan]

Adout many networks operate, how should i do? I have two idea: 1、 add the BC::nn::neuralnetwork to my struct, then i will create many my struct instances for predict manage, my struct as: typedef struct _LstmPredictTask { long fd; //struct fd char m_devId[256]; //obj id char m_datafile[MAX_PATH]; //train file path char m_networkfilepath[MAX_PATH]; //Network configuration parameters network m_pnetwork; //or auto network; //Network point or Variable } LstmPredictTask; then i will, malloc it, init the params, and add the struct pointer to my dlist, as LstmPredictTask lstmpredicttask = (LstmPredictTask*)malloc(sizeof(LstmPredictTask)); if (lstmpredicttask == NULL) { err_quit_ex("comclient != NULL"); return -2; } memset(lstmpredicttask, 0, sizeof(OpennnPredictTask));

    lstmpredicttask->fd = (int32)lstmpredicttask;

//初始化网络
lstmpredicttask->m_pnetwork = BC::nn::neuralnetwork(
    BC::nn::lstm(system_tag, 784/4, 64),
    BC::nn::recurrent(system_tag, 64, 10),
    BC::nn::softmax(system_tag, 10),
    BC::nn::logging_output_layer(system_tag, 10, BC::nn::MAPE).skip_every(100)
);

adt_dlist_lock(s_lst_lstmNet);
adt_dlist_add(s_lst_lstmNet, NULL, &lstmpredicttask);
adt_dlist_unlock(s_lst_lstmNet);

I search network by fd, then use it for predict. I attempt to operate,but don't know how to do:

2、supprot add the some variable to the BC::nn::neuralnetwork，the i make neuralnetwork， and assignment the variable, load the m_networkfilepath to init the neuralnetwork： auto network = neuralnetwork( feedforward(system_tag, 784, 256), tanh(system_tag, 256), feedforward(system_tag, 256, 10), softmax(system_tag, 10), logging_output_layer(system_tag, 10, BC::nn::RMSE).skip_every(100) ); //if you only need a few copies auto copy1 = network; copy1.fd = m_devId; copy1.m_datafile = m_datafile; auto copy2 = network; copy2.fd = m_devId; copy2.m_datafile = m_datafile; auto copy3 = network; copy3.fd = m_devId; copy3.m_datafile = m_datafile;

    //if you need many copies
std::vector<decltype(network)> list_of_networks;
for (int i = 0; i  < 10; ++i) {
    list_of_networks.push_back(network);
}

I search network by fd, then load data to train and for predict.

[xinsuinizhuan]

Adout network's continue to train operate, how should i do? 3 idea: 1、Add some function, when i done train, and new data add to inputs, i only need add the new data to the networks inputs, then retrain. 2、add resetfualt function, when new data arriver and need retrain, i only call the resetfualt function reset the networks's all params to initial value, add the all data set(that add the new data) to retrain. 3、network support destroy, i destroy it, then new it, and add the all data set(that add the new data) to retrain.

[josephjaspers]

auto network = neuralnetwork(
feedforward(system_tag, 784, 256),
tanh(system_tag, 256),
feedforward(system_tag, 256, 10),
softmax(system_tag, 10),
logging_output_layer(system_tag, 10, BC::nn::RMSE).skip_every(100)
);

neuralnetwork is a factory method. It returns a NeuralNetwork<Layers...> object. while each of the 'layer' methods are factory methods for their respective layers.

feedforward(system_tag, inputs, outputs) is the same as FeedForward<SystemTag, double>(inputs, outputs).

What you could do is... You can just 'move' the neuralnetwork object to reset it.

auto make_lstm_network() {
 return neuralnetwork(
        lstm(BC::host_tag(), inputs, outputs),
        lstm(BC::host_tag(), inputs, outputs),
        output_layer(BC::host_tag(), outputs)); 
}
using network_type = decltype(make_lstm_network()); 

struct _LstmPredictTask { 

         network_type my_network = make_lstm_network(); 

         void reset_neural_network() {
                my_network = std::move(make_lstm_network()); 
         }
};

let me know if this helps.

[xinsuinizhuan]

auto make_lstm_network() { return neuralnetwork( lstm(BC::host_tag(), inputs, outputs), lstm(BC::host_tag(), inputs, outputs), output_layer(BC::host_tag(), outputs)); } using network_type = decltype(make_lstm_network());

struct _LstmPredictTask {

     network_type my_network = make_lstm_network(); 

     void reset_neural_network() {
            my_network = std::move(make_lstm_network()); 
     }

};

It seems to help me, but i am puzzle.whether i can pass some parameter to make_lstm_network method,as: auto make_lstm_network(char file) { //Parse the file,and init the neuralnetwork input and output param return neuralnetwork( lstm(BC::host_tag(), inputs, outputs), lstm(BC::host_tag(), inputs, outputs), output_layer(BC::host_tag(), outputs)); } using network_type = decltype(make_lstm_network(char file));

struct _LstmPredictTask {

     network_type my_network = make_lstm_network(char* file); 

     void reset_neural_network() {
            my_network = std::move(make_lstm_network(char* file)); 
     }

};

In additon to, when i "move", compile it ,"move " is not identifier:

[xinsuinizhuan]

help me, how should i do? The below code, comiple is error, the two kinds of the assignment operator is error:

[josephjaspers]

I will check today.

[xinsuinizhuan]

I will check today. How about it? How to move it in struct, and how to pass some parameter to make_lstm_network method?

[josephjaspers]

Hi, I added fixes to it, should work now.

[xinsuinizhuan]

move it could work? In addition, how to pass the parameter?

[josephjaspers]

I am not sure what you mean by "how to pass the parameter?".

[xinsuinizhuan]

I am not sure what you mean by "how to pass the parameter?".

when i use the make_lstm_network method, i only want pass some parameters to make_lstm_network methodthe, as the input and output value as : auto make_lstm_network(int inputsize, int outputsize) { return neuralnetwork( BC::nn::lstm(BC::host_tag(), inputsize, outputsize), BC::nn::recurrent(BC::host_tag(), outputsize, outputsize), BC::nn::softmax(BC::host_tag(), outputsize), BC::nn::logging_output_layer(BC::host_tag(), outputsize, BC::nn::RMSE).skip_every(100)); }

then how to use them: using network_type = decltype(make_lstm_network());

struct _LstmPredictTask { int input; int output; network_type my_network = make_lstm_network(); void reset_neural_network() { my_network = std::move(make_lstm_network()); } }; using network_type = decltype(make_lstm_network());

[josephjaspers]

I am not sure what you mean by "how to pass the parameter?".

when i use the make_lstm_network method, i only want pass some parameters to make_lstm_network methodthe, as the input and output value as : auto make_lstm_network(int inputsize, int outputsize) { return neuralnetwork( BC::nn::lstm(BC::host_tag(), inputsize, outputsize), BC::nn::recurrent(BC::host_tag(), outputsize, outputsize), BC::nn::softmax(BC::host_tag(), outputsize), BC::nn::logging_output_layer(BC::host_tag(), outputsize, BC::nn::RMSE).skip_every(100)); }

then how to use them: using network_type = decltype(make_lstm_network());

struct _LstmPredictTask { int input; int output; network_type my_network = make_lstm_network(); void reset_neural_network() { my_network = std::move(make_lstm_network()); } }; using network_type = decltype(make_lstm_network());

Sorry for the late reply. If you want make_lstm_network to accept some arguments, you simply have to supply the arguments to decltype() as well.

 auto make_lstm_network(int inputsize, int outputsize) {
 return neuralnetwork(
     BC::nn::lstm(BC::host_tag(), inputsize, outputsize),
     BC::nn::softmax(BC::host_tag(), outputsize)
 }

//The arguments do not need to be the correct value, just the same type. 
using network_type = decltype(make_lstm_network(0,0));