josephjaspers / blackcat_tensors

Matrix-Vector Library Designed for Neural Network Construction. cuda (gpu) support, openmp (multithreaded cpu) support, partial support of BLAS, expression template based implementation PTX code generation identical to hand written kernels, and support for auto-differentiation
12 stars 4 forks source link

How to put my data into the model? #35

Closed xinsuinizhuan closed 4 years ago

xinsuinizhuan commented 4 years ago

I read my txt'file data to the vector, then when i put my vectror data to the model, compile, it error:

1569749104(1) how should i do?

I add the under function to Tensor_Utility.h, and it compile normal, is it ok? void data_format_convers(std::vector & maxmindata) { std::vector file_data;

    for (int i = 0; i < maxmindata.size(); i++)
    {
        value_type val = (value_type)maxmindata[i];
        file_data.push_back(val);
    }
    int copy_size = (unsigned)as_derived().size() > file_data.size() ? file_data.size() : as_derived().size();
    utility_l::HostToDevice(as_derived().internal().memptr(), file_data.data(), copy_size);
}
josephjaspers commented 4 years ago

Seems good for now. The only issue I can think of is that it won't work for non-continuous tensors. Feel free to make a pull-request on any features you want to add.

josephjaspers commented 4 years ago

Looks good to me! It seems that it won't work for non-continuous tensors (thought I will an nd-iterator to handle this later).

josephjaspers commented 4 years ago

It seems you are trying to assign a slice of a matrix to a std::vector. You can assign element-by-element though.

    BC::Cube<float> data_cube(4,2,1);
    std::vector<std::vector<float>> data = { {1,2,3,4}, {1,2,3,4}};

    for (std::size_t i = 0; i < data.size(); ++i) {
        for (std::size_t j = 0; j < data[i].size(); ++j) {
            data_cube[0][i][j] = data[i][j];
        }
    }
xinsuinizhuan commented 4 years ago

In addition, when i get the forward_propagation data, how could i put them into my vector?

mat hyps = network->m_pnetwork.forward_propagation(inputs[0]);

I want get the data from hyps to my vector and array? How should i do? Is it ok? std::vector& _outputvec; mat hyps = network->m_pnetwork.forward_propagation(inputs[0]); for (int j=0; j< hyps.tensor_dimension;j++) { _outputvec.push_back((BC::Scalar(hyps).data())[j]); }

josephjaspers commented 4 years ago

You can use BC's algorithms to copy data (include/algorithms/Algorithms.h)

outputvec = std::vector(hyps.size()); 
BC::copy(hyps.get_stream(), outputvec.begin(), outputvec.end(), hyps.cw_begin()); 

Its nearly identical to std::copy if you are only using host_tensors though it is data-safe if you ever want to switch to GPU mode (cuda).

The differences between cw.begin() and begin() is that cw_begin returns the coefficientwise iterator (element by element) while begin returns the 'multi-dimensional' iterator.

Here's the (rather brief) documentation on this https://github.com/josephjaspers/blackcat_tensors/blob/master/docs/algorithms.md

xinsuinizhuan commented 4 years ago

Yes, when i use : std::vector predict_vec(hyps.size()); BC::copy(hyps.get_stream(), predict_vec.begin(), predict_vec.end(), hyps.cw_begin()); that is ok! Thank you!