search

cyclops-community / ctf

Cyclops Tensor Framework: parallel arithmetic on multidimensional arrays
Other
199 stars 53 forks source link

Questions about using CTF::Tensor::read_all() #34

Closed Jakes-Schauer closed 6 years ago

Jakes-Schauer commented 7 years ago

1) If I write t.read_all(destination, true), for some CTF::Tensor t, are the contents stored into destination ordered according to the global index formula, that is, by a column-major enumeration? 2) Can I pass nullptr as the destination argument on all but one process, to limit copying?

solomonik commented 7 years ago
  1. yes
  2. no, not currently
solomonik commented 7 years ago

But for (2) you can gather all elements on a single processors, by defining (lazily allocated) int64_t inds[tot_size]; double data[tot_size]; for (int64_t i=0; i<tot_size; i++){ inds[i] =i; }; tsr.read(tot_size,inds,data); and run tsr.read(0,NULL,NULL) with processors that don't need data.

Jakes-Schauer commented 7 years ago

Thank you; that does indeed seem to do what I need. Would you say that this is the best method to use when the goal is to write a tensor into a single file on disk, the tensor being sufficiently small?

solomonik commented 6 years ago

@Jakes-Schauer sorry looks like I forgot to respond to your question, yes its a fine method so long as the tensor is small. I also now have some code that uses MPI-I/O with CTF tensors which may provide more scalable I/O if you need it.

Jakes-Schauer commented 6 years ago

Thanks, that indeed answers my question for now.