search

intel / xFasterTransformer

Apache License 2.0
344 stars 60 forks source link

[Layer] Add SequenceMeta, SequencePool and init pipeline parrallel function. #332

Closed changqi1 closed 4 months ago

changqi1 commented 4 months ago

Done.

src/models/models.cpp
@@ -85,6 +85,13 @@ void Model::input(std::vector<int32_t> &inputIds_, int batchSize_) {
     inputIds.resize(dims[1]);
     if (decoder->getRank() == 0) { inputIds = inputIds_; }
     messenger.broadcast(inputIds.data(), dims[1]);
+
+    if (this->isMaster()) {
+        for (int i = 0; i < 2; ++i) {
+            int sequenceID = SequencePool::getInstance().createSequenceID();
+            InputQueue::getInstance().push(SequencePool::getInstance().createMeta(sequenceID, seqLen, inputIds));
+        }
+    }
 }