BabitMF / bmf

Cross-platform, customizable multimedia/video processing framework. With strong GPU acceleration, heterogeneous design, multi-language support, easy to use, multi-framework compatible and high performance, the framework is ideal for transcoding, AI inference, algorithm integration, live video streaming, and more.
https://babitmf.github.io/
Apache License 2.0
816 stars 71 forks source link

fill_task_input don't consider the incomplete data , is it implied that the module must consider the incomplete data and carry out local caching and splicing processing when handling (TASK)? #85

Closed lxllsl closed 10 months ago

lxllsl commented 10 months ago

If a certain module (MODULE) requires multiple input streams (such as OVERLAY) to function normally, and only some of these input streams have data, is it implied that the module must consider the incomplete data and carry out local caching and splicing processing when handling (TASK)? Otherwise, there will be incomplete input data, leading to processing failure.

bool ImmediateInputStreamManager::fill_task_input(Task &task) { bool task_filled = false; for (auto & input_stream : inputstreams) {

        if (input_stream.second->is_empty()) {
            continue;
        }
        //one task cantain mult pkts, NEED add max pkts ctl?
        while (not input_stream.second->is_empty()) {
            Packet pkt = input_stream.second->pop_next_packet(false);
            if (pkt.timestamp() == BMF_EOF) {
                if (input_stream.second->probed_) {
                    BMFLOG(BMF_INFO) << "immediate sync got EOF from dynamical update";
                    pkt.set_timestamp(DYN_EOS);
                    input_stream.second->probed_ = false;
                } else
                    stream_done_[input_stream.first] = 1;
            }
            //READ:取到task的对应输入队列中
            task.fill_input_packet(input_stream.second->get_id(), pkt);
            task_filled = true;
        }
    }
HuHeng commented 10 months ago
If a certain module (MODULE) requires multiple input streams (such as OVERLAY) to function normally, and only some of these input streams have data, is it implied that the module must consider the incomplete data and carry out local caching and splicing processing when handling (TASK)?

Yes. By default, the answer is yes. The default input mode for the BMF module is "immediate," which corresponds to the ImmediateInputStreamManager you mentioned. In the case of multiple inputs, there is no guarantee that each input will have packet data on every process call. There is another inputmanager type called FramesyncInputStreamManager, which ensures that each input has packets. It may perform frame dropping or frame interpolation operations based on the timestamp of frame.