Video inference sample code

[siyangbing]

I used my own video classification trained with the configuration of ucf101. It seems that there is a little overfitting. I want to use my code for inference, but I can't find a related tutorial. I can only find it, and I don't know what to do next. plaese help me @mannatsingh

from classy_vision.generic.util import load_checkpoint
from classy_vision.models import ClassyModel

# This is important: importing models here will register your custom models with Classy Vision
# so that it can instantiate them appropriately from the checkpoint file
# See more information at https://classyvision.ai/api/models.html#classy_vision.models.register_model
from classy_vision import models

# Update this with your actual directory:
checkpoint_dir = '/home/sucom/hdd_1T/project/video_rec_0831/ClassyVision/checkpoint_hand/model_phase-2198_end.torch'
checkpoint_data = load_checkpoint(checkpoint_dir)
model = ClassyModel.from_checkpoint(checkpoint_data)

[siyangbing]

please help me , @mannatsingh

[mannatsingh]

@siyangbing you're right, we don't have a tutorial for inference at the moment since the API isn't finalized.

If you want to evaluate a trained checkpoint on a test dataset, you can create a new task similar to the one you created for training, with the following differences -

task.set_test_only(True)
task.set_num_epochs(1)
task.set_dataset(test_dataset, "test")
task.set_checkpoint("path/to/checkpoint")

Note that this API is liable to change in the future.

Let me know if this works for you!

[siyangbing]

@siyangbing，您是对的，由于API尚未最终确定，我们目前没有用于推理的教程。

如果要评估测试数据集上经过训练的检查点，则可以创建一个新任务，该任务类似于您为训练而创建的任务，但有以下区别：

任务。set_test_only（True）
 任务。set_num_epochs（1）
 任务。set_dataset（test_dataset，“ test”）
 任务。set_checkpoint（“路径/到/检查点”）

请注意，此API将来可能会更改。

让我知道这是否适合您！

Thank you very much for your reply, but for me, the task is too complicated for me to understand. I recently have a project that needs to identify a single video. Is it possible to infer a single video directly by loading a checkpoint instead of registering a data set and using tasks. What do I need to do with a single video before I can input the model for inference, please help me! @mannatsingh

[siyangbing]

@siyangbing，您是对的，由于API尚未最终确定，我们目前没有用于推理的教程。 如果要评估测试数据集上经过训练的检查点，则可以创建一个新任务，该任务类似于您为训练而创建的任务，但有以下区别：

任务。set_test_only（True）
 任务。set_num_epochs（1）
 任务。set_dataset（test_dataset，“ test”）
 任务。set_checkpoint（“路径/到/检查点”）

请注意，此API将来可能会更改。 让我知道这是否适合您！

Thank you very much for your reply, but for me, the task is too complicated for me to understand. I recently have a project that needs to identify a single video. Is it possible to infer a single video directly by loading a checkpoint instead of registering a data set and using tasks. What do I need to do with a single video before I can input the model for inference, please help me! @mannatsingh

please help me , the project is perfect if I can infer one video with my own model!

[mannatsingh]

@stephenyan1231 do you have any suggestions for the best way to do this?

[siyangbing]

@stephenyan1231 do you have any suggestions for the best way to do this?

it will get support in future relase? @mannatsingh

[mannatsingh]

@siyangbing we have good support for this for images, but that's missing a tutorial. For videos, we need to add better support!

[siyangbing]

@siyangbing我们对此图像有很好的支持，但是缺少教程。对于视频，我们需要添加更好的支持！

If there is support for video inference in the future, please let me know　@mannatsingh

[shinianzhihou]

state_path = "YOUR/PATH/TO/checkpoints/checkpoint.torch"
state = torch.load(state_path)
model = ClassyModel.from_checkpoint(state)
dataset = build_dataset(state["input_args"]["config"]["dataset"]["test"])

dataset.batchsize_per_replica = 10 # to test in the single GPU with small memory

targets,outputs = [],[]
model = model.cuda().eval()
for idx,data in enumerate(dataset.iterator()):
    output = model(data['input'].cuda()).argmax(dim=1)
    outputs += list(output.detach().cpu().numpy())
    targets += list(data["target"].numpy())

from sklearn.metrics import classification_report
print(classification_report(targets, outputs, target_names=CLASSES))

This is how I make my inference. May be useful

[mannatsingh]

Thanks for the code sample, @shinianzhihou !