项目自荐 | TimeWarp - 在线视频背景替换

[Charmve]

项目推荐

• 项目地址：https://github.com/Charmve/TimeWarp

• 类别：计算机视觉 + web

• 项目后续更新计划：开发在线视频客户端/server

• 项目描述：

  • 必写：你是否还在为老师突然的视频会议慌乱了脚步？你是否还在查岗吓得不知所措？你是否担心被别人看到你脏乱的房间？突如其来的疫情加速了虚拟世界的到来，在线视频的需求也得到了极致的展现。本项目结合在线视频抠图算法，实时替换视频背景。  ;

  • 推荐理由：即使现在zoom等少数平台已经开放此类功能，但是在线视频会议领域，能够哦做到快速替换背景、抠图效果达到如此之高的还是十分少的。运用的算法是今年华盛顿大学刚发布与CVPR的matting算法。

  • 编写详细的READEME和程序注释，开发者可很好的应用到自己的项目中。

👉 See image results with interactive zoom-in

✨ Features

• Real-Time High-Resolution Background Matting,  ;
• Replacing our background in meeting screen;
• Replace background with any size photo you want;
• Cross mutil-platform, like zoom plugin, web-browser, miniprogram in PC, mobile and iPad, any platform with web service;
• IFLYTEK voice translation between Chinese and English;
• ...

示例代码：

import torch
from torchvision.transforms.functional import to_tensor, to_pil_image
from PIL import Image
model = torch.jit.load('model/TorchScript/torchscript_resnet50_fp32.pth').eval() 
# RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a # # CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
# model = torch.load('model/TorchScript/torchscript_resnet50_fp32.pth',map_location ='cpu')
src = Image.open('dataset/img/12.png')
bgr = Image.open('dataset/bgr/12.png')
src = to_tensor(src).unsqueeze(0)
bgr = to_tensor(bgr).unsqueeze(0)
if src.size(2) <= 2048 and src.size(3) <= 2048:
  model.backbone_scale = 1/4
  model.refine_sample_pixels = 80_000
else:
  model.backbone_scale = 1/8
  model.refine_sample_pixels = 320_000
pha, fgr = model(src, bgr)[:2]
com = pha * fgr + (1 - pha) * torch.tensor([120/255, 255/255, 155/255], device='cpu').view(1, 3, 1, 1)
to_pil_image(com[0].cpu())
to_pil_image(pha[0].cpu()).save('result/pha.png')
to_pil_image(fgr[0].cpu()).save('result/fgr.png')
to_pil_image(com[0].cpu()).save('result/com.png')

截图

demo Real-Time High-Resolution Background Matting^[1]

[521xueweihan]

非常感谢您推荐项目。

该项目暂不能收录到 HelloGitHub 月刊中，HelloGitHub 推荐项目审核标准 #271。 期待持续完善该项目，后续推荐更多的项目。

再次感谢您对 HelloGitHub 的支持 🙏