腾讯犀牛鸟开源人才培养计划Tencent Rhino-bird Open-source Training Program为响应国家十四五规划的开源战略,腾讯于2021年5月发布“首届腾讯犀牛鸟开源人才培养计划”,希望企业界和教育界形成合力,打造面向高校学生的开源课程和开源实践培养方案,培育开源人才,普及开源文化,助力开源人才生态的发展。
Project: ncnn Mat conversion for MacOS and iOS platforms
项目:ncnn MacOS 和 iOS 平台的 Mat 转换
Brief Explanation: There are conversion functions for android Bitmap object, but the corresponding MacOS and iOS ones are missing at the moment. We propose a project to add convenient functions between MacOS and iOS platform image object and ncnn::Mat. In addition, testing and documentation is a MUST to keep maintainability.
ncnn::from_apple_cgimage and ncnn::to_apple_cgimage
ncnn::from_macos_nsimage and ncnn::to_macos_nsimage
ncnn::from_ios_uiimage and ncnn::to_ios_uiimage
Testing and documentation.
期望结果:
ncnn::from_apple_cgimage 和 ncnn::to_apple_cgimage
ncnn::from_macos_nsimage 和 ncnn::to_macos_nsimage
ncnn::from_ios_uiimage 和 ncnn::to_ios_uiimage
测试和文档
Knowledge Prerequisite:
C++ / Objective-C
Image Processing
预备知识:
C++ / Objective-C
图像处理
Level: Easy/Medium
难度:简单/中等
Mentor: nihui
导师:nihui
Project: ncnn Mat pixel type RGB565 and convert_color family
项目:ncnn Mat pixel 类型 RGB565 和 convert_color 函数
Brief Explanation: It is quite common that RGB565 is used as the native window buffer format on mobile deivces. RGB565 saves memory usage compared to RGBA8888. We propose a project to support the RGB565 pixel format conversion and expose more accelerated pixel manipulation functions. In addition, testing and documentation is a MUST to keep maintainability.
ncnn::from_pixels and ncnn::to_pixels family gain RGB565 support.
ncnn::convert_color for most pixel format conversion.
Testing and documentation.
期望结果:
ncnn::from_pixels 和 ncnn::to_pixels 获得 RGB565 支持
ncnn::convert_color 支持各种 pixel 格式转换
测试和文档
Knowledge Prerequisite:
C++
Image Processing
预备知识:
C++
图像处理
Level: Easy/Medium
难度:简单/中等
Mentor: nihui
导师:nihui
Project: ncnn support AVX only CPU
项目:ncnn 支持 AVX only CPU
Brief Explanation: There are some processors without AVX2 and FMA capability. In such circumstance, ncnn fallbacks to SSE2 optimized path now. We propose a project to add AVX-only kernel implementation to ncnn so that these chips could benefit floating point arithmetic speedup from AVX. In addition, testing and documentation is a MUST to keep maintainability.
Automatic dispatch AVX-only layer class on AVX-only CPU.
Feasible speedup.
Testing and documentation.
期望结果:
在 AVX-only CPU 上自动使用 AVX-only 优化的层
可观的速度提升
测试和文档
Knowledge Prerequisite:
C++
预备知识:
C++
Level: Medium
难度:中等
Mentor: nihui
导师:nihui
Project: ncnn improve simpleomp (openmp runtime)
项目:ncnn 改善 simpleomp (openmp 运行时)
Brief Explanation: simpleomp is the minimal openmp runtime implementation mainly used in WebAssembly. It brings multithreading support on platforms without any openmp runtime library. We propose a project to improve the threading pool implementation inside simpleomp via spinlock and futex extension. We expect to see speed improvements on small model inference. In addition, testing and documentation is a MUST to keep maintainability.
Project: ncnn better android vulkan camera frame importing and rendering
项目:ncnn 更好的 android vulkan 相机帧导入和渲染
Brief Explanation: GPU is a suitable device for image processing and machine learning workflow. For realtime processing, we shall try best to eliminate all possible overheads involing the inefficient CPU-GPU data copy. VK_ANDROID_external_memory_android_hardware_buffer is a new vulkan extension that allows us to access the on-device camera captured frame pixels on Android platform. However, it is quite hard to write raw vulkan code directly for utilizing it. We propose a project to emerge an easy to use high-level api for Android vulkan and ncnn interop. In addition, testing and documentation is a MUST to keep maintainability.
Brief Explanation: RISC-V Vector Extension is the RISC-V SIMD instruction set. ncnn has established an infrastructure for RISC-V code optimization, but the optimization related to RISC-V is lacking. We propose a project to add RISC-V kernel implementation to ncnn so that these chips could benefit floating point arithmetic speedup from RISC-V Vector Extension. In addition, testing and documentation is a MUST to keep maintainability.
Implement some optimized kernel for RISC-V architecture.
Testing and documentation.
期望结果:
为 RISC-V 架构实现一些优化的 kernel
测试和文档
Knowledge Prerequisite:
C++
RISC-V Assembly
预备知识:
C++
RISC-V 汇编
Level: Advanced
难度:高级
Mentor: nihui
导师:nihui
Project: ncnn 3D convolution neural network
项目:ncnn 3D 卷积神经网络
Brief Explanation: 3D CNN operators are often used in video processing models. There is almost zero support of them currently. We propose a project to introduce the essential data structure changes and operator implementations for deploying a basic 3D CNN model. In addition, testing and documentation is a MUST to keep maintainability.
简要:3D CNN 算子常用在视频处理模型中。当前 ncnn 不支持 3D 算子。我们提议本项目,做出必要的数据结构更改和算子实现,实现基本的 3D CNN 模型的部署。此外,必须有测试和文档确保可维护性。
腾讯犀牛鸟开源人才培养计划Tencent Rhino-bird Open-source Training Program为响应国家十四五规划的开源战略,腾讯于2021年5月发布“首届腾讯犀牛鸟开源人才培养计划”,希望企业界和教育界形成合力,打造面向高校学生的开源课程和开源实践培养方案,培育开源人才,普及开源文化,助力开源人才生态的发展。
首届项目由教育部计算机教学指导委员会、教育部软件工程教学指导委员会、中国信息通信研究院作为联合指导单位,包括开源基础培训、开源进阶研学、开源项目实战三个环节,项目涉及云原生、微服务、容器、AI、大数据、前端、物联网/边缘计算等多项技术热点。
各位同学报名即可参加线上开源系列课程,学习开源知识,对话学界及产业界大咖。其中入选开源项目实战的学生将在导师的一对一指导下,利用暑假开展编程实战,并有机会获得腾讯开源贡献者证书(目前全球只发出了10+张)。
更多项目信息请看官网:https://opensource.tencent.com/summer-of-code
项目简介
一个为手机端极致优化的高性能神经网络前向计算框架。基于 ncnn,开发者能够将深度学习算法轻松移植到手机端高效执行,开发出人工智能 APP,将 AI 带到你的指尖。
项目GitHub地址:https://github.com/Tencent/ncnn
项目导师
倪辉,ncnn作者
导师寄语
ncnn是业界知名的AI推理框架,社区支持很好,参与研发是能学到知识和技术的,导师会很耐心帮助你的,对吧对吧!
Welcome to advocate your proposal !
欢迎发出您提案!
Project: ncnn Mat conversion for MacOS and iOS platforms
项目:ncnn MacOS 和 iOS 平台的 Mat 转换
Brief Explanation: There are conversion functions for android Bitmap object, but the corresponding MacOS and iOS ones are missing at the moment. We propose a project to add convenient functions between MacOS and iOS platform image object and
ncnn::Mat
. In addition, testing and documentation is a MUST to keep maintainability.简要:android Bitmap 对象的转换函数已经有了,但对应的 MacOS 和 iOS 的转换函数还没有。我们提议本项目,为 MacOS 和 iOS 平台添加的图像对象与
ncnn::Mat
互转的方便函数。此外,必须有测试和文档确保可维护性。Expected Results:
ncnn::from_apple_cgimage
andncnn::to_apple_cgimage
ncnn::from_macos_nsimage
andncnn::to_macos_nsimage
ncnn::from_ios_uiimage
andncnn::to_ios_uiimage
期望结果:
ncnn::from_apple_cgimage
和ncnn::to_apple_cgimage
ncnn::from_macos_nsimage
和ncnn::to_macos_nsimage
ncnn::from_ios_uiimage
和ncnn::to_ios_uiimage
Knowledge Prerequisite:
预备知识:
Level: Easy/Medium
难度:简单/中等
Mentor: nihui
导师:nihui
Project: ncnn Mat pixel type RGB565 and convert_color family
项目:ncnn Mat pixel 类型 RGB565 和 convert_color 函数
Brief Explanation: It is quite common that RGB565 is used as the native window buffer format on mobile deivces. RGB565 saves memory usage compared to RGBA8888. We propose a project to support the RGB565 pixel format conversion and expose more accelerated pixel manipulation functions. In addition, testing and documentation is a MUST to keep maintainability.
简要:在移动设备上,RGB565 是很常见用于原生窗口缓冲区的格式。RGB565 相较于 RGBA8888 节省了内存使用。我们提议本项目,增加 RGB565 pixel 格式的转换,并提供加速的 pixel 处理函数。此外,必须有测试和文档确保可维护性。
Expected Results:
ncnn::from_pixels
andncnn::to_pixels
family gain RGB565 support.ncnn::convert_color
for most pixel format conversion.期望结果:
ncnn::from_pixels
和ncnn::to_pixels
获得 RGB565 支持ncnn::convert_color
支持各种 pixel 格式转换Knowledge Prerequisite:
预备知识:
Level: Easy/Medium
难度:简单/中等
Mentor: nihui
导师:nihui
Project: ncnn support AVX only CPU
项目:ncnn 支持 AVX only CPU
Brief Explanation: There are some processors without AVX2 and FMA capability. In such circumstance, ncnn fallbacks to SSE2 optimized path now. We propose a project to add AVX-only kernel implementation to ncnn so that these chips could benefit floating point arithmetic speedup from AVX. In addition, testing and documentation is a MUST to keep maintainability.
简要:有些 CPU 不具备 AVX2 和 FMA 指令集。这种情况下,ncnn 会回退到 SSE2 的优化代码。我们提议本项目,ncnn 添加 AVX-only 的 kernel 实现,以便在这些芯片上享受到 AVX 的加速效果。此外,必须有测试和文档确保可维护性。
Expected Results:
期望结果:
Knowledge Prerequisite:
预备知识:
Level: Medium
难度:中等
Mentor: nihui
导师:nihui
Project: ncnn improve simpleomp (openmp runtime)
项目:ncnn 改善 simpleomp (openmp 运行时)
Brief Explanation: simpleomp is the minimal openmp runtime implementation mainly used in WebAssembly. It brings multithreading support on platforms without any openmp runtime library. We propose a project to improve the threading pool implementation inside simpleomp via spinlock and futex extension. We expect to see speed improvements on small model inference. In addition, testing and documentation is a MUST to keep maintainability.
简要:simpleomp 是最小化的 openmp 运行时实现,主要用于 WebAssembly。它可以在没有 openmp 运行时库的平台中带来多线程加速。我们提议本项目,改善 simpleomp 线程池实现,如利用 spinlock 和 futex 扩展。此外,必须有测试和文档确保可维护性。
Expected Results:
set_kmp_blocktime
interface.期望结果:
set_kmp_blocktime
接口Knowledge Prerequisite:
预备知识:
Level: Medium
难度:中等
Mentor: nihui
导师:nihui
Project: ncnn better android vulkan camera frame importing and rendering
项目:ncnn 更好的 android vulkan 相机帧导入和渲染
Brief Explanation: GPU is a suitable device for image processing and machine learning workflow. For realtime processing, we shall try best to eliminate all possible overheads involing the inefficient CPU-GPU data copy.
VK_ANDROID_external_memory_android_hardware_buffer
is a new vulkan extension that allows us to access the on-device camera captured frame pixels on Android platform. However, it is quite hard to write raw vulkan code directly for utilizing it. We propose a project to emerge an easy to use high-level api for Android vulkan and ncnn interop. In addition, testing and documentation is a MUST to keep maintainability.简要:GPU 是个适合用作图像处理和机器学习的运算设备。对于实时处理,我们应当尽可能减少影响效率的 CPU-GPU 数据拷贝。
VK_ANDROID_external_memory_android_hardware_buffer
是新的 vulkan 扩展,在 android 平台上可以用该扩展访问相机捕获到的帧像素。然而,编写原生的 vulkan 代码相当困难。我们提议本项目,实现一套简单易用的高层 API 用作 Android vulkan 与 ncnn 交互。此外,必须有测试和文档确保可维护性。Expected Results:
AHardwareBuffer
toVkMat
orVkImageMat
.VkMat
orVkImageMat
toANativeWindow
.期望结果:
AHardwareBuffer
到VkMat
或VkImageMat
VkMat
或VkImageMat
到ANativeWindow
Knowledge Prerequisite:
预备知识:
Level: Medium/Advanced
难度:中等/高级
Mentor: nihui
导师:nihui
Project: ncnn RISC-V V-extension optimization
项目:ncnn RISC-V V-extension 优化
Brief Explanation: RISC-V Vector Extension is the RISC-V SIMD instruction set. ncnn has established an infrastructure for RISC-V code optimization, but the optimization related to RISC-V is lacking. We propose a project to add RISC-V kernel implementation to ncnn so that these chips could benefit floating point arithmetic speedup from RISC-V Vector Extension. In addition, testing and documentation is a MUST to keep maintainability.
简要:RISC-V Vector 扩展是 RISC-V SIMD 指令集。ncnn 已经实现了 RISC-V 代码优化的基础架构,但缺少 RISC-V 相关优化。我们提议本项目,为 ncnn 添加 RISC-V kernel 实现,让这些芯片可以享受到 RISC-V Vector 扩展带来的浮点计算加速。此外,必须有测试和文档确保可维护性。
Expected Results:
期望结果:
Knowledge Prerequisite:
预备知识:
Level: Advanced
难度:高级
Mentor: nihui
导师:nihui
Project: ncnn 3D convolution neural network
项目:ncnn 3D 卷积神经网络
Brief Explanation: 3D CNN operators are often used in video processing models. There is almost zero support of them currently. We propose a project to introduce the essential data structure changes and operator implementations for deploying a basic 3D CNN model. In addition, testing and documentation is a MUST to keep maintainability.
简要:3D CNN 算子常用在视频处理模型中。当前 ncnn 不支持 3D 算子。我们提议本项目,做出必要的数据结构更改和算子实现,实现基本的 3D CNN 模型的部署。此外,必须有测试和文档确保可维护性。
Relevant issues:
相关 issue:
Expected Results:
ncnn::Mat d
(depth) member.期望结果:
ncnn::Mat d
(depth) 成员Knowledge Prerequisite:
预备知识:
Level: Advanced
难度:高级
Mentor: nihui
导师:nihui
联系导师
倪辉:nihuini@tencent.com