#2021Tencent Rhino-bird Open-source Training Program—ncnn#

[tencent-adm]

腾讯犀牛鸟开源人才培养计划Tencent Rhino-bird Open-source Training Program为响应国家十四五规划的开源战略，腾讯于2021年5月发布“首届腾讯犀牛鸟开源人才培养计划”，希望企业界和教育界形成合力，打造面向高校学生的开源课程和开源实践培养方案，培育开源人才，普及开源文化，助力开源人才生态的发展。

首届项目由教育部计算机教学指导委员会、教育部软件工程教学指导委员会、中国信息通信研究院作为联合指导单位，包括开源基础培训、开源进阶研学、开源项目实战三个环节，项目涉及云原生、微服务、容器、AI、大数据、前端、物联网/边缘计算等多项技术热点。

各位同学报名即可参加线上开源系列课程，学习开源知识，对话学界及产业界大咖。其中入选开源项目实战的学生将在导师的一对一指导下，利用暑假开展编程实战，并有机会获得腾讯开源贡献者证书（目前全球只发出了10+张）。

更多项目信息请看官网：https://opensource.tencent.com/summer-of-code

项目简介

一个为手机端极致优化的高性能神经网络前向计算框架。基于 ncnn，开发者能够将深度学习算法轻松移植到手机端高效执行，开发出人工智能 APP，将 AI 带到你的指尖。

项目GitHub地址：https://github.com/Tencent/ncnn

项目导师

倪辉，ncnn作者

导师寄语

ncnn是业界知名的AI推理框架，社区支持很好，参与研发是能学到知识和技术的，导师会很耐心帮助你的，对吧对吧！

Welcome to advocate your proposal !

欢迎发出您提案！

---

Project: ncnn Mat conversion for MacOS and iOS platforms

项目：ncnn MacOS 和 iOS 平台的 Mat 转换

Brief Explanation: There are conversion functions for android Bitmap object, but the corresponding MacOS and iOS ones are missing at the moment. We propose a project to add convenient functions between MacOS and iOS platform image object and ncnn::Mat. In addition, testing and documentation is a MUST to keep maintainability.

简要：android Bitmap 对象的转换函数已经有了，但对应的 MacOS 和 iOS 的转换函数还没有。我们提议本项目，为 MacOS 和 iOS 平台添加的图像对象与ncnn::Mat互转的方便函数。此外，必须有测试和文档确保可维护性。

Expected Results:

• ncnn::from_apple_cgimage and ncnn::to_apple_cgimage
• ncnn::from_macos_nsimage and ncnn::to_macos_nsimage
• ncnn::from_ios_uiimage and ncnn::to_ios_uiimage
• Testing and documentation.

期望结果：

• ncnn::from_apple_cgimage 和 ncnn::to_apple_cgimage
• ncnn::from_macos_nsimage 和 ncnn::to_macos_nsimage
• ncnn::from_ios_uiimage 和 ncnn::to_ios_uiimage
• 测试和文档

Knowledge Prerequisite:

• C++ / Objective-C
• Image Processing

预备知识：

• C++ / Objective-C
• 图像处理

Level: Easy/Medium

难度：简单/中等

Mentor: nihui

导师：nihui

---

Project: ncnn Mat pixel type RGB565 and convert_color family

项目：ncnn Mat pixel 类型 RGB565 和 convert_color 函数

Brief Explanation: It is quite common that RGB565 is used as the native window buffer format on mobile deivces. RGB565 saves memory usage compared to RGBA8888. We propose a project to support the RGB565 pixel format conversion and expose more accelerated pixel manipulation functions. In addition, testing and documentation is a MUST to keep maintainability.

简要：在移动设备上，RGB565 是很常见用于原生窗口缓冲区的格式。RGB565 相较于 RGBA8888 节省了内存使用。我们提议本项目，增加 RGB565 pixel 格式的转换，并提供加速的 pixel 处理函数。此外，必须有测试和文档确保可维护性。

Expected Results:

• ncnn::from_pixels and ncnn::to_pixels family gain RGB565 support.
• ncnn::convert_color for most pixel format conversion.
• Testing and documentation.

期望结果：

• ncnn::from_pixels 和 ncnn::to_pixels 获得 RGB565 支持
• ncnn::convert_color 支持各种 pixel 格式转换
• 测试和文档

Knowledge Prerequisite:

• C++
• Image Processing

预备知识：

• C++
• 图像处理

Level: Easy/Medium

难度：简单/中等

Mentor: nihui

导师：nihui

---

Project: ncnn support AVX only CPU

项目：ncnn 支持 AVX only CPU

Brief Explanation: There are some processors without AVX2 and FMA capability. In such circumstance, ncnn fallbacks to SSE2 optimized path now. We propose a project to add AVX-only kernel implementation to ncnn so that these chips could benefit floating point arithmetic speedup from AVX. In addition, testing and documentation is a MUST to keep maintainability.

简要：有些 CPU 不具备 AVX2 和 FMA 指令集。这种情况下，ncnn 会回退到 SSE2 的优化代码。我们提议本项目，ncnn 添加 AVX-only 的 kernel 实现，以便在这些芯片上享受到 AVX 的加速效果。此外，必须有测试和文档确保可维护性。

Expected Results:

• Automatic dispatch AVX-only layer class on AVX-only CPU.
• Feasible speedup.
• Testing and documentation.

期望结果：

• 在 AVX-only CPU 上自动使用 AVX-only 优化的层
• 可观的速度提升
• 测试和文档

Knowledge Prerequisite:

• C++

预备知识：

• C++

Level: Medium

难度：中等

Mentor: nihui

导师：nihui

---

Project: ncnn improve simpleomp (openmp runtime)

项目：ncnn 改善 simpleomp (openmp 运行时)

Brief Explanation: simpleomp is the minimal openmp runtime implementation mainly used in WebAssembly. It brings multithreading support on platforms without any openmp runtime library. We propose a project to improve the threading pool implementation inside simpleomp via spinlock and futex extension. We expect to see speed improvements on small model inference. In addition, testing and documentation is a MUST to keep maintainability.

简要：simpleomp 是最小化的 openmp 运行时实现，主要用于 WebAssembly。它可以在没有 openmp 运行时库的平台中带来多线程加速。我们提议本项目，改善 simpleomp 线程池实现，如利用 spinlock 和 futex 扩展。此外，必须有测试和文档确保可维护性。

Expected Results:

• set_kmp_blocktime interface.
• Feasible multithreading speedup on small models.
• Testing and documentation.

期望结果：

• set_kmp_blocktime 接口
• 在小型模型上有可观的速度提升
• 测试和文档

Knowledge Prerequisite:

• C++
• Linux

预备知识：

• C++
• Linux

Level: Medium

难度：中等

Mentor: nihui

导师：nihui

---

Project: ncnn better android vulkan camera frame importing and rendering

项目：ncnn 更好的 android vulkan 相机帧导入和渲染

Brief Explanation: GPU is a suitable device for image processing and machine learning workflow. For realtime processing, we shall try best to eliminate all possible overheads involing the inefficient CPU-GPU data copy. VK_ANDROID_external_memory_android_hardware_buffer is a new vulkan extension that allows us to access the on-device camera captured frame pixels on Android platform. However, it is quite hard to write raw vulkan code directly for utilizing it. We propose a project to emerge an easy to use high-level api for Android vulkan and ncnn interop. In addition, testing and documentation is a MUST to keep maintainability.

简要：GPU 是个适合用作图像处理和机器学习的运算设备。对于实时处理，我们应当尽可能减少影响效率的 CPU-GPU 数据拷贝。VK_ANDROID_external_memory_android_hardware_buffer是新的 vulkan 扩展，在 android 平台上可以用该扩展访问相机捕获到的帧像素。然而，编写原生的 vulkan 代码相当困难。我们提议本项目，实现一套简单易用的高层 API 用作 Android vulkan 与 ncnn 交互。此外，必须有测试和文档确保可维护性。

Expected Results:

• Easy to use high-level api for importing AHardwareBuffer to VkMat or VkImageMat.
• Easy to use high-level api for rendering VkMat or VkImageMat to ANativeWindow.
• An end-to-end vulkan GPU processing android sample application.
• Testing and documentation.

期望结果：

• 好用的高层 API，用于导入 AHardwareBuffer 到 VkMat 或 VkImageMat
• 好用的高层 API，用于渲染 VkMat 或 VkImageMat 到 ANativeWindow
• 端到端 vulkan GPU 处理的 android 示例应用程序
• 测试和文档

Knowledge Prerequisite:

• C++
• Android
• Vulkan

预备知识：

• C++
• Android
• Vulkan

Level: Medium/Advanced

难度：中等/高级

Mentor: nihui

导师：nihui

---

Project: ncnn RISC-V V-extension optimization

项目：ncnn RISC-V V-extension 优化

Brief Explanation: RISC-V Vector Extension is the RISC-V SIMD instruction set. ncnn has established an infrastructure for RISC-V code optimization, but the optimization related to RISC-V is lacking. We propose a project to add RISC-V kernel implementation to ncnn so that these chips could benefit floating point arithmetic speedup from RISC-V Vector Extension. In addition, testing and documentation is a MUST to keep maintainability.

简要：RISC-V Vector 扩展是 RISC-V SIMD 指令集。ncnn 已经实现了 RISC-V 代码优化的基础架构，但缺少 RISC-V 相关优化。我们提议本项目，为 ncnn 添加 RISC-V kernel 实现，让这些芯片可以享受到 RISC-V Vector 扩展带来的浮点计算加速。此外，必须有测试和文档确保可维护性。

Expected Results:

• Implement some optimized kernel for RISC-V architecture.
• Testing and documentation.

期望结果：

• 为 RISC-V 架构实现一些优化的 kernel
• 测试和文档

Knowledge Prerequisite:

• C++
• RISC-V Assembly

预备知识：

• C++
• RISC-V 汇编

Level: Advanced

难度：高级

Mentor: nihui

导师：nihui

---

Project: ncnn 3D convolution neural network

项目：ncnn 3D 卷积神经网络

Brief Explanation: 3D CNN operators are often used in video processing models. There is almost zero support of them currently. We propose a project to introduce the essential data structure changes and operator implementations for deploying a basic 3D CNN model. In addition, testing and documentation is a MUST to keep maintainability.

简要：3D CNN 算子常用在视频处理模型中。当前 ncnn 不支持 3D 算子。我们提议本项目，做出必要的数据结构更改和算子实现，实现基本的 3D CNN 模型的部署。此外，必须有测试和文档确保可维护性。

Relevant issues:

• https://github.com/Tencent/ncnn/issues/637
• https://github.com/Tencent/ncnn/issues/889
• https://github.com/Tencent/ncnn/issues/1358
• https://github.com/Tencent/ncnn/issues/1726

相关 issue：

• https://github.com/Tencent/ncnn/issues/637
• https://github.com/Tencent/ncnn/issues/889
• https://github.com/Tencent/ncnn/issues/1358
• https://github.com/Tencent/ncnn/issues/1726

Expected Results:

• Extend ncnn::Mat d(depth) member.
• Implement the new naive Convolution3D Pooling3D BatchNorm3D and some 3D activation operators.
• Bring 3D operator conversion with caffe/onnx/MLIR/etc. converter.
• Make simple 3D CNN pytorch models deployable with ncnn.
• Testing and documentation.

期望结果：

• 扩展 ncnn::Mat d(depth) 成员
• 实现最简单的 Convolution3D Pooling3D BatchNorm3D 和一些 3D 激活算子
• 为 caffe/onnx/MLIR等转换器添加 3D 算子转换支持
• 使用 ncnn 部署简单的 3D CNN pytorch 模型
• 测试和文档

Knowledge Prerequisite:

• C++
• PyTorch

预备知识：

• C++
• PyTorch

Level: Advanced

难度：高级

Mentor: nihui

导师：nihui

联系导师

倪辉：nihuini@tencent.com

[BUG1989]

粗略对比了下其他几个任务，ncnn 的任务分解很用心啊。 @tencent-adm

[liguoyu666]

请问只能高校学生参加课程吗？已经工作的没法参加吗？这也太蛋疼了吧