DeepLearning Excellent Open Course

Bryce1010 commented 4 years ago

个人主页，个人学习生涯！

学习流程：

第一遍，通读全文，了解内容

第二遍，针对性阅读，并记录心得

第三遍，理论结合实践

一点一点搬运到博客上

ref

Foreword:

[x] 已完成

[ ] 已完成完成但需要多遍复习

还在计划学习

5. 深度学习

博客

Videos & Images Theory and Analytics Laboratory (VITAL) of Sherbrooke University[laboratory] [blog]

非常好的公共资源，有他们组的weekly-reading group& weekly machine learning videos.
Lil’Log [blog]
Berkeley Artificial Intelligence Research [Bair]
Jay Alammar(NLP 大佬) [blog]
Createmomo [blog]

包含许多深度学习的基础
Jonathan Hui blog [blog]

关于深度学习，很久不更新了
daiwk’s blog [blog]

覆盖深度学习各方面
AI Learn Notes [github]
telesens.co[blog]
研究资源列表 A curated list of research resources [github]
Yoshua Bengio [blog]

Bryce1010 commented 4 years ago

公开课与资料

[ ] 图像处理 Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital [[couresa Duke]](Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital)
[ ] Deep Bayes 2019[url]
[ ] DeepLearning.ai C4W1L01[url]
[ ] 数学推荐[url]
[ ] 2019 CIFAR DLRL Summer School Lectures & ZBengio深度学习&强化学习暑期班课程[url]
[ ] Neral Networks: Tricks of the Trade[url]
[ ] 迁移学习简明手册[url]
[ ] 机器学习(Machine Learning)、深度学习(Deep Learning)、NLP面试中常考到的知识点和代码实现[url]
[ ] PyData NYC python 和 Machine Learning[url]
[ ] Hacker Tools程序员超实用工具课程[url]
[ ] Cousera[url]
[ ] The Age of A.I. （AI纪录片）[url]
[ ] Learn TensorFlow in 3 Hours [url]
[ ] CVSS 2019 Computational Vision Summer School [url]
[ ] NIPS 2019 Hightlight papers [url]
- [ ] Advanced AI Deep Reinforcement Learning in Python [url]
[ ] 编程随想电子书目录 [github]
[ ] fun with graphics 图像学相关课程、书籍、教程[github]
[ ] Deep Learning Papers Reading Roadmap [github]
[ ] CMU2020深度学习进阶 [url]
[ ] 【公开课】旷视x北大《深度学习实践》（28课时全) [bilibili]
[ ] Introducing to Parallel Programming [udacity cs344] [nvidia]
[ ] 纽约大学《深度学习(PyTorch)》课程(2020) by yann Lecun, Alfredo Canziani [bilibili]
[ ] 机器学习进阶列表公开课 [reddit]
[ ] 【2020伯克利】深度无监督学习 [bilibili]

书籍

https://github.com/loveunk/Deep-learning-books

Bryce1010 commented 4 years ago

顶会与期刊

AI Conference Deadlines [url]
2019ICCV [all papers] [github]
2019CVPR [all papers] [github]
2020CVPR [github]

这一部分是DeepLearning的代码模板，主要任务

收集deeplearning各方向的优秀代码

PyTorch, TensorFlow, Keras等框架的训练技巧

优秀的知识点总结和review

Bryce1010 commented 4 years ago

Python

[ ] Python练习册，每天一个小程序 [github]
[ ] python-cheatsheet [github]
[x] 自学Python——编程基础、科学计算及数据分析-中文python笔记 [github]
[ ] What the fuck python [github]
[ ] GitHub标星8100：Python中文资源，从新手到老司机，只要100天: [github]
[ ] Python 进阶-Real Python tutorials [url]
《Intermediate Python》的中译版本 [doc]
Python Data Science Handbook [book]
[ ] Python有趣的小例子 [url]
[ ] Pandas [url]
[ ] Data-Science–Cheat-Sheet [github]
分享8点超级有用的Python编程建议 [weixin]
[ ] 【个人项目】——Learning DataScience [github]
- 存储DataScience的代码模板；
- 存储DataScience的资料；
[ ] python 文件处理 [zhihu]
[ ] python magic method [url]

Bryce1010 commented 4 years ago

PyTorch

[ ] Pytorch 进阶
Pytorch分布式训练 [zhihu]
[ ] Apachecn:PyTorch 中文文档 & 教程
[ ] pytorch入门与实践推荐
[ ] pytorch-tutorial
[ ] Awesome pytorch
[ ] pytorch 模型训练实用教程推荐
[ ] Pytorch 常用代码段整理
[ ] 国外视频教程：另外一个国外大佬的视频教程，在YouTube上有很高的点击率: https://www.youtube.com/playlist?list=PLlMkM4tgfjnJ3I-dbhO9JTw7gNty6o_2m
[ ] Awesome-Pytorch-Chinese
[ ] Deep Learning Models
[ ] DL_RL_with_pytorch强化学习
[ ] Pytorch Hand book
[ ] PyTorch Ignite 高级API [url]

TensorboardX

Tensorflow

[ ] TensorFlow 2.0官方教程 [url]

Hands-on ML第二版[github]
[ ] Data-Science-Study-Paths-March-2019 [github]
GitHub：TensorFlow 最全资料集锦 [url
tensorflow 中文社区 [url]
tensorflow2.0 入门教程[zhihu]
mofan tensorflow [url]
Hands-on TensorFlow 2.0 from IJCAI19 [IJCAI]

Fast.ai

Fast.ai深度学习实战课程(2019)学习笔记 [weibo]

OpenCV-Python

[ ] OpenCV-Python图像处理源码及教程 [github]
[ ] Youtube: OpenCV Python Tutorial For Beginners
[ ] OpenCV Document [url]

Bryce1010 commented 4 years ago

Writing & Reading

[ ] Latex 笔记模板 [ref]
[x] Youtube: LaTex Tutorial [videos]
[x] How to read a paper [paper]
[ ] Writing Technical Articles [ref]
[ ] Research Skills [ref]
Academic Writing [中文笔记]

5.1 Basic Network

1. Attention

What is Attention?
[x] On the Relationship between Self-Attention and Convolutional Layers [paper] [pytorch]
- Provides evidence that attention layers can perform convolution.
- multi-head self-attention layer with sufficient number of heads is at least as powerful as any convolutional layer
An overview of all the three metods CAM+Grad-CAM+Grad-CAM++[overview]
Grad-CAM [overview][gradcam++ pytorch][gradcam pytorch] [3D grad-cam (theano) ]
Grad-CAM++[pytorch]
[x] (2020 Data Augmentation) GridMask Data Augmentation

2. Capsule Nets

A curated list of awesome resources related to capsule networks[github]
Understanding Hinton’s Capsule Networks. Part I,II,III,IV[Medium]
Understanding Capsule Networks[Medium]
[ ] CVPR2019 Capsule Network [video] [slides]

3. Loss Function

[ ] Loss Function && Machine Learning Metrics 总结 [blog]
Optimization Algorithms for Deep Learning[Medium]
An overview of gradient descent optimization algorithms[url]
On Optimization Methods for Deep Learning[ICML11]
[ ] An Overview of gradient descent optimization algorithms [Arxiv]
[x] (2015 ICLR) ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION [paper]
[ ] 机器学习中的评价指标
[ ] Understanding Optimizers

4. Classification Networks

[ ] awesome image classification[url]
[ ] Classification 综述整理 [bryce1010]
[ ] 深入理解反向传播[url]
[ ] 如何调参[url]
[ ] (2019 ICML )EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks [paper]
[ ] (2019 ICCV )Searching for MobileNetV3 [paper]
[x] (2018 CVPR )Squeeze-and-Excitation Networks(SENet) [paper][code][review]
[ ] (2018 CVPR )Learning Transferable Architectures for Scalable Image Recognition(NASNet) [paper]
[x] (2017 CVPR )Aggregated Residual Transformations for Deep Neural Networks (ResNext) [paper]
(2020 ICLR )Deforming kernels to adapt towards object deformation [github]
pytorch-CIFAR10

torchvision datasets+densenet+dpn+efficientnet+googlenet+lenet+mobilenet+mobilenetv2

+pnasnet+preact_resnet+resnet+resnext+senet+shufflenet+shufflenetv2+vgg

7. Crowd Counting

Awesome Crowd Counting[github]
Crowd_counting_from_scratch[github]
multi-scale convolutional neural network (CNN) for crowd counting[Arxiv2017]
Counting Crowds and Lines with AI[url]
datasets[github]

8. Domain Adaptation

Awesome doamin adaptation[github]

9. Knowledge Distillation

knowledge distillation papers[github]
Distilling the Knowledge in a Neural Network[Medium]
[ ] (实践)Teacher-Student Architecture in Plant Disease Classification [medium]

10. Transfer Learning

迁移学习简明手册 [github]

11. Person-Re-Identification

Awesome -Person-Re-Identification [github]

Bryce1010 commented 4 years ago

5.2 Medical Image Analysis

State-of-the-art medical image segmentation methods based on various challenges[github]
2017 data science bowl dicom图像肺癌检测 [url]
Medical-Classification-APTOS2019_Winner_Review

Bryce1010 commented 4 years ago

5.3 Generative Adversarial Network

SIGAI:生成式对抗网络模型综述[zhihu]
深入浅出：GAN原理与应用入门介绍[zhihu]
github:YadiraF-GAN[github]
Resources and Implementations of Generative Adversarial Nets: GAN, DCGAN, WGAN, CGAN, InfoGAN
github:hindupuravinash-the-gan-zoo[github] A list of all named GANs!
github: Zhangqianhui-AdversarialNetsPapers[github] The classical paper list with code about generative adversarial nets
GAN万字长文综述（中文·下载）[weixin]
Introduction to Turing Leanring and GANs (nice)[Medium]
How to measure GAN performance?[Medium]
Understanding Generative Adversarial Networks (GANs)[Medium]
PyTorch implementations of Generative Adversarial Networks.[github]
视频教程
- 台湾大学李宏毅GAN教程 https://www.bilibili.com/video/av24011528/
- CS236: Deep Generative Models https://github.com/SKKSaikia/CS236_DGM
(2020)A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications [paper]
VAE 教程
- The Variational Autoencoder as a Two-Player Game PartI II III[Medium]
Intuitively Understanding Variational Autoencoders[Medium]
- VAE Careful Walkthrough[Medium]
- From Autoencoder to Beta-VAE[LILI github]
- Review on online VAE[[Review on github]]([https://grigorisg9gr.github.io/machine%20learning/neural%20networks/vae-review/](https://grigorisg9gr.github.io/machine learning/neural networks/vae-review/))
- Understanding Variational Autoencoders(VAEs)[Medium]
- (2019) An Introduction to Variational Autoencoders [paper]
[ ] (CVPR2019 StyleGAN v1)A Style-Based Generator Architecture for Generative Adversarial Networks (StyleGAN) [paper]
[x] (arXiv2019 StyleGAN v2)Analyzing and Improving the Image Quality of StyleGAN [paper]
[ ] (ArXiv 2019 + overview) An Introduciton to Variational Autoencoders [paper]
[ ] [StarGAN v2] StarGAN v2: Diverse Image Synthesis for Multiple Domains [paper] [code]
(ICLR 2020)Overcoming the Disentanglement vs Reconstruction Trade-off via Jacobian Supervision [paper] [github]
(ICLR2020) RealnessGAN [github]

it enables the basic DCGAN architecture to generate realistic images at 10241024 resolution when trained from scratch

Bryce1010 commented 4 years ago

5.4 OCR

[x] [TPAMI 2016] (CRNN) An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition [paper] [crnn torch7/PyTorch]
[ ] 使用TensorFlow Attention API进行OCR文本图像提取 [blog]
[ ] OCR 文本检测综述
[ ] awesome-deep-text-detection-recognition [github]
[ ] OCR project [github]

Bryce1010 commented 4 years ago

5.5 Object Detection

[ ] 目标检测综述 [blog]
[x] [ICLR 2020]Computation Reallocation for Object Detection [paper]
- 对目标检测框架，学习针对不同特征尺度和像素空间位置的计算重分配策略
[ ] (2019 NIPS)DetNAS: Backbone Search for Object Detection [paper]
[ ] (2019 ICCV)CenterNet: Keypoint Triplets for Object Detection [paper]
[ ] (2019 ICCV) ThunderNet: Towards Real-time Generic Object Detection on Mobile Devices
[x] (2019 ICCV Attention) GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond [github]

Non-Local Network + SENet
[ ] (2019 CVPR)NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection [paper]
[ ] (2019 CVPR) HRNet: Deep High-Resolution Representation Learning for Human Pose Estimation [paper]

keypoint detection
[x] (CVPR 2019)Hybrid Task Cascade for Instance Segmentation [paper] [mmdetection]
[ ] (ECCV 2018) CornerNet: Detecting Objects as Paired Keypoints [paper]
[x] (CVPR2018) Cascade R-CNN: Delving Into High Quality Object Detection[paper]
[ ] (2018)YOLOv3: An Incremental Improvement [paper]
(ICLR2018)Mixed Precision Training [paper] [论文笔记]
[ ] (ICCV 2017)Mask R-CNN [paper]
(ICCV 2017) soft-nms : Improving Object Detection With One Line of Code [paper] [论文笔记]
[ ] (ICCV 2017) Feature Pyramid Networks for Object Detection(FPN) [paper]
[ ] (ICCV 2017 Best Student Paper) Focal Loss for Dense Object Detection (RetinaNet) [paper]
[x] (ECCV 2016) SSD: Single Shot MultiBox Detector [paper]
(CVPR2016) OHEM : Training Region-based Object Detectors with Online Hard Example Mining [paper] [论文笔记]
[x] (NIPS 2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks [paper] [pytorch]
[ ] Detectron项目[url]
[ ] 目标检测中不平衡样本综述 [github]
[ ] MMDetection [url]

RPN + Faster-RCNN + Mask-RCNN + Fast-RCNN + RetinaNet + Cascade R-CNN

+Cascade Mask-RCNN + Hybrid Task Cascade + SSD + Group Normalization +

Weight Standardization + Deformable Convolution v2 + Instaboost + Libra R-CNN +

Guided Anchoring + FCOS + FoveaBox + RepPoints + FreeAnchors + Grid R-CNN(plus)+

GHM + GCNet + Mask Scoring RCNN + Train from Scratch + NAS-FPN + ATSS +
[ ] Imbalance Problems in Object Detection [[github]]( Imbalance Problems in Object Detection)

Bryce1010 commented 4 years ago

5.6 Video Analysis

[x] (CVPR 2018) Non-local Neural Networks [paper][code]
[ ] TSM: Temporal Shift Module for Efficient Video Understanding [paper]
[ ] [ArXiv 2019] Depth-Aware Video Frame Interpolation [paper]
[ ] (Skill) PyTorch : Video Dataset (Pytorch载入视频数据的dataset.py) [repo]
[ ] (Skill) Getting started with Videos from OpenCV [url]
PySlowFast is an open source video understanding codebase from FAIR that provides state-of-the-art video classification models [github]

Bryce1010 commented 4 years ago

5.7 Segmentation

DeepLearning Segmentation 总结 [博客]
github: awesome semantic segmentation[github]
Resources of semantic segmantation based on Deep Learning model[github]
2019语义分割综述[weixin]
[x] PointRend: Image Segmentation as Rendering [paper][review]

把语义分割问题以及实例分割问题当做一个图像渲染问题来解决。本质是一个新型的上采样方法，针对物体边缘的分割进行优化，使得其在难以分割的物体边缘部分有更好的表现。
[ ] PyTorch for Semantic Segmentation [github]

Bryce1010 commented 4 years ago

5.8 Graphic Neural Network

[ ] awesome graph classification[github]
[ ] Must-read papers and continuous track on Graph Neural Network[github]
[ ] 【图神经网络：模型与应用】《Graph Neural Networks: Models and Applications(AAAI 2020 Tutorial)》 [ppt]

5.9 Auto MachineLearning

Survey on Automated Machine Learning:[paper]
Auto Machine Learning 学习笔记[weixin]
AutoML: A Sruvey of the State-of-the-Art[paper]
Attention[paper]
Everything you need to know about AutoML and Neural Architecture Search[Medium]
LITERATURE ON NEURAL ARCHITECTURE SEARCH[url]
Awesome NAS[github]
Awesome ML Model Compression[github]
awesome-AutoML-and-Lightweight-Models[github]
[ArXiv 2019] Network Pruning via Transformable Architecture Search [paper]

5.10 Meta Learning

[ ] Learning to Learn [berkeley bari]
[ ] Stanford Deep Multi-Task and Meta Learning[weixin]

5.11 3D Object

Review: deep learning on 3D point clouds [paper]
PyTorch3D is FAIR's library of reusable components for deep learning with 3D data [github]

Bryce1010 commented 4 years ago

5.12 Unsupervised Learning

[ ] 伯克利课程：深度无监督学习(2020) [course homepage]

5.13 Few-shot Learning

Awesome few-shot from Duan[url]
Awesome few-shot from bryce1010[url]
[ ] Few-shot Classification 综述 [bryce1010]
[ ] Few-shot Learning detection综述 [bryce1010]
(ArXiv 2020)Continual Local Replacement for Few-shot Image Recognition [paper] [github]

Bryce1010 commented 4 years ago

6. 强化学习

英文教程：A Free course in Deep Reinforcemet Learning form beginner to expert[github]
综述论文140页：An Introduction to Deep Reinforcement Learning[paper]
增强学习笔记[zhihu]
Reinforcement Learning: An Introduction (Richard S. Sutton and Andrew G. Barto) (http://incompleteideas.net/book/the-book-2nd.html)
Reinforcement Learning - David Silver[B站地址(中文字幕)] [youtube] [课程PPT] [课程笔记]
李宏毅《深度强化学习》 [B站地址(中文字幕)] [课程原地址] [课程PP] [课程笔记]
[ ] A(Long) Peek into Reinforcement Learning [LiLi github]
[ ] Deep Reinforcement Learning Algorithms with PyTorch [github]
[ ] Deep Reinforcement LAB [github]
[ ] 强化学习开源资源大列表：框架、环境和数据(任务) [google sheets]
[ ] 【deeppavlov.ai的深度强化学习进阶课程】“Course: Advanced Topics in Deep Reinforcement learning | deeppavlov.ai [url]
[ ] (2017) Deep Reinforcement Learning: An Overview [paper]

Bryce1010 commented 4 years ago

7. 自然语言处理（NLP）

李纪为-出入NLP领域的一些小建议[zhihu]
斯坦福2019深度学习NLP课程视频+PPT+参考资料+优秀项目 [zhihu]
Overview of Modern Deep Learning Techniques Applied to Natural Language Processing[github]
深度学习之自然语言处理斯坦福大学CS224n课程集训营[github]
邱锡鹏：五个练习上手深度学习 [Medium]
CS224n 斯坦福深度自然语言处理课 [2019版课程主页] [B站英文字幕版]
[ ] CS224n 2019一周通关攻略 [zhihu]
[ ] 自然语言NLP基本概念大全 [blog]
[ ] NLP 中文综述整理[blog]
[ ] NLP Paper Summary [github]
[ ] [XLNet] XLNet: Generalized Autoregressive Pretraining for Language Understanding [paper] [code]
[x] Understanding BERT [medium]
Pytorch Transformers [github]

BERT, GPT, GPT-2, Transformer-XL, XLNet, et.
Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch [github]
[ ] Papers & presentations from Hugging Face's weekly science day [github]
[ ] an Open Course Platform for Stanford CS224n (2020 Winter) [github]
[ ] Google QUEST Q&A Labeling 1st place solution with code [kaggle]

Bryce1010 commented 4 years ago

Bryce1010 / DeepLearning-Project