Feature Extraction Outline

[johko]

I decided to join the Feature Extraction team and helped prepare an outline for the section. So here it is:

Feature Detection

• What are features?
• What makes a good feature?
• Simple Feature Detection (e.g. Harris Corner Detection, Hough Transform, Shi-Tomasi)

Feature Description

• How can we represent features in data structures?
• What makes a good descriptor (think of different image conditions, rotation, translation, etc.)
• Example algorithm: SIFT & SURF

Feature Matching

• How can we match detected features from image-to-image?
• Examples (Brute Force, FLANN, maybe others)

CNN Features + Visualization

• How do ANN (especially CNNs) detect features?
• What do the features “look like”?
• Some nice visualizations, maybe some DeepDream for fun

Resources: https://docs.opencv.org/4.8.0/db/d27/tutorial_py_table_of_contents_feature2d.html https://towardsdatascience.com/image-feature-extraction-traditional-and-deep-learning-techniques-ccc059195d04 https://fpcv.cs.columbia.edu/

The CNN Features part is probably the most controversial, as it actually talks about CNNs before the CNN chapter. But then again it also is a really interesting topic related to feature extraction. I also didn't find anything about it in the CNN outline, so maybe we can write about it and if needed move it to the CNN section later on.

Notebooks and .mdx files For now we will start writing separate .mdx files for each of the topics, mainly for ease of parallel working, but depending on how extensive they are, we might also merge some of them later on.

We also plan to have two notebooks, one about Feature Extraction, which gives a walkthrough with a complete example and one for CNN visualization part (if we do this here)

Let us know what you think @merveenoyan , @lunarflu :)

[kfahn22]

Thanks for taking the lead on this!

[ATaylorAerospace]

This is a great outline and yes thanks for taking the lead on this.

I had thoughts on a couple of additional high level topics for Feature Detection

• Feature Detection in Different Conditions (Would Address at a high level challenges in various environmental conditions like low light, high contrast, or motion blur.)

• Techniques for feature detection in non-ideal conditions.

[merveenoyan]

Hello @johko very cool outline indeed! I just gave a review to #62 which also has certain traditional imaging techniques for feature extraction (that might overlap a bit) so I'd suggest making sure you're in communication. Moreover, we could also draw a picture that explains why people needed CNNs in the end, where they were using a bunch of different kernels in brute-force manner to extract features, which would be good storytelling while the reader passes to CNNs. I think #49 and #39 focuses a bit on this.

[johko]

This is a great outline and yes thanks for taking the lead on this.

I had thoughts on a couple of additional high level topics for Feature Detection

• Feature Detection in Different Conditions (Would Address at a high level challenges in various environmental conditions like low light, high contrast, or motion blur.)
• Techniques for feature detection in non-ideal conditions.

Good points @ATaylorAerospace , I think I had them in mind (because they are covered in the resources) but not spelled it out here. Will add :slightly_smiling_face:

[johko]

@merveenoyan I agree, I'll be in touch with them.

As for the storytelling - this might be a good way to integrate the CNN feature visualization, giving a good incentive why CNNs are needed and then leading to the next chapter.