Image and Imaging Outline

[bellabf]

Hello everyone, We finally have an outline for the Image and Imaging chapter.

1. What is an image? [image.mdx]

• [ ] Definition of Image
• [ ] Difference between Image and Video
• [ ] Image-specific challenges (i.e.comparing images to other types of data)

2. Fundamentals of Imaging and its technical aspects of images [imaging.mdx]

• [ ] Image formation: Bayer Grid / Filter, Digital Image Sensing, Color space representations
• [ ] Image representation: Type of image representations, RGB's, image as the matrix,
• [ ] Output of images: Image resolution, Pixel/Voxel, Dimensions Channels, Dynamic range, Bit depth, File Compression, Image formats, Metadata

3. How we imaged everything/When in doubt, image it. [extension-image.mdx]

• [ ] Challenges in imaging real-life
• [ ] Imaging everything (how we extend sensors to make images from things that are not usually thought of as images)
• [ ] Perspective on what we obtained from this and its impact on us (i.e. we have imaged outer space and electrons; we have an MRI machine capable of imaging inside of us without ever touching 🤯)
• [ ] Example(s) of how image-acquired characteristics of images come from what types of sensors they record, challenges/difficulties in computer vision change according to the image/data, not all data is RGB

Our biggest concern was the possible overlap of our content with the “Feature Detection/Feature Extraction chapter” content, especially regarding image transformations, filters, and convolution. We opt to left it out, considering that the proposed outline for Feature Detection.

Let us know what you think!

Team members - @bellabf, @seshu-pavan, @mfidabel @froestiago, @froestiago bcc - @johko @lunarflu @merveenoyan @MKhalusova

Best, Image and Imaging team

[MKhalusova]

The outline looks really good to me, thank you for working on this! Lots of good fundamental information on images to build up learners' understanding. For the "Imagine everything" feel free (if you'd like) to link an example that we have in the Audio course on audio classification using spectrograms to encourage learners to think out of the box on what else can be viewed as an image.

[ATaylorAerospace]

@bellabf This is a fantastic outline!

In between Section 2 and 3 what do you think about going into more detail on this topic?

• Image Processing Techniques:

• [ ] Basic Operations: Including filtering, histogram manipulation, thresholding, and morphing ops for example.

• [ ] Image Enhancement and Restoration: Techniques for improving image visibility and reducing noise.

• [ ] Edge Detection: Discuss algorithms like Sobel, Cann, Prewitt etc for identifying edges in images.

[bellabf]

@bellabf This is a fantastic outline!

In between Section 2 and 3 what do you think about going into more detail on this topic?

• Image Processing Techniques:
• [ ] Basic Operations: Including filtering, histogram manipulation, thresholding, and morphing ops for example.
• [ ] Image Enhancement and Restoration: Techniques for improving image visibility and reducing noise.
• [ ] Edge Detection: Discuss algorithms like Sobel, Cann, Prewitt etc for identifying edges in images.

Hi! Thank you for the feedback. I think it would be great to have image processing techniques in the course content. It would also ease the transition into the feature extraction chapter. I think we can first add enhancement and restoration within section two, and go into more detail in section three (introduce the concept with a simple example and then add different examples and go into more depth as we discuss other imaging technique). As for basic operations and edge detection, I think should be after section 3. By adding into afterward we can show output of what histograms, thresholds techniques, edge detection look like in different types of images and make a stronger overall connection to the other points made within this chapter. I think it would make a more impactful delivery. What do you think?

[merveenoyan]

This is very cool! I wouldn't like this section to be very overwhelming content-wise though as computer vision with ML itself is the opposite of traditional image processing where one e.g. applies kernels manually, does histogram matching or contrast stretching and so forth. We could mention these briefly IMHO. Maybe we could also mention how one uses these as a layer of pre-processing to train a model.

[bellabf]

This is very cool! I wouldn't like this section to be very overwhelming content-wise though as computer vision with ML itself is the opposite of traditional image processing where one e.g. applies kernels manually, does histogram matching or contrast stretching and so forth. We could mention these briefly IMHO. Maybe we could also mention how one uses these as a layer of pre-processing to train a model.

I see your point. Completely agree on it not being too overwhelming. Personally, I think there is a lot of complexity here. For instance, you can do conventional machine learning to do segmentation on engineered features based on pre-processing techniques.. Or you can do deep learning (i.e autoencoders) and use that for registering/rendering afterward. That is part of the reason why we decided "draw a line" between pre-processing and computer vision when we introduce what computer vision is (the outline it was proposed here).

So, if we assume we will be going with a clear definition then we use this part to highlight with examples like downsampling and the use of geometric transformation for data augmentation.

To recap, we will add a section 4 where the first part is using pre-processing techniques to compare images types (i.e. histograms, thresholds techniques, edge detection look like in image coming from different conditions), and the second part is common operations before training (downsampling and data augmentation).

[johko]

Hey, sorry for my late reply.

I also really like the content you came up with! :hugs:

Similar to @merveenoyan, I would say don't go too deep into actual methods, but of course it is good for the course participants to have an intuition for what edge detection does, as that is fundamental for feature extraction and is a good connection to the next chapter.

If you wanna go wild you can also add a notebook with more of the classical methods and just refer to it in the .mdx file, so curious people can have a look. But I would see that as low priority.

[lunarflu]

Overall looks good! Building off what others have said, I think you could start now with tackling individual sections + documents with individual PRs (it's fine to have them as rough drafts) 🚀 WDYT?