Accuracy

[deepsea6034625]

Hello. First of all, thanks for releasing this excellent project.

I tried to segment lenses from images. But as you can see here, accuracy is not good. Is there any way to improve the accuracy?

Hope to please help me. Thanks again.

[deepsea6034625]

One more.

I used this code.

import numpy as np
from glasses_detector import GlassesSegmenter

# Initialize full segmentation model of size large
seg_full = GlassesSegmenter(kind='lenses', size='medium') # TODO: change medium to large

# Process the directory of images and save the results in various ways
seg_full.process_dir('images', 'results', format='img', batch_size=6, pbar=False)

[mantasu]

Better Weights

Hi @deepsea920415, unfortunately, there's not much that can be done because the dataset is very small. Based on the data (standalone glasses) you are showing, I tried training a new large model which seems to have slightly improved accuracy.

segmenter = GlassesSegmenter(kind="lenses", size="large", weights="path/to/weights.pth")

Here are two weight files:

• Same lenses dataset, larger model: segmentation-lenses-large.pth
• Fine-tuned on data without people: segmentation-lenses-large-standalone-glasses.pth

Square Inputs

Also, you might want to pad your image from both sides to make sure it is a square:

OPTION 1: preprocess image files

```python import os from PIL import Image from glasses_segmenter import GlassesSegmenter # Both directories must already exist INPUT_DIR = "path/to/non_square" PREPROCESS_DIR = "path/to/square" def make_image_square(image_path): img = Image.open(image_path) width, height = img.size target_size = max(width, height) new_img = Image.new(img.mode, (target_size, target_size), img.getpixel((0,0))) left_padding = (target_size - width) // 2 top_padding = (target_size - height) // 2 new_img.paste(img, (left_padding, top_padding)) return new_img for filename in os.listdir(INPUT_DIR): img = make_image_square(os.path.join(INPUT_DIR, filename)) img.save(os.path.join(PREPROCESS_DIR, filename)) segmenter = GlassesSegmenter(kind="lenses", size="large", weights="path/to/weights.pth") segmenter.process_dir(PREPROCESS_DIR) ```

OPTION 2: extend GlassesSegmenter

```python from glasses_detector import GlassesSegmenter class MyGlassesSegmenter(GlassesSegmenter): @staticmethod def make_image_square(image_path): img = Image.open(image_path) width, height = img.size target_size = max(width, height) new_img = Image.new(img.mode, (target_size, target_size), img.getpixel((0,0))) left_padding = (target_size - width) // 2 top_padding = (target_size - height) // 2 new_img.paste(img, (left_padding, top_padding)) return new_img def predict(self, image_paths, *args, **kwargs): # WARNING: image_paths must be a list of paths! images = [self.make_image_square(path) for path in image_paths] return super().predict(images, *args, **kwargs) segmenter = MyGlassesSegmenter(kind="lenses", size="large", weights="path/to/weights.pth") segmenter.process_dir("path/to/non_square") ```

Further Improvements

To further improve accuracy, here are some further suggestions (they are, however, beyond the scope of this package):

• Use more data, and experiment with various data augmentation techniques
• Use dilation and erosion to smoothen the masks, e.g., close segmentation holes
• Subtract frames segmenter mask from full segmenter mask to get a "helper" mask
• Tune for specific segmenter architecture and hyperparaeters (e.g., using optuna, ray, pytorch lightning packages)
• Use cross-validation because the dataset is very small
• Train an ensemble of models, possibly with varying architectures

[deepsea6034625]

Thank you. I'll test with them.

[deepsea6034625]

The accuracy is improved a lot. But still some bad samples.

But totally, I think overall accuracy is good. If we add some more dataset, the accuracy will be better. Thanks again.

[mantasu]

Yup, training on more data is the best way to go!