Custom copy_paste augmentation

[Thatonedudefromouterspace]

Search before asking

• [X] I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Im attempting to make a detection model to detect squirrels. a common problem I've found is that it mistakes humans, specifically their hair, for squirrels. A solution I thought of was to use copy and paste augmentation to take unannotated images of humans and put them next to annotated squirrel images to introduce noise into my dataset so my model learns to distinguish between humans and squirrels more effectively. I opted for this instead of trying to create a dataset of specifically humans and squirrels in the same picture due to time. I cant seem to find anywhere I can add any sort of other images to the copy and paste to introduce this noise I want, does anyone know how to do it? if there is a better approach then what I'm doing that accomplishes the same thing please let me know!

Additional

.

[github-actions[bot]]

👋 Hello @Thatonedudefromouterspace, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@Thatonedudefromouterspace hello! It looks like you're on the right track with trying to improve your model's ability to distinguish between squirrels and humans. Copy-paste augmentation can indeed be a good strategy for this kind of issue. Currently, YOLOv8 does not have built-in support for explicitly inserting unannotated images directly via the augmentation pipeline you described. However, you can create a custom dataset where you manually place unrelated objects (humans in this case) in the training images of squirrels.

Here’s a possible idea: if you have a separate dataset of humans, you could manually overlay these images onto your squirrel images. Below is a simple Python example using PIL to merge images:

from PIL import Image

# Open the foreground (human) and background (squirrel) image
foreground = Image.open('human.jpg')
background = Image.open('squirrel.jpg')

# Resize or transform the foreground
foreground = foreground.resize((100, 100))

# Paste the foreground over the background
background.paste(foreground, (50, 50), foreground)
background.show()

This script assumes you want to resize the human image to 100x100 pixels and paste it at position (50, 50) on the squirrel image. You'd need to adjust the parameters and possibly use more complex transformations depending on your specific needs.

Alternatively, employing a more advanced augmentation library like albumentations, which supports a variety of operations, might simplify this process. This won't integrate directly into the YOLO train process without some custom coding, but it can be employed during dataset preparation.

I hope this helps! Let me know if there’s anything else you’d like to discuss. 🌿🐿️