RuiyuZ
commented
2 years ago Hi @VirajVaitha123,
Thank you for your interest in our package!
Perception and unity do support very high-quality models, and I understand your concern on the gap between real and synthetic data; we have many projects that have testified the improvement on ML model with the assistance of synthetic data. Here are some blog posts and research papers on perception package that provides more information: Unity Perception: Generate Synthetic Data for Computer Vision Boosting computer vision performance with synthetic data Training a performant object detection ML model on synthetic data using Unity computer vision tools
As for your questions:
- Yes, please check out the "Scanning" part in this article Getting started with 3D content for synthetic data
There are a range of technologies for 3D scanning, each suited to different types of objects, quality levels, and budgets. Some notable examples are:
Photogrammetry: Photogrammetry tools such as Agisoft Metashape and Unity’s AR Companion app synthesize a 3D object from a set of photographs taken from different angles. Visit Unity’s photogrammetry page for more on this technique. Pros: Suitable for objects of any size or shape; captures texture and fine detail Cons: Sensitive to lighting conditions and object color; somewhat tedious
Flatbed scanner and 3D modeling: A flatbed scanner can be used to scan different sides of a box-like object and then combined into textures and mapped onto a hand-authored 3D mesh using standard 3D authoring tools. For more, see a video tutorial on this method from our recent grocery object detection project. Pros: Inexpensive, highly accurate textures Cons: Only suitable for flattenable objects; some hand-authoring required
Industrial 3D scanner: Specialized devices for 3D scanning using laser or structured light produce high geometric accuracy. This approach is ideal for detecting mechanical parts and complex electronic equipment. Pros: Highly accurate geometry on most objects; fast Cons: Expensive; object sizes are limited
-
For info about high-quality visualization on unity hdrp, I think this page would be helpful: How to set up Unity's High Definition Render Pipeline for high-end visualizations
-
Perception package does support HDRP out of the box. And for your reference: there is a "PerceptionHDRP" project under TestProjects/ that you can open in unity editor directly. PerceptionHDRP project location: https://github.com/Unity-Technologies/com.unity.perception/tree/main/TestProjects/PerceptionHDRP
Please let me know that answers your questions 😃 thank you!
VirajVaitha123
Hi,
I was wondering what's the highest quality 3D models you can use for synthetic data generation. I couldn't see a discussions section so I just posted here hope that's okay.
I love the concept of this tool and it can really drive data centric model development. The only concern I have is the gap between real and synthetic data (which I assume will improve in the future).
The grocery store dataset didn't look to realistic in my opinion, and really want to test the models ability to generalise to real data when using synethic data. If you wanted to have more realistic 3D models what would you suggest?
Many Thanks, Viraj