DermAI-Viz: AI-Driven Skin Condition Visualization

DermAI-Viz is a pioneering platform that utilizes deep learning to simulate the progression and treatment outcomes of various skin conditions. By generating realistic images of skin rashes, this tool aids dermatologists in diagnosis and treatment planning, and serves as an educational resource for medical students.

Generating skin rash examples using a diffusion model

This model needs more work right now the images look blurry

Generating images of butterflies using a diffusion model

Features

• Visualize Skin Conditions: Generates images showing the progression of skin conditions over time.
• Treatment Outcome Simulation: Helps predict the response of skin conditions to various treatments.
• Supports Diverse Skin Types: Includes data across different skin colors and types to ensure broad applicability.

Technical Stack

• Diffusion Models: Utilizes state-of-the-art generative models for high-quality image synthesis.
• Python: Built using Python with libraries such as TensorFlow and PyTorch for model implementation.
• Web Interface: Easy-to-use web interface for interacting with the model outputs.

Completed:

• Figured out how to train a Diffusion Model on custom data using a simple Unet : Notebook
• Trained a simple model and uploaded it on HuggingFace : DhruvParth/sd-class-skinrash-64
• Trained V2 Model : DhruvParth/ddpm-celebahq-256-fineTuned-skin_rash_v2_12epochs
• Explored Guidance using the CLIP model from OpenAI to generate rash images from text based prompt: Notebook
• Fine Tuned a CLIP model to learn to match Text caption to images: finetuningclip.ipynb | DhruvParth/openai-clip-vit-base-patch32-skinRash-FineTuned | Reference Article

Next Steps:

• Understand training and validation characteristics and figure out if we can train the CLIP model Better
• Further fine tuning the Diffusion model for additional epochs
• Test the pipeline by generating images based on captions

Model:

from diffusers import DDPMPipeline
pipeline = DDPMPipeline.from_pretrained('DhruvParth/ddpm-celebahq-256-fineTuned-skin_rash_v2_12epochs')
image = pipeline().images[0]
image

Contributing

We welcome contributions from the community, including feature enhancements, bug fixes, and documentation improvements. Please fork the repository, make your changes, and submit a pull request.

License

Distributed under the MIT License. See LICENSE for more information.

Acknowledgments

• Collaborators and data providers from dermatological institutions
• Open-source projects and libraries that inspired and supported our development