This repository contains the code of ChartSpark, a novel system that embeds semantic context into chart based on text-to-image generative model. ChartSpark generates pictorial visualizations conditioned on both semantic context conveyed in textual inputs and data information embedded in plain charts. The method is generic for both foreground and background pictorial generation, satisfying the design practices identified from an empirical research into existing pictorial visualizations. We further develop an interactive visual interface that integrates a text analyzer, editing module, and evaluation module to enable users to generate, modify, and assess pictorial visualizations.
gitclone https://github.com/SerendipitysX/ChartSpark.git
cd ChartSpark
conda create --name <environment_name> --fisle requirements.txt
In this work, we benefit from some excellent pretrained models, including WebVectors for seeking the relevant words, DIS for background removal, and diffusers for generating the image containing semantic context.
To use these models, please follow the steps below:
runwayml/stable-diffusion-v1-5
, stabilityai/stable-diffusion-2-depth
, and Fantasy-Studio/Paint-by-Example
. You can download and load them as local files, which can be specified in chartSpeak.py line 37-44
.mask/bg_removal.pth
.theme_extract/
. You will have three files: meta.json
, model.bin
, model.txt
.python chartSpeak.py
see this video for real-world usage:video
npm install
npm run dev
We are glad to hear from you. If you have any questions, please feel free to contact xrakexss@gmail.com or open issues on this repository.
@ARTICLE {xiao2023CharSpark,
author = {S. Xiao and S. Huang and Y. Lin and Y. Ye and W. Zeng},
journal = {IEEE Transactions on Visualization & Computer Graphics},
title = {Let the Chart Spark: Embedding Semantic Context into Chart with Text-to-Image Generative Model},
year = {2024},
volume = {30},
number = {01},
issn = {1941-0506},
pages = {284-294},
doi = {10.1109/TVCG.2023.3326913},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
month = {jan}
}