kaizen-ai / kaizenflow

KaizenFlow is a framework for Bayesian reasoning and AI/ML stream computing
GNU General Public License v3.0
105 stars 74 forks source link

Draw figures for research paper #772

Open sonaalKant opened 2 months ago

sonaalKant commented 2 months ago

Resources available:

FYI @gpsaggese

gpsaggese commented 2 months ago

1) Yes, let's pick a tool first. Personally, my favorite is Lucidchart. You can use the free version and then we can get a license if we like it 2) Let's do quick iterations, so you can send us a quick draft so we can converge more quickly

Vedanshu7 commented 2 months ago

@sonaalKant , @gpsaggese This is the link for the the architecture : https://lucid.app/lucidchart/3e256df2-c19f-4b9b-8478-c13b6f27df83/edit?invitationId=inv_8acc0a33-1c44-4001-bed7-0f6305d98d28

And for the second chart, Is this design good? image

gpsaggese commented 2 months ago

They are going in the right direction @Vedanshu7

@sonaalKant, how can we provide better specs? Should we draw on a piece of paper and then take a picture?

The specs for the first one were more like: "read our paper and do some plots explaining the flow, using the style in the fig", instead of converting the pic to LucidChart

The second one is an interesting direction.

@Vedanshu7 / @tkpratardan based on the draft of the paper, anything else you think it can help explain the flow graphically?

FYI @samarth9008

Vedanshu7 commented 2 months ago

@gpsaggese I have tried to make the flow:

image

let me know if i am wrong or in which part i need to refine.

Vedanshu7 commented 2 months ago

image

gpsaggese commented 2 months ago

Interesting. Let's try to use tools that allow us to create these plots automatically, e.g., https://mermaid.js.org/ or PlantUML We can get someone on fiverr to make the plots look better, if really needed.

Using draw.io is the last resort when we can't use those tools.

Vedanshu7 commented 2 months ago

The flow of the architecture Untitled diagram-2024-04-18-185937

LLM vs KG Untitled diagram-2024-04-18-182957

Drew in Mermaid

gpsaggese commented 1 month ago

Good stuff. Can you also post the source code?

Vedanshu7 commented 1 month ago

``` mermaid

---
config:
  theme: dark
  themeVariables:
    primaryColor: '#FFF'
    secondaryColor: '#FFF'
    tertiaryColor: '#000'
    primaryFontFamily: Arial
    fontSize: 20
---

flowchart LR
    sources["Knowledge Sources<br>(Web, Databases,<br>Experts, etc.)"] -- Data --> kgc["Knowledge Graph<br>Construction"]
    query["User Query"] -- Query --> qpkge["Query Parsing &amp;<br>KG Extraction"]
    dskg["Domain-Specific<br>Knowledge Graph"] -- Data --> qpkge
    kgc -- Data --> dskg
    qpkge -- Extracted KG --> kgsub["Relevant KG<br>Subset"]
    kgsub -- Data --> bnc["Bayesian Network<br>Conversion"]
    bnc -- Model --> rrd["Retrieve<br>Relevant Data"]
    rrd -- Data --> eval["Evaluate BN Model<br>(e.g., KaizenFlow)"] & preproc["Data Preprocessing<br>&amp; Feature Engineering"]
    eval -- Response --> resp["Query Response<br>(with uncertainty<br>estimates)"]
    ext_data["External Data Sources<br>(APIs, Streams, etc.)"] -- Data --> eval
    preproc -- Model --> training["Model Training &amp;<br>Hyperparameter Tuning"]
    training -- Model --> eval
    classDef preproc fill:#DCDCDC,stroke:#808080
    classDef training fill:#DCDCDC,stroke:#808080
    classDef ext_data fill:#DCDCDC,stroke:#808080
    style sources stroke-width:2px,stroke-dasharray: 0
    style preproc stroke-width:1px,stroke-dasharray: 0
Vedanshu7 commented 1 month ago

``` mermaid

%%{init: {'theme': 'dark', 'themeVariables': { 'fontFamily': 'Arial, sans-serif', 'fontSize': '16px'}}}%%

graph LR
  %% Large Language Models Section
  subgraph LLM [Large Language Models]
    direction TB
    LLM_A[Large Language Models]
    LLM_Adv[Advantages_LLM:advantage]
    LLM_Lim[Limitations_LLM:limitation]

    LLM_A -->|General knowledge| LLM_Adv
    LLM_A -->|Language processing| LLM_Adv
    LLM_A -->|Generalizability| LLM_Adv
    LLM_A -->|Black-box: lack of interpretability| LLM_Lim
    LLM_A -->|Implicit knowledge| LLM_Lim
    LLM_A -->|Hallucination| LLM_Lim
    LLM_A -->|Indecisiveness| LLM_Lim
    LLM_A -->|Lack of domain-specific knowledge| LLM_Lim
    LLM_A -->|Lack of new knowledge| LLM_Lim
  end

  %% Knowledge Graphs Section
  subgraph KG [Knowledge Graphs]
    direction TB
    KG_A[Knowledge Graphs]
    KG_Adv[Advantages_KG:advantage]
    KG_Lim[Limitations_KG:limitation]

    KG_A -->|Accuracy| KG_Adv
    KG_A -->|Decisiveness| KG_Adv
    KG_A -->|Interpretability| KG_Adv
    KG_A -->|Domain-specific knowledge| KG_Adv
    KG_A -->|Evolving knowledge| KG_Adv
    KG_A -->|Incomplete knowledge| KG_Lim
    KG_A -->|Lack language understanding| KG_Lim
    KG_A -->|Unseen facts| KG_Lim
  end