search

jwzhanggy / Graph_Toolformer

MIT License
241 stars 26 forks source link

readme

Graph-Toolformer

Note

  • 8-bit tensor cores are not supported on the CPU. bitsandbytes can be run on 8-bit tensor core-supported hardware, which are Turing and Ampere GPUs (RTX 20s, RTX 30s, RTX 40s, A40-A100, T4+).

framework

Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT

Paper URL at IFMLab: http://www.ifmlab.org/files/paper/graph_toolformer.pdf

Paper URL at arxiv: https://arxiv.org/pdf/2304.11116.pdf

Paper description in Chinese: 文章中文介绍

References

@article{Zhang2023GraphToolFormerTE,
  title={Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT},
  author={Jiawei Zhang},
  journal={ArXiv},
  year={2023},
  volume={abs/2304.11116}
}

Organization of the project source code, data and model checkpoints

Source code

This project source code is divided into two directories:

Dataset

The datasets used in this paper inclode both the the generated graph reasoning prompt datasets, and the raw graph benchmark datasets

Model checkpoints

The pre-trained/fine-tuned model checkpoints released by this project also have two parts

How to play with the code?

Environment setup

First of all, as mentioned above, please set up the environment for running the code. We recommend you create the code environment with the environment.yml file shared from us. You can create the environment with the following command

conda env create -f environment.yml

For the packages cannot be installed with the above conda command, you may consider to install via pip.

Play with LLM_Tuning code

After downloading the LLM_Tuning directory, installing the conda environment (see this file), you can just go ahead to run the code with the following command to start the LLMs fine-tuning with the prompt datasets:

Note

1. To avoid getting OOM, depending on your machine (GPU memory capacty), please also adjust the batch_size and max_length parameters accordingly.

2. For 8bit models, it seems some CPU will not support the mixed precision computation. We recommend you use GPU instead of CPU for running the code.

3. If you plan to use the fine-tuned LLM checkpoint for graph reasoning, please (1) replace the fine-tuned checkpoints in the downloaded Graph_Toolformer_Package/koala/language_models/gptj_8bit/local_data/finetuned_model/graph_toolformer_GPTJ directory of the Graph_Toolformer_Package for graph reasoning demos, (2) also remember to change the checkpoint names to graph_toolformer_GPTJ before pasting it to the koala folder, so the demo framework will load the checkpoint.

python3 gtoolformer_gptj_script.py

For more information, you can also refer to the README.md provided in the LLM_Tuning directory as well.

Play with Graph_Toolformer Demo code

After downloading the Graph_Toolformer_Package directory, also downloading the graph_datasets.zip, koala/graph_models.zip and koala/language_models.zip, installing the conda environment, you can just go ahead to run the code with the following command to start the demo

python3 ./src/Framework_Graph_Toolformer.py

You can type in inputs which are similar to the prompt inputs, and the model will carry out the reasoning task and return the otuputs. The reasoning process will call both the LLMs and GNN models, so generating the output will take some time. The GNN models are pre-trained according to the previous papers, and the LLMs is fine-tuned with the code in the LLM_Tunign directory based on the prompt datasets.

By changing the "if 0" to "if 1" in the bottom main function of Framework_Graph_Toolformer.py, you can try different reasoning tasks.

For more information, you can also refer to the README.md provided in the Graph_Toolformer_Package directory as well.

🔴 🟠 ⚫ ⚪ 🟣 🟢 🟡 🔵

Tasks to be done