search

ZGC-LLM-Safety / TrafficLLM

The repository of TrafficLLM, a universal LLM adaptation framework to learn robust traffic representation for all open-sourced LLM in real-world scenarios and enhance the generalization across diverse traffic analysis tasks.
121 stars 16 forks source link

Inquiry About Missing Modules in TrafficLLM Project #4

Closed ReamonYim closed 1 month ago

ReamonYim commented 2 months ago

Dear author,

Thank you for your prompt and helpful response earlier. I followed your advice and tried to directly train the model using the provided datasets. However, when I executed the following command:

PRE_SEQ_LEN=128
LR=2e-2
NUM_GPUS=1
export CUDA_VISIBLE_DEVICES=1

torchrun --standalone --nnodes=1 --nproc-per-node=$NUM_GPUS main.py \
    --do_train \
    --train_file ../datasets/instructions/instructions.json \
    --validation_file ../datasets/instructions/instructions.json \
    --preprocessing_num_workers 10 \
    --prompt_column instruction \
    --response_column output \
    --overwrite_cache \
    --cache_dir /cache \
    --model_name_or_path ../models/chatglm2/chatglm2-6b \
    --output_dir ../models/chatglm2/peft/instruction \
    --overwrite_output_dir \
    --max_source_length 1024 \
    --max_target_length 32 \
    --per_device_train_batch_size 1 \
    --per_device_eval_batch_size 1 \
    --gradient_accumulation_steps 16 \
    --predict_with_generate \
    --max_steps 20000 \
    --logging_steps 10 \
    --save_steps 4000 \
    --learning_rate $LR \
    --pre_seq_len $PRE_SEQ_LEN

I encountered the following error:

ModuleNotFoundError: No module named 'trainer_seq2seq'

After inspecting main.py, I found that there were missing imports for both trainer_seq2seq and arguments. It appears that the trainer_seq2seq.py and arguments.py files are not present in the repository. 1727691293023

Could you kindly confirm if these files were mistakenly omitted during upload? If so, would you be able to provide these missing files or direct me to the correct location to obtain them?

Thank you very much for your assistance!

Best regards, Reamon

CuiTianyu961030 commented 1 month ago

Many thanks for your bug report! We have uploaded the required modules in the dual-stage-tuning dir. These modules are based on ChatGLM2. If you want to use other LLMs, please refer to their training codes in their released repository.

Thanks again for your valuable feedback.