search

kiddyboots216 / lottery-ticket-adaptation

Lottery Ticket Adaptation
Apache License 2.0
37 stars 4 forks source link

Steps to replicate the results given in Table 3 and 4 #1

Closed au-revoir closed 2 months ago

au-revoir commented 4 months ago

Hi, can you provide the steps, settings and dataset that can be used to replicate the results of Lotto method given in Table 3 and 4? Is the instruction following dataset among https://huggingface.co/datasets/TRI-ML/dpo-rlaif-data/tree/main? Where can I find the Reasoning and Summarization datasets used here?

If I understand correctly, I need to run python -u train_single_gpu.py do_first_eval=False \ mask_path=../masks/$mask_dir/${sparsity_ratio}_mask.pt \ loss=sft \ model=${model} \ model.archive=${model_archive} \ datasets=[${dataset_name}] \ exp_name=${exp_name} \ eval_batch_size=16 \ sample_during_eval=false \ lr=$lr \ trainer=$trainer_type \ activation_checkpointing=True \ data_fraction=$data_fraction \ save_every=epoch_$n_epochs \ eval_every=100000 \ n_epochs=$n_epochs \ batch_size=$batch_size \ gradient_accumulation_steps=1 \ model.fsdp_policy_mp=bfloat16 \ fsdp_port=${MASTER_PORT} \ optimizer=RMSprop \ grad_norm_strategy=even \ max_grad_norm=$grad_norm followed by the rest of the commands under rlaif. In the above command, what are the values I need to use for lr, data_fraction, n_epochs (should be 3 for Table3 and 1 for Table4?), batch_size, grad_norm and dataset_name? Should I follow the exact values used in https://github.com/kiddyboots216/lottery-ticket-adaptation/blob/main/rlaif/scripts/train_single_gpu.sh?

Thanks!

kiddyboots216 commented 4 months ago

Hi Govind,

Thanks for raising this issue. I will create a PR to add an e2e LoTTO example using datasets where it is more accessible to evaluate.

au-revoir commented 4 months ago

That will be great! Thanks.

au-revoir commented 3 months ago

Hi Is there any update on this?

kiddyboots216 commented 3 months ago

Hi Govind,

I should be able to push a reproduction script tomorrow that has a number of baseline and methods with hyperparameters and expected results. Thank you very much for your patience!

Thanks -Ashwinee

On Fri, Aug 2, 2024 at 10:21 PM Govind K Gangadhar @.***> wrote:

Reopened #1 https://github.com/kiddyboots216/lottery-ticket-adaptation/issues/1.

— Reply to this email directly, view it on GitHub https://github.com/kiddyboots216/lottery-ticket-adaptation/issues/1#event-13747946020, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFM2FYL3L2EZROJEHAKHZWDZPQ5D3AVCNFSM6AAAAABKIMOTCWVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTG42DOOJUGYYDEMA . You are receiving this because you commented.Message ID: <kiddyboots216/lottery-ticket-adaptation/issue/1/issue_event/13747946020@ github.com>

kiddyboots216 commented 3 months ago

Hello @au-revoir I have uploaded a long script https://github.com/kiddyboots216/lottery-ticket-adaptation/blob/main/rlaif/scripts/continual_learning.sh that should walk through a reproduction. Here are the expected results;

Method Task A (GSM8k) Task B (Commonsense)
FFT (baseline) 2.3 86.4
FFT (alternating freeze) 2.4 86.15
LoRA (baseline) 4.2 86.3
LoRA (alternating freeze) 47.26 84.9
LoTTO (ours) 57.81 85.12

I elected to do this baseline with GSM8k->Commonsense because the instruction following results in the main paper all require AlpacaEval2.0. Steps are very similar, for that you can see the instructions in the rlaif README.

kiddyboots216 commented 3 months ago

Accidentally closed this issue, sorry -we can leave this open for any questions that you have on reproducing the results in the paper.

au-revoir commented 3 months ago

Thank you for providing details and the scripts to run them. I updated the environment variables to mine but there could be some I missed. There are some missing files: rlaif/modify_yaml.py, No module named 'trainers', rlaif/cleanup.py I tried running them using bash scripts/continual_learning.sh but ran into the following error trace:


expr: syntax error: missing argument after ‘+’
Could not override 'flip_mask'.
To append to your config use +flip_mask=false
Key 'flip_mask' is not in struct
    full_key: flip_mask
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
expr: syntax error: missing argument after ‘+’
Could not override 'flip_mask'.
To append to your config use +flip_mask=false
Key 'flip_mask' is not in struct
    full_key: flip_mask
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/data/local/gg676/continual_learning/LTA/rlaif-cache/commonsense_/gsm8k_mistralai/Mistral-7B-v0.1_1_5e-7_32_1_none_0.0/epoch-3/_1_5e-7_32_1_none_0.0/epoch-1/'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 368, in <module>
    model, tokenizer = load_model_tokenizer(args)
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 267, in load_model_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(args.model)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 767, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 600, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/data/local/gg676/continual_learning/LTA/rlaif-cache/commonsense_/gsm8k_mistralai/Mistral-7B-v0.1_1_5e-7_32_1_none_0.0/epoch-3/_1_5e-7_32_1_none_0.0/epoch-1/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
expr: syntax error: missing argument after ‘+’
Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/train_single_gpu_lora.py", line 11, in <module>
    import trainers
ModuleNotFoundError: No module named 'trainers'
Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/convert_policy_to_hf_lora.py", line 17, in <module>
    state_dict = torch.load(args.policy_path, map_location='cpu')
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/torch/serialization.py", line 998, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/torch/serialization.py", line 445, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/torch/serialization.py", line 426, in __init__
    super().__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/policy.pt'
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 368, in <module>
    model, tokenizer = load_model_tokenizer(args)
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 267, in load_model_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(args.model)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 767, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 600, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/generate_samples.py", line 188, in <module>
    main()
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/generate_samples.py", line 52, in main
    policy = transformers.AutoModelForCausalLM.from_pretrained(model_path, device_map='balanced')
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
python: can't open file '/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/cleanup.py': [Errno 2] No such file or directory
expr: syntax error: missing argument after ‘+’
Could not override 'flip_mask'.
To append to your config use +flip_mask=false
Key 'flip_mask' is not in struct
    full_key: flip_mask
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/data/local/gg676/continual_learning/LTA/rlaif-cache/commonsense_/gsm8k_mistralai/Mistral-7B-v0.1_1_5e-7_32_1_none_0.0/epoch-3/_1_5e-7_32_1_none_0.0/epoch-1/'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 368, in <module>
    model, tokenizer = load_model_tokenizer(args)
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 267, in load_model_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(args.model)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 767, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 600, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/data/local/gg676/continual_learning/LTA/rlaif-cache/commonsense_/gsm8k_mistralai/Mistral-7B-v0.1_1_5e-7_32_1_none_0.0/epoch-3/_1_5e-7_32_1_none_0.0/epoch-1/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
expr: syntax error: missing argument after ‘+’
Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/train_single_gpu_lora.py", line 11, in <module>
    import trainers
ModuleNotFoundError: No module named 'trainers'
Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/convert_policy_to_hf_lora.py", line 17, in <module>
    state_dict = torch.load(args.policy_path, map_location='cpu')
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/torch/serialization.py", line 998, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/torch/serialization.py", line 445, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/torch/serialization.py", line 426, in __init__
    super().__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/policy.pt'
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 368, in <module>
    model, tokenizer = load_model_tokenizer(args)
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 267, in load_model_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(args.model)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 767, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 600, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/generate_samples.py", line 188, in <module>
    main()
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/generate_samples.py", line 52, in main
    policy = transformers.AutoModelForCausalLM.from_pretrained(model_path, device_map='balanced')
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/data/local/gg676/continual_learning/LTA/rlaif-cache/gsm8k_mistralai/Mistral-7B-v0.1_1_1e-4_32_1.0_none_0.0_LORA_8_32//epoch-3/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
python: can't open file '/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/cleanup.py': [Errno 2] No such file or directory
expr: syntax error: missing argument after ‘+’
Could not override 'flip_mask'.
To append to your config use +flip_mask=false
Key 'flip_mask' is not in struct
    full_key: flip_mask
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/data/local/gg676/continual_learning/LTA/rlaif-cache/commonsense_/gsm8k_mistralai/Mistral-7B-v0.1_1_5e-7_32_1_none_0.0/epoch-3/_1_5e-7_32_1_none_0.0/epoch-1/'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 368, in <module>
    model, tokenizer = load_model_tokenizer(args)
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 267, in load_model_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(args.model)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 767, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 600, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/data/local/gg676/continual_learning/LTA/rlaif-cache/commonsense_/gsm8k_mistralai/Mistral-7B-v0.1_1_5e-7_32_1_none_0.0/epoch-3/_1_5e-7_32_1_none_0.0/epoch-1/'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
python: can't open file '/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/modify_yaml.py': [Errno 2] No such file or directory
scripts/make_diffs_and_masks.sh: line 19: mergekit-yaml: command not found
Namespace(merge_path='gsm8k-mistral-1', sparsities=[99])
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/scratch/gpfs/ashwinee/alignment-durability/output-merges/gsm8k-mistral-1'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/save_mask.py", line 364, in <module>
    model = transformers.AutoModelForCausalLM.from_pretrained(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/scratch/gpfs/ashwinee/alignment-durability/output-merges/gsm8k-mistral-1'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
expr: syntax error: missing argument after ‘+’
Could not override 'flip_mask'.
To append to your config use +flip_mask=false
Key 'flip_mask' is not in struct
    full_key: flip_mask
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
expr: syntax error: missing argument after ‘+’
Could not override 'flip_mask'.
To append to your config use +flip_mask=true
Key 'flip_mask' is not in struct
    full_key: flip_mask
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 160, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: ''.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 368, in <module>
    model, tokenizer = load_model_tokenizer(args)
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 267, in load_model_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(args.model)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 767, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 600, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: ''. Please provide either the path to a local folder or the repo_id of a model on the Hub.
python: can't open file '/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/modify_yaml.py': [Errno 2] No such file or directory
scripts/make_diffs_and_masks.sh: line 19: mergekit-yaml: command not found
Namespace(merge_path='gsm8k-commonsense-mistral-1', sparsities=[99])
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/scratch/gpfs/ashwinee/alignment-durability/output-merges/gsm8k-commonsense-mistral-1'. Use `repo_type` argument if needed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/save_mask.py", line 364, in <module>
    model = transformers.AutoModelForCausalLM.from_pretrained(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: '/scratch/gpfs/ashwinee/alignment-durability/output-merges/gsm8k-commonsense-mistral-1'. Please provide either the path to a local folder or the repo_id of a model on the Hub.
expr: syntax error: missing argument after ‘+’
Could not override 'flip_mask'.
To append to your config use +flip_mask=false
Key 'flip_mask' is not in struct
    full_key: flip_mask
    object_type=dict

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Traceback (most recent call last):
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 398, in cached_file
    resolved_file = hf_hub_download(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
    return f(*args, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
    validate_repo_id(arg_value)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 160, in validate_repo_id
    raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: ''.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 368, in <module>
    model, tokenizer = load_model_tokenizer(args)
  File "/common/home/gg676/continual_learning/lottery-ticket-adaptation/rlaif/eval_model_all.py", line 267, in load_model_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(args.model)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 767, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 600, in get_tokenizer_config
    resolved_config_file = cached_file(
  File "/common/home/gg676/anaconda3/envs/llama/lib/python3.9/site-packages/transformers/utils/hub.py", line 462, in cached_file
    raise EnvironmentError(
OSError: Incorrect path_or_model_id: ''. Please provide either the path to a local folder or the repo_id of a model on the Hub.```
kiddyboots216 commented 3 months ago

Thanks Govind. Let me set up the code on a new machine and go through it to make sure things are working. I know where most of the errors are coming from. I really appreciate your patience.

kiddyboots216 commented 3 months ago

Hi @au-revoir I've pushed some bugfixes, specifically:

This should take care of a number of the errors. However, one error that is outside the scope of these two is the error that says "mergekit-yaml is not a recognized command". For this, you need to install my fork of mergekit from within this library.

au-revoir commented 3 months ago

Thank you. I got held up with some work. I will try with the latest changes as soon as I can.

au-revoir commented 3 months ago

Where can I get the commonsense dataset used here?

kiddyboots216 commented 3 months ago

I got it from here https://github.com/NVlabs/DoRA/tree/main/commonsense_reasoning

Thanks -Ashwinee

On Mon, Aug 26, 2024 at 7:50 PM Govind K Gangadhar @.***> wrote:

Where can I get the commonsense dataset used here?

— Reply to this email directly, view it on GitHub https://github.com/kiddyboots216/lottery-ticket-adaptation/issues/1#issuecomment-2311465463, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFM2FYLKTP7XMT35HUPOZPLZTPSNTAVCNFSM6AAAAABKIMOTCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJRGQ3DKNBWGM . You are receiving this because you modified the open/close state.Message ID: @.*** com>

au-revoir commented 3 months ago

I ran the code in continual_learning.sh till line 25. I had uncommented lines 15 and 21 to first fully fine-tune mistral 7b on first GSM8K followed by commonsense (correct me if I am wrong). However, after gsm8k, commonsense evaluation on line 25 gives the following results:

Screenshot 2024-08-29 at 11 18 21 AM

These look different than the above one you had posted. GSM8K is only 1.7% in my run vs 2.3%. Also, for the commonsense is the final score you have got, the average of the 8 datasets- boolq, piqa, social_i_qa, ARC-C, ARC-Easy, obqa, hellaswag, winogrande? If it's the average then it's giving 85.2. I haven't changed any hyperparameters. Can you let me know if this is correct or I am doing something wrong?

kiddyboots216 commented 3 months ago

If it's within 1% then I would just chalk it up to variance. There is definitely significant variance across runs. So in the main paper we'll typically report the mean/std across 5 runs. For these results that I quickly wrote this script for, I don't have the mean/std.

au-revoir commented 2 months ago

I skipped the LoRA finetuning and jumped directly to mask creation. It is missing modify_yaml.py. Could you please add this file as well? Thanksgt

kiddyboots216 commented 2 months ago

Added modify_yaml.py and task_vector.yaml! Thanks.

au-revoir commented 2 months ago

Thank you for your patience and helpful responses. I am closing this issue.