search

goodbai-nlp / AMRBART

Code for our paper "Graph Pre-training for AMR Parsing and Generation" in ACL2022
MIT License
92 stars 28 forks source link

Using HuggingFace AMRBartTokenizer #21

Open Zoher15 opened 1 year ago

Zoher15 commented 1 year ago

Hi @goodbai-nlp ,

This is great work! Thanks for making your model available on huggingface. Makes things easier.

However, I am not sure I follow the instructions for generating AMRs. I want to simply generate an AMR for a sentence. In your instructions the easiest way to do so seems to be using huggingface:

from transformers import BartForConditionalGeneration
from model_interface.tokenization_bart import AMRBartTokenizer      # We use our own tokenizer to process AMRs

model = BartForConditionalGeneration.from_pretrained("xfbai/AMRBART-large-finetuned-AMR3.0-AMRParsing-v2")
tokenizer = AMRBartTokenizer.from_pretrained("xfbai/AMRBART-large-finetuned-AMR3.0-AMRParsing-v2")

Are you expecting us to install your repo as a python package? If not, how do expect us to import your tokenizer in our scripts from model_interface?

Zoher15 commented 1 year ago

@goodbai-nlp Okay so I have been able to get all the import working without error. I still don't know I can generate an AMR for a sentence like "The cat sat on the mat".

Following the examples on BartForConditionalGeneration: I tried to do the following which did not work:

from transformers import BartForConditionalGeneration
from model_interface.tokenization_bart import AMRBartTokenizer
model = BartForConditionalGeneration.from_pretrained("xfbai/AMRBART-large-v2")
tokenizer = AMRBartTokenizer.from_pretrained("xfbai/AMRBART-large-v2")

# my improvisation:
inputs = tokenizer(["the cat sat on the mat"])
amr = model.generate(inputs)

Would really appreciate help. I am trying to use your model for a downstream task that I wish to submit for EMNLP, and I wish to cite your model for comparative purposes.

Best, Zoher

goodbai-nlp commented 1 year ago

Hi, @Zoher15

Our AMRBART model needs to do preprocessing and postprocessing when transforming a sentence into an AMR graph, thus your code can not work as expected. You can refer to here to parse your own data.

Zoher15 commented 1 year ago

@goodbai-nlp

So the huggingface extensions are unusable??

I did see those instructions. But can you give an intuition of what the preprocessing is doing? It is a lot of steps....

Just to give you context: I am working on a project with AMRs for downstream tasks, and it would help me a lot if there was an easier way to parse AMRs using your method for my own data.

Zoher15 commented 1 year ago

@goodbai-nlp

Also as mentioned before I am trying to use your model for a downstream task and I am working on a deadline for a paper that we need to submit soon. As much as I would really like to cite your work and your method, if I am unable to get it to work by the 6/8/2023 we may have to save the results from your model for later extensions of the paper.

I know you are probably also working on conference deadlines, but just wanted to give you a heads-up about my deadline. Sorry!

Best, Zoher

goodbai-nlp commented 1 year ago

Hi, @Zoher15

The simplest way to use our code is to convert your downstream task data into the jsonl format here, then run

bash inference_amr.sh "xfbai/AMRBART-large-finetuned-AMR3.0-AMRParsing-v2"

If you want to learn more about data preprocessing and postprocessing, you may refer to here and here, respectively.

Zoher15 commented 1 year ago

Hi @goodbai-nlp ,

I just followed your instructions, however:

06/02/2023 17:56:36 - WARNING - __main__ - Process rank: -1, device: cuda:0, n_gpu: 1distributed training: False, 16-bits training: True
06/02/2023 17:56:36 - INFO - __main__ - Training/evaluation parameters Seq2SeqTrainingArguments(
_n_gpu=1,
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-08,
auto_find_batch_size=False,
bf16=False,
bf16_full_eval=False,
data_seed=None,
dataloader_drop_last=False,
dataloader_num_workers=8,
dataloader_pin_memory=True,
ddp_bucket_cap_mb=None,
ddp_find_unused_parameters=False,
debug=[],
deepspeed=None,
disable_tqdm=False,
do_eval=False,
do_predict=True,
do_train=False,
early_stopping=5,
eval_accumulation_steps=None,
eval_dataloader_num_workers=2,
eval_delay=0,
eval_lenpen=1.0,
eval_steps=None,
evaluation_strategy=no,
fp16=True,
fp16_backend=auto,
fp16_full_eval=False,
fp16_opt_level=O1,
fsdp=[],
fsdp_min_num_params=0,
full_determinism=False,
generation_max_length=1024,
generation_num_beams=5,
gradient_accumulation_steps=1,
gradient_checkpointing=False,
greater_is_better=None,
group_by_length=False,
half_precision_backend=cuda_amp,
hub_model_id=None,
hub_private_repo=False,
hub_strategy=every_save,
hub_token=<HUB_TOKEN>,
ignore_data_skip=False,
include_inputs_for_metrics=True,
jit_mode_eval=False,
label_names=None,
label_smoothing_factor=0.0,
learning_rate=5e-05,
length_column_name=length,
load_best_model_at_end=False,
local_rank=-1,
log_level=-1,
log_level_replica=-1,
log_on_each_node=True,
logging_dir=/N/u/zkachwal/BigRed3/AMRBART/fine-tune/outputs/Infer-examples-AMRBART-large-AMRParing-bsz16-lr-1e-5-UnifiedInp/logs,
logging_first_step=False,
logging_nan_inf_filter=True,
logging_steps=500,
logging_strategy=steps,
lr_scheduler_type=linear,
max_grad_norm=1.0,
max_steps=-1,
metric_for_best_model=None,
mp_parameters=,
no_cuda=False,
num_train_epochs=3.0,
optim=adamw_hf,
output_dir=/N/u/zkachwal/BigRed3/AMRBART/fine-tune/outputs/Infer-examples-AMRBART-large-AMRParing-bsz16-lr-1e-5-UnifiedInp,
overwrite_output_dir=True,
past_index=-1,
per_device_eval_batch_size=16,
per_device_train_batch_size=8,
predict_with_generate=True,
prediction_loss_only=False,
push_to_hub=False,
push_to_hub_model_id=None,
push_to_hub_organization=None,
push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
ray_scope=last,
remove_unused_columns=True,
report_to=['tensorboard'],
resume_from_checkpoint=None,
run_name=/N/u/zkachwal/BigRed3/AMRBART/fine-tune/outputs/Infer-examples-AMRBART-large-AMRParing-bsz16-lr-1e-5-UnifiedInp,
save_on_each_node=False,
save_steps=500,
save_strategy=steps,
save_total_limit=None,
seed=42,
sharded_ddp=[],
skip_memory_metrics=True,
smart_init=False,
sortish_sampler=False,
task=text2amr,
tf32=None,
torchdynamo=None,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_ipex=False,
use_legacy_prediction_loop=False,
warmup_ratio=0.0,
warmup_steps=0,
weight_decay=0.0,
xpu_backend=None,
)
Traceback (most recent call last):
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/site-packages/transformers/configuration_utils.py", line 623, in _get_config_dict
    resolved_config_file = cached_path(
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/site-packages/transformers/utils/hub.py", line 284, in cached_path
    output_path = get_from_cache(
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/site-packages/transformers/utils/hub.py", line 486, in get_from_cache
    os.makedirs(cache_dir, exist_ok=True)
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/mnt/nfs-storage'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 506, in <module>
    main()
  File "main.py", line 124, in main
    config = AutoConfig.from_pretrained(
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/site-packages/transformers/models/auto/configuration_auto.py", line 725, in from_pretrained
    config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/site-packages/transformers/configuration_utils.py", line 561, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/N/u/zkachwal/Carbonate/miniconda3/envs/amrbart/lib/python3.8/site-packages/transformers/configuration_utils.py", line 663, in _get_config_dict
    raise EnvironmentError(
OSError: Can't load config for 'xfbai/AMRBART-large-finetuned-AMR3.0-AMRParsing-v2'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'xfbai/AMRBART-large-finetuned-AMR3.0-AMRParsing-v2' is the correct path to a directory containing a config.json file

Best, Zoher

Zoher15 commented 1 year ago

@goodbai-nlp Your help would be really appreciated. As I mentioned earlier, I am trying to cite your work in my comparison and I am on a deadline for the end of this week.....

goodbai-nlp commented 1 year ago

@Zoher15 Thanks, and hope your project goes well :)

Grandi-Luca commented 1 year ago

Hi @goodbai-nlp, I have implemented a new module based on main.py that only predicts the AMR graph of a sentence using the Seq2SeqTrainer. However, I have noticed that this solution is slower compared to other parser models like Spring or amrlib. Are there any other easier and faster alternatives to using the AMRBART model without relying on the Trainer class?

Best regards, Luca