search

allenai / rainbow

Neural models of common sense. 🤖
Apache License 2.0
94 stars 6 forks source link

Using GPU #42

Closed puraminy closed 3 years ago

puraminy commented 3 years ago

Hello, I don't have access to Google Cloud! and I want to do some experiments using my Cuda GPU. What should I do? Thanks

puraminy commented 3 years ago

okay I found the solution!

mshen2 commented 2 years ago

okay I found the solution!

Hi! I am just wondering do you mind sharing the solution you had previously? Thank you!

puraminy commented 2 years ago

I can't recall it, but you may check if you use the following code in finetune.py can work or not

        model = t5.models.MtfModel (
            tpu=None,
            model_dir=model_dir,
            model_parallelism=8,
            batch_size=batch_size,
            sequence_length={"inputs": 128, "targets": 128},
            mesh_shape="model:1,batch:1",
            mesh_devices=["gpu:0"],
            learning_rate_schedule=learning_rate,
            save_checkpoints_steps=save_checkpoints_steps,
            keep_checkpoint_max=n_checkpoints_to_keep,
            iterations_per_loop=100,
        )
        model.finetune(
            mixture_or_task_name=mixture,
            pretrained_model_dir=pm_dir,
            finetune_steps=n_steps,
            split=split,
         )