search

srush / annotated-s4

Implementation of https://srush.github.io/annotated-s4
https://srush.github.io/annotated-s4
MIT License
460 stars 61 forks source link

Add two tasks: Listops and IMDB Review #50

Closed NathanYanJing closed 2 years ago

NathanYanJing commented 2 years ago

Adding two tasks of LRA benchmark.

jxiw commented 2 years ago

ListOps

Downolad dataset in GitHub page. Unzip the downloaded dataset, and move it to the project folder.

python -m s4.train --dataset listops-classification --model s4 --epochs 100 --bsz 50 --d_model 128 --n_layers 4 --ssm_n 64 --lr 1e-2 --p_dropout 0 --lr_schedule

Training with the previously defined hyper-parameters yields the test accuracy 54.3% on the test set.

IMDB

We used the huggingface datasets for LRA IMDB review task.

python -m s4.train --dataset imdb-classification --model s4 --epochs 100 --bsz 50 --d_model 64 --n_layers 4 --ssm_n 64 --lr 1e-2 --p_dropout 0

Training with the previously defined hyper-parameters yields the test accuracy 80.7% on the test set.

jxiw commented 2 years ago

Here is the training curve for IMDB Review.

jax-imdb
jxiw commented 2 years ago

Ok, working on it.

srush commented 2 years ago

Added some review comments. I think there was a merge failure, so if you can try again to pull.

Also be sure to run make autoformat to check the formatting.

NathanYanJing commented 2 years ago

Results of ListOps:

listops