Closed ptyin closed 3 months ago
As said in datasets.md,
https://github.com/openai/transformer-debugger/blob/dc1f898725113bec6cf1006e48f9c5219f8fbdde/datasets.md?plain=1#L17
But I believe the neuron size is should be 4 times d_model which is 3072 instead of 3084. Could you kindly explain the number here?
d_model
that's an error in the documentation, feel free to make a PR to fix :)
As said in datasets.md,
https://github.com/openai/transformer-debugger/blob/dc1f898725113bec6cf1006e48f9c5219f8fbdde/datasets.md?plain=1#L17
But I believe the neuron size is should be 4 times
d_model
which is 3072 instead of 3084. Could you kindly explain the number here?