Paper reports: "For all MassiveText subsets, we filter out non-English documents, process data into a homogeneous text-only format, deduplicate documents, and filter out documents too similar to those in our test sets." Therefore it seems safe to assume no test-set contamination.
kingoflolz
commented
2 years ago
Paper reports: "For all MassiveText subsets, we filter out non-English documents, process data into a homogeneous text-only format, deduplicate documents, and filter out documents too similar to those in our test sets." Therefore it seems safe to assume no test-set contamination.
https://storage.googleapis.com/deepmind-media/research/language-research/Training%20Gopher.pdf