search

facebookresearch / atlas

Code repository for supporting the paper "Atlas Few-shot Learning with Retrieval Augmented Language Models",(https//arxiv.org/abs/2208.03299)
Other
517 stars 67 forks source link

Possibility of releasing closed-book Atlas models? #14

Closed lovhag closed 1 year ago

lovhag commented 1 year ago

Hi, Firstly, thank you for sharing the Atlas model in a very well-documented repo.

I'm currently working on an evaluation of Atlas on ParaRel (a dataset that tests for model consistency, with corresponding paper). However, to do a proper evaluation of potential benefits of retrieval augmentation for the evaluation results, I need to compare against a closed-book Atlas.

From what I can see, you have not shared your closed-book Atlas model weights. So I was wondering if there is a possibility to release these model weights as well? I could train my own attempt at a closed-book Atlas model, but it wouldn't be the same, since you haven't released your 350M train passages from common crawl.

Grateful for any help! /Lovisa

mlomeli1 commented 1 year ago

hi @lovhag , thank you for your interest. I am afraid there are no plans to release these, I am sorry.