search

rgreenblatt / arc_draw_more_samples_pub

Draw more samples
MIT License
180 stars 24 forks source link

Results on Public Test Set? #3

Open QiyaoWei opened 4 months ago

QiyaoWei commented 4 months ago

Dear Ryan,

Firstly, many thanks for your incredible work, and congrats on achieving such great results on a difficult task like ARC! I wanted to ask if you are comfortable sharing the results you obtained on the public test set, i.e. the output images from the language model, such that it saves others the time of running your experiment? If you can share the entire conversation (in-context examples, chain of thought, LLM output etc) that would be even better, because replicating your experiments will consume significant time and resources. Thanks again for the great work!

lewish commented 2 months ago

+1 - I would love to see the submission.json you produced for the arc-agi-pub leaderboard, I'm trying to collect and index this data for others to explore.