search

LearningOpt / pie

Apache License 2.0
36 stars 3 forks source link

Try solving perf-ninja puzzles #2

Closed dendibakh closed 4 months ago

dendibakh commented 4 months ago

Hello, just discovered your paper, this is great work! Do you want to try your best model in solving perf-ninja puzzles? https://github.com/dendibakh/perf-ninja I'm the main author and I'm very curious to see what modern LLMs can do for perf optimizations. -Denis

AlexShypula commented 4 months ago

Hi Denis! Thanks for your interest! We don't have the spare funds for GPT credits to use the best model and time right now to get to this.

We've open sourced our fine-tuned codellama models here and the code for generating edits in this project. You could start with those, or you could also try to see how GPT-4 or GPT-4o works for this task. Given that we fine-tuned many of these models directly for competitive programming problems: the patterns these models learn are likely to be more high-level; whereas it seems on a quick glance perf-ninja is more low-level. As a result I'm not sure these specific "best models" will be an ideal fit. Happy to chat more if you'd like about this as well!

I'll close this issue, but feel free to drop me an email if you want to get in touch!