Benchmarks fail for the gpt-3.5-turbo model family

[MkuuWaUjinga]

Hi Josh,

thanks for this awesome project :tada:.

Some observations from running python main.py --sourcedir ../benchmarks/flask-nodejs/source --targetlang nodejs --model gpt-3.5-turbo-16k --temperature 0:

• The Dockerfile is already not generated correctly. It just contains one line named CODE. This is probably due to the instructions in p4_output_formats/single_file.
• Internal dependency resolve of benchmarks/flask-nodejs/source/db.py is stuck in an infinite loop cause the output of the LLM is [db.py] again.

Perhaps this can be fixed with a lot of prompt engineering but as of now I'd probably communicate gpt-3.5-turbo is not supported. Wdyt? Also, you could easily check that file x isn't in the list of internal dependencies of file x to fix getting stuck in an infinite loop. Happy to add a PR for this.

[joshpxyne]

I'll add this to the readme. Indeed, 3.5 is much weaker at programming so this behavior isn't unexpected.

Interesting that you're finding an infinite loop, feel free to submit a PR and I'll review. Thanks!