Empty output in slightly more complex prompt

[PhilWallace]

Hello, many thanks for the brilliant work!

I've tried the models, including llm4decompile-6.7b and 33b, the uo version (llm4decompile-6.7b-uo). However, when I tried to add more information in the prompt (e.g., adding some more detailed requirements), llm4decompile will output nothing at all.

I tried the same prompt on the DeepSeek-Coder 33b, and it can output decompiled code normally.

Could you give some hint about why this happens? Could it be some type of overfitting? Should llm4decompile be used only under the fixed prompt?

[albertan017]

Thank you for reaching out and for your kind words about our work!

Regarding your experience with the llm4decompile models, the behavior you're observing is indeed linked to how these models have been fine-tuned. Specifically, the llm4decompile models have been fine-tuned to work with a very specific input format:

# This is the assembly code:
{asm}
# What is the source code?

During this fine-tuning process, the models were optimized to recognize and respond to this format. Consequently, they may not handle inputs that deviate significantly from this layout well, including more generalized conversational prompts or detailed requirements that don't conform to the expected format. This specialized training enhances their performance on the specific task of decompiling assembly code but reduces their flexibility in handling other types of input.

If you need the model to respond to more varied prompts or to incorporate additional information, you might consider using a more general model like the DeepSeek-Coder 33b, which you mentioned performs better with diverse inputs.

We appreciate your feedback as it helps us improve our models. If you have further questions or need additional assistance, please don't hesitate to ask.