When I use the same model and the same table structure information, the same user question , runs it on the local machine and on the demo website https://defog.ai/sqlcoder-demo . The results are different. Why?

[bigmancomeon]

When I use the same model and the same table structure information, the same user question , runs it on the local machine and on the demo website https://defog.ai/sqlcoder-demo . The results are different. Why?

[NPap0]

Do you also run greedy encoding on your local machine? Same quantization as the web demo? (If it is even quantized?)

[bigmancomeon]

Do you also run greedy encoding on your local machine? Same quantization as the web demo? (If it is even quantized?)

I don't quite understand what greedy encoding and quantization mean. The configuration I run locally is the same as the configuration in the inference.py file.

[NPap0]

Do you also run greedy encoding on your local machine? Same quantization as the web demo? (If it is even quantized?)

I don't quite understand what greedy encoding and quantization mean. The configuration I run locally is the same as the configuration in the inference.py file.

The decoding strategy in the inference.py file in this repo uses beam search = 5

The demo you use on the browser does not use this setting. (Uses greedy decoding instead)

IF that is the only difference then you will need to lower the beam searches to 1. @rishsriv Let me know if I made any mistakes. Also, is the web demo model in fp16?

[github-actions[bot]]

Stale issue message

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.