Modify LLM Prompts

[Archly2022]

Large language models usually gravitate towards relation patterns(vector embeddings) to generate response especially when corresponding towards a predictable output. So I made some changes:

The excepted output example json schema: { "method": "SVC", "dataset": "Iris", "task": "classification", "accuracy": 1.0, "C": 1.0, "degree": 3, "tol": 0.001, "cache_size": 200, "max_iter": -1, "test_size": 0.2, "random_state": 42, "kernel": "linear", "condition_as_natural_langauge": [ "Using linear kernel on SVC model.", "Excluding class 2 from Iris dataset.", "Splitting data into 80% training and 20% testing." ], "advice_to_improve_acc": [ "Consider using cross-validation for better performance evaluation.", "Experiment with different kernels to optimize results.", "Increase the dataset size to improve generalization." ] }

The output is pretty much the same since it was directed towards a generative response, almost!

The output when removed example schema:

response: { "method": "Support Vector Machine", "dataset": "Iris", "task": "Classification", "accuracy": 1.0, "other_param_here": { "C": 1.0, "kernel": "linear", "degree": 3, "gamma": "auto", "coef0": 0.0, "shrinking": true, "probability": false, "tol": 0.001, "cache_size": 200, "class_weight": null, "verbose": false, "max_iter": -1, "decision_function_shape": "ovr", "break_ties": false, "random_state": null, "test_size": 0.2, "random_state_tts": 42, "shuffle": true, "stratify": true }, "condition_as_natural_language": [ "Binarized class labels (excluding Class 2).", "Excluded Class 1 from evaluation." ], "advice_to_improve_acc": [ "Use a more complex model.", "Use a larger dataset.", "Try different hyperparameters." ] }

The response without excepted output referred on the notebook:

response without excepted parameters and example output on notefile --> {"condition_as_natural_language": [ "Iris dataset with two classes (0 and 2)" ], "param_name_1": "test_size", "param_value_1": 0.2, "param_name_2": "random_state", "param_value_2": 42, "param_name_3": "shuffle", "param_value_3": true, "param_name_4": "stratify", "param_value_4": "labels", "param_name_5": "C", "param_value_5": 1.0, "param_name_6": "kernel", "param_value_6": "linear", "param_name_7": "degree", "param_value_7": 3, "param_name_8": "gamma", "param_value_8": "auto", "param_name_9": "coef0", "param_value_9": 0.0, "param_name_10": "shrinking", "param_value_10": true, "param_name_11": "probability", "param_value_11": false, "param_name_12": "tol", "param_value_12": 0.001, "param_name_13": "cache_size", "param_value_13": 200, "param_name_14": "class_weight", "param_value_14": null, "param_name_15": "verbose", "param_value_15": false, "param_name_16": "max_iter", "param_value_16": -1, "param_name_17": "decision_function_shape", "param_value_17": "ovr", "param_name_18": "break_ties", "param_value_18": false, "param_name_19": "random_state", "param_value_19": null, "param_name_20": "excluded_class", "param_value_20": 1, "result_name_1": "accuracy", "result_value_1": 1.0, "result_name_2": "precision", "result_value_2": 1.0, "result_name_3": "recall", "result_value_3": 1.0, "result_name_4": "f1-score", "result_value_4": 1.0, "result_name_5": "support", "result_value_5": 10, "advice": "Try different kernels (e.g., 'rbf', 'poly') and tune hyperparameters like C and gamma to improve accuracy." }

Seemed more accurate and informative when retuned. configuration response --> Temperature 0.1

[shure-dev]

Thanks, could you let me know why you want to remove json output example? to save cost/token?

and temperature should be zero

[Archly2022]

"advice": "Try different kernels (e.g., 'rbf', 'poly') and tune hyperparameters like C and gamma to improve accuracy." } This is the reason it got different recommendations

"advice_to_improve_acc": [ "Consider using cross-validation for better performance evaluation.", "Experiment with different kernels to optimize results.", "Increase the dataset size to improve generalization." ]