search

mrwadams / stride-gpt

An AI-powered threat modeling tool that leverages OpenAI's GPT models to generate threat models for a given application based on the STRIDE methodology.
https://stridegpt.streamlit.app
MIT License
335 stars 100 forks source link

Do you get wierdly formatted output #18

Open priyanka-raghavan-maersk opened 1 month ago

priyanka-raghavan-maersk commented 1 month ago

HI @mrwadams - This is more of a question. Sometimes even though prompt is programmed to return a json, i sometimes get an output from the open AI/Azure open AI that's not json but a string. If it's only string, it's ok but i sometimes also get a statement like this "Here's a JSON response providing a threat model and improvement suggestions for the web application described:\n\n```json" and there are other variants:)

I now suddenly have a very big exception handler that handles JsonDecodeError where i spend time parsing. Is this normal? Have you seen this?

hype8912 commented 1 month ago

Yes. I've had to add code to find where the real data begins in the response. For example, on the threat_model I've found the index of the first "{" and last "}" then slice the string using the indexes to get the real json_content. I do something similar for the markdown and mermaid responses.

mrwadams commented 1 month ago

Thanks for highlighting this @priyanka-raghavan-maersk. I don't use the Azure OpenAI Service integration much so haven't come across the issue.

It would be helpful to know which model you're using in your deployment, along with the API version.