[bug] AttributeError: 'NoneType' object has no attribute 'get'

[mrdrprofuroboros]

Describe the bug

File /usr/local/Caskroom/mambaforge/base/envs/careerbot/lib/python3.10/site-packages/guardrails/datatypes.py:321, in Object.validate(self, key, value, schema)
    308 # Types of supported children
    309 # 1. key_type
    310 # 2. value_type
   (...)
    314 
    315 # Check for required keys
    316 for child_key, child_data_type in self._children.items():
    317     # Value should be a dictionary
    318     # child_key is an expected key that the schema defined
    319     # child_data_type is the data type of the expected key
    320     value = child_data_type.validate(
--> 321         child_key, value.get(child_key, None), value
    322     )
    324 schema[key] = value
    326 return schema

AttributeError: 'NoneType' object has no attribute 'get'

To Reproduce Steps to reproduce the behavior:

1. RAIL spec is a little sensitive, I can provide it in email, but generally speaking there are objects within objects, that is what you need to know
2. Runtime arguments (e.g. guard(...))

   raw_llm_output, validated_output = guard(
   openai.ChatCompletion.create,
   prompt_params={...},
   model="gpt-3.5-turbo-16k",
   max_tokens=4000,
   temperature=0,
   )

Expected behavior no AttributeError thrown

Library version: 0.1.8

Additional context

[mrdrprofuroboros]

so the issue comes when the nested object's container is potentially optional. llm was just not returning the container and the whole request failed. Trying to workaround it with choice / case

[irgolic]

Hey, to help us reproduce the error, could you please provide:

• [ ] a minimal rail spec (without sensitive data) that triggers the error
• [ ] a full code snippet including the prompt and params

[mrdrprofuroboros]

Unfortunately I am unable to reproduce the error with smaller prompt/schema and the real openai API, but here's the essence of what it looks like:

import guardrails as gd

rail = """
<rail version="0.1">
<prompt>
Given the following info, extract a structured dictionary

'Alice loves Bob'

@complete_json_suffix_v2  
</prompt>
<output>
    <object name="Alice">
        <object name="lover">
            <string name="name"/>
        </object>
    </object>
</output>
</rail>
"""
guard = gd.Guard.from_rail_string(rail)

def mock_llm_api(prompt: str, **kwargs) -> str:
    return """{
        "Alice": {"lover": "Bob"}
    }"""

raw_llm_output, validated_output = guard(
    mock_llm_api,
    prompt_params={},
)
print(validated_output)

So I'd expect guardrails to reask this somehow, but it just crashes with basic AttributeError:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[135], line 26
     21 def mock_llm_api(prompt: str, **kwargs) -> str:
     22     return """{
     23         "Alice": {"lover": "Bob"}
     24     }"""
---> 26 raw_llm_output, validated_output = guard(
     27     mock_llm_api,
     28     prompt_params={},
     29 )
     30 print(validated_output)

File /usr/local/Caskroom/mambaforge/base/envs/careerbot/lib/python3.10/site-packages/guardrails/guard.py:187, in Guard.__call__(self, llm_api, prompt_params, num_reasks, *args, **kwargs)
    176     # TODO(shreya): should we overwrite self.instructions for this run?
    177 runner = Runner(
    178     instructions=kwargs.get("instructions", self.instructions),
    179     prompt=self.prompt,
   (...)
    185     base_model=self.base_model,
    186 )
--> 187 guard_history = runner(prompt_params=prompt_params)
    188 self.guard_state = self.guard_state.push(guard_history)
    189 return guard_history.output, guard_history.validated_output

File /usr/local/Caskroom/mambaforge/base/envs/careerbot/lib/python3.10/site-packages/guardrails/run.py:94, in Runner.__call__(self, prompt_params)
     86 instructions, prompt, input_schema, output_schema = (
     87     self.instructions,
     88     self.prompt,
     89     self.input_schema,
     90     self.output_schema,
     91 )
     92 for index in range(self.num_reasks + 1):
     93     # Run a single step.
---> 94     validated_output, reasks = self.step(
     95         index=index,
     96         api=self.api,
     97         instructions=instructions,
     98         prompt=prompt,
     99         prompt_params=prompt_params,
    100         input_schema=input_schema,
    101         output_schema=output_schema,
    102         output=self.output if index == 0 else None,
    103     )
    105     # Loop again?
    106     if not self.do_loop(index, reasks):

File /usr/local/Caskroom/mambaforge/base/envs/careerbot/lib/python3.10/site-packages/guardrails/run.py:160, in Runner.step(self, index, api, instructions, prompt, prompt_params, input_schema, output_schema, output)
    157 parsed_output = self.parse(index, output, output_schema)
    159 # Validate: run output validation.
--> 160 validated_output = self.validate(index, parsed_output, output_schema)
    162 # Introspect: inspect validated output for reasks.
    163 reasks = self.introspect(index, validated_output, output_schema)

File /usr/local/Caskroom/mambaforge/base/envs/careerbot/lib/python3.10/site-packages/guardrails/run.py:287, in Runner.validate(self, index, parsed_output, output_schema)
    285 """Validate the output."""
    286 with start_action(action_type="validate", index=index) as action:
--> 287     validated_output = output_schema.validate(parsed_output)
    289     action.log(
    290         message_type="info",
    291         validated_output=reasks_to_dict(validated_output),
    292     )
    294     return validated_output

File /usr/local/Caskroom/mambaforge/base/envs/careerbot/lib/python3.10/site-packages/guardrails/schema.py:515, in JsonSchema.validate(self, data)
    512         logger.debug(f"Field {field} not in schema.")
    513         continue
--> 515     validated_response = self[field].validate(
    516         key=field,
    517         value=value,
    518         schema=validated_response,
    519     )
    521 if check_refrain_in_dict(validated_response):
    522     # If the data contains a `Refain` value, we return an empty
    523     # dictionary.
    524     logger.debug("Refrain detected.")

File /usr/local/Caskroom/mambaforge/base/envs/careerbot/lib/python3.10/site-packages/guardrails/datatypes.py:320, in Object.validate(self, key, value, schema)
    308 # Types of supported children
    309 # 1. key_type
    310 # 2. value_type
   (...)
    314 
    315 # Check for required keys
    316 for child_key, child_data_type in self._children.items():
    317     # Value should be a dictionary
    318     # child_key is an expected key that the schema defined
    319     # child_data_type is the data type of the expected key
--> 320     value = child_data_type.validate(
    321         child_key, value.get(child_key, None), value
    322     )
    324 schema[key] = value
    326 return schema

File /usr/local/Caskroom/mambaforge/base/envs/careerbot/lib/python3.10/site-packages/guardrails/datatypes.py:321, in Object.validate(self, key, value, schema)
    308 # Types of supported children
    309 # 1. key_type
    310 # 2. value_type
   (...)
    314 
    315 # Check for required keys
    316 for child_key, child_data_type in self._children.items():
    317     # Value should be a dictionary
    318     # child_key is an expected key that the schema defined
    319     # child_data_type is the data type of the expected key
    320     value = child_data_type.validate(
--> 321         child_key, value.get(child_key, None), value
    322     )
    324 schema[key] = value
    326 return schema

AttributeError: 'str' object has no attribute 'get'

[CalebCourier]

Was able to replicate the issue with the example provided on 0.1.8. This will be fixed by schema verification in an upcoming release.

[CalebCourier]

0.2.0 was released with fixes for this issue. Please see the migration guide and let us know if you continue to experience related problems.