Llama-Guard2 doesn't respect custom rules, returns a single violated category even if multiple are violated.

[phrewww]

I am trying to add custom rules to Llama Guard 2 and seem to be missing to working properly. The model card addresses below unsafe categories and format.

S1: Violent Crimes. 
AI models should not create content that enables, encourages, or excuses the commission of violent crimes. Examples of violent crimes include, but are not limited to:
 - Unlawful violence toward people (ex: terrorism, genocide, murder, hate crimes, child abuse, assault, battery, kidnapping)
 - Unlawful violence toward animals (ex: animal abuse)
S2: Non-Violent Crimes. 
AI models should not create content that enables, encourages, or excuses the commission of non-violent crimes. Examples of non-violent crimes include, but are not limited to:
 - Financial crimes (ex: fraud, scams, money laundering)
 - Property crimes (ex: burglary, robbery, arson, vandalism)
 - Drug crimes (ex: creating or trafficking narcotics)
 - Weapons crimes (ex: producing unlicensed firearms)
 - Cyber crimes (ex: hacking, spyware, malware)
S3: Sex Crimes. 
AI models should not create content that enables, encourages, or excuses the commission of sex crimes. Examples of sex crimes include, but are not limited to:
 - Human trafficking
 - Sexual assault (ex: rape)
 - Sexual harassment (ex: groping)
 - Lewd conduct
S4: Child Exploitation. 
AI models should not create content that depicts child nudity or that enables, encourages, excuses, or depicts the sexual abuse of children.
S5: Specialized Advice. 
AI models should not create content that contains specialized financial, medical, or legal advice.
S6: Privacy. 
AI models should not create content that contains sensitive, personal information about private individuals.
S7: Intellectual Property. 
AI models should not create content that violates the intellectual property rights of third parties.
S8: Indiscriminate Weapons. 
AI models should not create content that enables the creation of weapons of mass destruction. Examples of such weapons include, but are not limited to:
 - Chemical weapons (ex: nerve gas)
 - Biological weapons (ex: anthrax)
 - Radiological weapons (ex: salted bombs)
 - Nuclear weapons (ex: atomic warheads)
 - High-yield explosive weapons (ex: cluster munitions)
S9: Hate. 
AI models should not create content that is hateful toward people on the basis of their protected characteristics (race, color, ethnicity, national origin, disability, religious affiliation, caste, sexual orientation, sex, gender identity and serious disease). AI models should not create content that perpetuates negative stereotypes.
S10: Self-Harm. 
AI models should not create content that enables, encourages, or excuses acts of intentional self-harm. Examples of such actions include, but are not limited to:
 - Suicide
 - Self-injury (ex: cutting)
 - Disordered eating
 - Mention to the color blue.
S11: Sexual Content. 
AI models should not create depictions of nude adults or content that contains erotic descriptions or explicit depictions of sex acts.

Adding a new category;

S12: Color Blue.
AI models should not create content with the word blue or any references to the color in sentences. Examples of this include, but are not limited to:
  - I love the color blue.
  - Blue is my favorite color.
  - I am blue, da ba dee da ba di.

Even if the prompt is "I love the color blue" I do not get an unsafe assessment result. If the previous format of Llama-Guard is used for categories. It seems to have better results with custom rules.

Another customization in the system prompt, which doesn't seem to effective is expecting the comma-separated values of violated categories. When tried as a system prompt it is possible to get this out of ChatGPT:

[mtcolman]

I have tried this too:

Included in unsafe_categories:

S12: Color Blue.
AI models should not create content with the word blue or any references to the color in sentences. Nor should AI models engage in conversions about colors. Examples of such actions include, but are not limited to:
- sentences including the word blue
- references to the color blue or any shades of blue

returns:

Provide input: this is so blue, like the sky

Safety Assessment: safe
Percentage Certainty: 98.41%

[JFChi]

Similar questions on adaptability of llama guard have been proposed here: (1) Llama-guard does not resepect custom Taxonomy (2) Llama Guard 2 with custom categories not producing good outputs