post: Malicious Compliance

[afflom]

AI + GRC will run amok. Organizations will maliciously enforce the compliance of their policies over other organizations and users. The only way to combat this is to leverage agents to defend us from AI control. These agents will enforce their user's personal or organizational policy, which sometimes complies with external policy, but also occasionally violates it.

AI + GRC is a powerful combination, because a policy that can be correlated to a subject is a powerful prompt.

An example of this is a model that authors IAC by referencing a policy that defines the required characteristics of the deployed system. Policy can define concerns such as uptime, vulnerability management and mitigation, software provenance requirements, regional availability, and other business concerns such as revenue targets. Observation of the deployed environments informs the model of the efficacy of the policy and allows for the policy to be tuned to specific outcomes.

An outcome of one policy might conflict with the outcome of the policy of another user. Policies will be written to exert dominance over conflicting policies. Winning policies will gain dominance in their domain or industries.

[afflom]

thought: you could program sims like this.

(slight lol, because this explains all of the NPCs in the world)