search

armsp / AIFU

AI flub ups
https://aifu.shantam.io/
1 stars 1 forks source link
accountability ai algorithm algorithmic-bias algorithmic-fairness algorithmic-governance disaster ethics government llm natural-language natural-language-processing nlp policy public-policy societal-impact tragedy watchdog

readme

AIFU

Demo

AI flub ups

AIFU aims to be a platform for researchers and anyone looking for a place to learn about the repercussions of using AI blatently, too soon, indiscriminately, harmfully. Its an experiment in meta analysis. What makes it interesting is that all of the information is "generated" and "retreived" using language model(s). Currently, the design of the platform leans towards using causal language models as judge, jury and executioner if you will.

How to use the platform?

We intend you to submit links to articles that you may think document/inform of cases where harm "may" have been caused becasue of AI. Your submission will be recorded publicly in our GitHub repository as an issue and the processing pipeline will start. You may have to wait for a few seconds to a few minutes for the workflow to finish and a short response of the outcome of the workflow run will be posted as a reply to the issue. For example -> https://github.com/armsp/AIFU/issues/84

Steps -

  1. Go to the country of the concerned article/news. If the article/news does not pertain to a country then submit it as global.
  2. Enter the headline, url, media organization and other details you can discern (see examples when you navigate to a country)
  3. Press Submit to start the workflow

Research Question

AI is going to pervade software systems just as software systems have pervaded the social fabric. And this work is quite prescient because we have already seen how AI systems can cause widespread harm.

“Had a great conversation with Yoshua Bengio. Both of us agreed that a good step forward for AI risk is to articulate the concrete scenarios where AI can lead to significant harm.” - Andrew Ng

Through this platform, my aim is to monitor the utilization of AI by governmental entities. By conducting thorough analysis of the collected data, I want to find out if we can provide "concrete course(s) of action" or policy recommendations to lawmakers, the general public, and other stakeholders regarding their engagement with AI-based services. Crucially, this initiative will enable us to gain insights into the preferences and purposes of government agencies in employing AI. By doing so, we can proactively address potential risks associated with AI based implementation in specific domains such as child welfare and recidivism, while promoting fair and equitable utilization of AI.

Outline

The objective of this project is to establish a comprehensive archive or repository that serves as a resource for policymakers, researchers, students, and other stakeholders seeking to acquire knowledge regarding the potential adverse consequences arising from the indiscriminate, premature, or injudicious use of algorithms and AI.

Through this platform, my aim is to monitor the utilization of AI by governmental entities. By conducting thorough analysis of the collected data, I want to find out if we can provide "concrete course(s) of action" or policy recommendations to lawmakers, the general public, and other stakeholders regarding their engagement with AI-based services. Crucially, this initiative will enable us to gain insights into the preferences and purposes of government agencies in employing AI. By doing so, we can proactively address potential risks associated with AI based implementation in specific domains such as child welfare and recidivism, while promoting fair and equitable utilization of AI.

Motivation Necessity: Knowing about something harmful is crucial for several reasons -

What makes this project intresting is that I plan to archive, extract and collate relevant information from the articles using AI itself. Therefore this platform and project is an exercise in -

In short causal language model is the judge, jury, and executioner.

Related literature

Most of the existing databases and projects around this are more about general (deeper) understanding of AI, accountability, fairness and transparency, where real life examples are just case studies. They are very broad in their general reach which I believe dilutes their effectiveness. Some of the important resources that I found in my literature review research are -

The AI Index by The Stanford Institute for Human-Centered Artificial Intelligence

They do not gather information about the harms caused by AI but about the developments, growth, measuring trends and progress and market of AI in various countries. They also track global AI legislation. Quoting them -

The AI Index Report tracks, collates, distills, and visualizes data related to artificial intelligence. Its mission is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, executives, journalists, and the general public to develop intuitions about the complex field of AI.

Reports are available from the year 2017 with a gap in 2020.

The AI Now Institute

They focus on accountability and on policy recommendations. Their aim is not to create any database of harms. However studying their reports and recommendations would be instrumental for the last step of this project which is policy recommendations. Quoting them -

Founded in 2017, the AI Now Institute produces diagnosis and policy research to address the concentration of power in the tech industry. We develop policy strategy to redirect away from the current trajectory: unbridled commercial surveillance, consolidation of power in very few companies, and a lack of public accountability.

Data Harm Record by Data Justice Lab

This is not a database but a running document that has not been updated since August 2020. The definition of harm is closer to the kind of cases that I wish to have in my database but the cases they have studied range from generic harm to specific harms. Since its not being maintained anymore, my plan is to use all the articles that it references and to input it to the system I have designed. The idea is that all those cases should be integrated and to observe why a particular case was rejected.

AI Incident Database

This is the closest database to what I am building. Quoting them -

The AI Incident Database is dedicated to indexing the collective history of harms or near harms realized in the real world by the deployment of artificial intelligence systems. Like similar databases in aviation and computer security, the AI Incident Database aims to learn from experience so we can prevent or mitigate bad outcomes.

They however collect a rather broad spectrum of cases - incidents where intelligent systems have caused safety, fairness, or other real-world problems. It has also changed a lot since the time I started working. A while ago you could only browse and read the content of submitted incident's article(s). Since then they have added a multitude of features such as Named Entities, CSET Taxonomy Classifications, GMF Taxonomy Classifications, a timeline of reports and a list of similar reports/variants. However not all incidents have all this information associated with them and its not apparently clear on what basis are incients chosen to be worked upon to provide all the relevant information.

To the best of my knowledge, nothing like AIFU has been done before.

Data

How does the corpus help you answer your research question? Citing possible solutions based on implementations in other countries, latest developments in AI that successfully mitigate the shortcomings of previous scenarios/models, guardrails placed by other countries and regulations in other countries. With all the information and analysis, I want to find out if in the end we can “Recommend” some “concrete” course of action(s) or policy decisions to lawmakers, general public and other stakeholders.

Approach

Once a user submits any article by providing its URI and Headline, we first try to figure out if the article is not at all related to what we are looking for by analyzing the headline itself. The idea here is to reject irrelevant articles and not so much accept relevant articles becasue that happens in the next step. However, it is important to understand that subsequent articles about an event may be ignored if the workflow depends too heavily on Headline itself. To mitigate that we use the content of the article itself and also provide method to "update" a case by providing subsequent articles for that case.

After that, we extract the contents of the article using the provided URI. Following which, using prompt engineering, we try to extract all the relevant content by asking questions. Finally we also summarize the article. The outputs are then serialized in the database.

Before actually storing the contents in the database, we also query the database for exactly similar articles (maybe the same article was already stored before or maybe a different media reported on the same case, both of which are highly likely). If an exactly similar article is found then we try to merge the information extracted now with whatever we had extracted earlier and update the database record. If the article and case is completely new then we just make a new record in the database. However, Near Duplicate Detection or identifying similar articles by different media is a research field in itself. Currently we have observed that using using universal sentence encoder as a baseline works for us. However as the workflow becomes more complicated and streamlined and accurate, we will certainly have to revisit this part for automatic detection of duplicates and similar articles.

API Endpoints Endpoint Request type Comment
/records GET Responsible for providing the details of cases per country to the frontend.
/submit POST Responsible for creating Automatic GitHub issues/discussions whenever a user submits an article. Also triggers the information extraction endpoint in the background (steps listed in Approach).
/heartbeat GET Used for testing if the server is alive.
/export GET Provides a json file of exported content from the database collection for a given country.

More endpoints may be added later on based on the needs of the project.

Policy Recommendations Literature Review and Starting Point

(Near) Duplicate Detection This is a whole research area in itself. After trying a few approaches I have settled with Universal Sentence Encoder as a baseline. This part of the pipeline would be made more robust over time.
Methods that do not yield good results - TFIDF, Jaccquard Distance. See jupyter notebook for experiments. My first attempt was to use TFIDF. For making the vocabulary and TFIDF weights I used the latest news dataset from n24 news released as part of the research N24News: A New Dataset for Multimodal News Classification Method deployed - Universal Sentence Encoder. USE can work on full text, the embedddings just get diluted.

Methods to explore for Future Work -

  1. there is a research by google for huge datasets though - https://github.com/google-research/deduplicate-text-datasets
  2. Try USE based on other architectures - https://tfhub.dev/s?q=google%2Funiversal-sentence-encoder%2F4%20OR%20google%2Funiversal-sentence-encoder-large%2F5
  3. Try a multi modal approach where we use similarity with topic modelling as well as journalistic five W and one H questions (5W1H) from news articles: who did what, when, where, why, and how?

Definition of Success for the project

Technical Details and Workflow

Git/GitHub Branches

BACKEND SERVER

stateDiagram-v2

    [maintainer] --> CommitBackend: update backend
    CommitBackend --> LoginToAzureContainerRegistry
    LoginToAzureContainerRegistry --> BuildAndTagDockerImage
    BuildAndTagDockerImage --> PushImageToRegistry
    PushImageToRegistry --> AzureContainerGroupDeploy: az container create --resource-group <"resource group name"> --file <"deploymentfile.yml">

FRONTEND UI

stateDiagram-v2

    [maintainer] --> CommitFrontEnd: update frontend
    CommitFrontEnd --> PushToGitHub
    PushToGitHub --> RunGitHubAction : GitHub PAT, Secret Key
    RunGitHubAction --> Deployment
    state Deployment {
        UpdateDeploymentCodeOnBranch --> DeployToGHPages: build and deploy (yaml) |ubuntu, node, checkout, build, commit, push
    }

User Submission

flowchart TB
    SubmitArticle --> A[Detect Headline Language] --> |Language Supported| D
    A --> |Language Not Supported| C[Make GH Issue] --> Stop
    D[Classify Headline] --> |Accepted|E[Detect Content Language]
    D --> |Rejected|C 
    E --> |Language Not Supported| C
    E --> |Language Supported| F
    F[Make GH Issue] --> J[Information Retreival Pipeline]

Information Extraction Pipeline

flowchart TB
    A[Submit Link and Headline] --> B{"Does the headline suggest<br> it might be related to AI<br> and its potential harm?"}
    B --> |Yes| C[Extract text from article link]
    B --> |No| E[End]
    C --> D[Assess from content if it concerns actual harm caused to humans]
    D --> |No harm to humans| F[Reject, save in a different collection]
    D --> |Harm caused to humans| G[Extract Information based on provided Prompts]
    G --> H[Check if exactly similar article exists in the database]
    H --> |No| I
    H --> |Yes| J[Merge context and update extracted information]
    J --> I[Update information in Database]

Major Changes Log

Features to come

Future Development Activities

Decision Log and Tradeoffs

Clock - 95 hrs