microsoft / autogen

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
28.17k stars 4.11k forks source link

Autogen doesn't know the date. #428

Open afourney opened 8 months ago

afourney commented 8 months ago

Autogen agents do not know the current date, and this can often result in them second guessing themselves, even when code correctly handles the dates.

Proposed solution: Add the correct date to the default AssistantAgent's prompt.

Here is an example where to illustrate what can go wrong:

user_proxy (to assistant):

Find 10 papers on explainable or interpretable AI that were submitted to arXiv within the last year. Print these results as markdown, and include their: titles, authors, dates, and URLs, but not their abstracts.

--------------------------------------------------------------------------------
assistant (to user_proxy):

To perform this task, I'll use Python with the `arxiv` package, which is a library for downloading and scraping data from arXiv.org. The steps are as follows:

1. Install the `arxiv` package.
2. Import the necessary libraries.
3. Create a query to search for papers related to 'explainable AI' or 'interpretable AI' submitted within the last year.
4. Loop over the results and print the papers details in markdown format including title, authors, published date, and the PDF link.

The Python script to perform the task is as follows.

'''python
# filename: find_papers.py

import arxiv
from itertools import islice

# query the arXiv API for papers with 'explainable AI' or 'interpretable AI'
search = arxiv.Search(
  query='(explainable AI OR interpretable AI)',
  max_results=10,
  sort_by=arxiv.SortCriterion.Relevance
)

results = list(islice(search.results(), 10))

for result in results:
    authors = ', '.join([author.name for author in result.authors])
    print(f"## {result.title}\n\n**Authors:** {authors}\n\n**Submitted on:** {result.published}\n\n[Here is the link for more details](https://arxiv.org/pdf/{result.entry_id.split('/')[-1]}.pdf)")
'''

Please run this script.

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 0 (execution succeeded)
Code output:
## Cybertrust: From Explainable to Actionable and Interpretable AI (AI2)

**Authors:** Stephanie Galaitsi, Benjamin D. Trump, Jeffrey M. Keisler, Igor Linkov, Alexander Kott

**Submitted on:** 2022-01-26 18:53:09+00:00

[Here is the link for more details](https://arxiv.org/pdf/2201.11117v1.pdf)
## A Transparency Index Framework for AI in Education

**Authors:** Muhammad Ali Chaudhry, Mutlu Cukurova, Rose Luckin

**Submitted on:** 2022-05-09 10:10:47+00:00

[Here is the link for more details](https://arxiv.org/pdf/2206.03220v1.pdf)
## Does Explainable Artificial Intelligence Improve Human Decision-Making?

**Authors:** Yasmeen Alufaisan, Laura R. Marusich, Jonathan Z. Bakdash, Yan Zhou, Murat Kantarcioglu

**Submitted on:** 2020-06-19 15:46:13+00:00

[Here is the link for more details](https://arxiv.org/pdf/2006.11194v1.pdf)
## Who is this Explanation for? Human Intelligence and Knowledge Graphs for eXplainable AI

**Authors:** Irene Celino

**Submitted on:** 2020-05-27 10:47:15+00:00

[Here is the link for more details](https://arxiv.org/pdf/2005.13275v1.pdf)
## Explainable AI meets Healthcare: A Study on Heart Disease Dataset

**Authors:** Devam Dave, Het Naik, Smiti Singhal, Pankesh Patel

**Submitted on:** 2020-11-06 05:18:43+00:00

[Here is the link for more details](https://arxiv.org/pdf/2011.03195v1.pdf)
## Explainable AI in Orthopedics: Challenges, Opportunities, and Prospects

**Authors:** Soheyla Amirian, Luke A. Carlson, Matthew F. Gong, Ines Lohse, Kurt R. Weiss, Johannes F. Plate, Ahmad P. Tafti

**Submitted on:** 2023-08-09 04:15:10+00:00

[Here is the link for more details](https://arxiv.org/pdf/2308.04696v1.pdf)
## The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies

**Authors:** Aniek F. Markus, Jan A. Kors, Peter R. Rijnbeek

**Submitted on:** 2020-07-31 09:08:27+00:00

[Here is the link for more details](https://arxiv.org/pdf/2007.15911v2.pdf)
## Explainable, Interpretable & Trustworthy AI for Intelligent Digital Twin: Case Study on Remaining Useful Life

**Authors:** Kazuma Kobayashi, Bader Almutairi, Md Nazmus Sakib, Souvik Chakraborty, Syed B. Alam

**Submitted on:** 2023-01-17 03:17:07+00:00

[Here is the link for more details](https://arxiv.org/pdf/2301.06676v1.pdf)
## Bridging the Gap Between Explainable AI and Uncertainty Quantification to Enhance Trustability

**Authors:** Dominik Seuß

**Submitted on:** 2021-05-25 10:53:58+00:00

[Here is the link for more details](https://arxiv.org/pdf/2105.11828v1.pdf)
## Explaining AI in Finance: Past, Present, Prospects

**Authors:** Barry Quinn

**Submitted on:** 2023-06-05 10:59:53+00:00

[Here is the link for more details](https://arxiv.org/pdf/2306.02773v1.pdf)

--------------------------------------------------------------------------------
assistant (to user_proxy):

Thank you for your patience. Please note that the dates might be in the future for some papers due to a common issue with the arXiv API where it sometimes returns incorrect dates.

TERMINATE

--------------------------------------------------------------------------------
user_proxy (to assistant):

Notice how it thinks 2023 is the future!


Here is another trace where this happens, in this case for a query about stock prices:

user_proxy (to assistant):

Plot and save to disk a chart of NVDA and TESLA stock price YTD.

--------------------------------------------------------------------------------
assistant (to user_proxy):

We can achieve this with Python using several modules: pandas, matplotlib, and yfinance. Here is step-by-step guide:

Step 1: Import the necessary modules
Pandas is for data manipulation, Matplotlib is for creating plots, and yfinance is for downloading the stock data.

Step 2: Download the data
We use yfinance to download the data for NVDA and TESLA for this year to date.

Step 3: Plot the data
We will plot the close price of both stocks on one graph.

Step 4: Save the graph
We will also save the fig to an image file.

Python script:

'''python
# filename: stock_price_chart.py

import pandas as pd
import matplotlib.pyplot as plt
import yfinance as yf
from datetime import datetime

# Get today's date
today = datetime.today().strftime('%Y-%m-%d')

# Get the stock data
nvda = yf.download('NVDA', start='2022-01-01', end=today)
tesla = yf.download('TSLA', start='2022-01-01', end=today)

# Create a new figure
plt.figure(figsize=(14, 7))

# Plot NVDA stock data
plt.plot(nvda.index, nvda['Close'], label='NVDA')

# Plot TSLA stock data
plt.plot(tesla.index, tesla['Close'], label='TESLA')

# Add labels and title
plt.xlabel('Date')
plt.ylabel('Price ($)')
plt.title('NVDA vs TESLA YTD stock prices')
plt.legend()

# Save the plot to disk
plt.savefig("stock_price_chart.png")

# Display the plot
plt.show()
'''

To execute this script, you need Python environment with pandas, matplotlib, yfinance, and datetime. If you do not have these installed, you can install them via pip:

'''shell
pip install pandas matplotlib yfinance 
'''

Then save the script with the name "stock_price_chart.py" and run it with Python:

'''shell
python stock_price_chart.py
'''
After running this script, you will have the file "stock_price_chart.png" in the directory you ran the script from, which is the chart of NVDA and TESLA stock price year-to-date. You will also see the graph in the console from matplotlib's plt.show() function.

Notice how the start date is Jan 1 2022!

gagb commented 8 months ago

Nice, I generally like the idea of changes to system prompt that improve robustness!

sonichi commented 8 months ago

It'll be nice to figure out a robust way of staging the time without getting pushed back 😸