cohere-ai / sandbox-grounded-qa

A sandbox repo for grounded question answering with Cohere and Google Search
MIT License
136 stars 17 forks source link

Mystifying, but correct response #11

Open saraswat opened 1 year ago

saraswat commented 1 year ago

It is critical -- for the application I have in mind -- that the API return the text (and its url) justifying the answer.

Oddly, this time I do get a response, and it is correct. But the stated url does not have this information! The entire trace does not have this information (CL). It just mysteriously produces the correct answer.

Looks like the log does not show all the texts processed...?

  python3 cli_demo.py --cohere_api_key ...  --serp_api_key ... --verbosity 2000
question:  What is the GLOBEX Code for Crude Oil Futures on the Chicago Mercantile Exchange?
contextual question prompt: user: How much does a watermellon weight?
bot: The average watermellon weighhts about 20 - 25 pounds
user: where do they grow?
-where do watermellons grow? 
---
user: who was the president in 2010
bot: Obama was president in 2010
user: when was Obama born?
bot: August 4, 1961
user: who wrote attention is all you need?
-who wrote attention is all you need? 
---
user: can people live on mars?
bot: People cannot live without advanced technology.
user: what is the killers latest album?
bot: Pressure Machine
user: was it well recieved
-was Pressure Machine by the Killers well recieved?
---
user: When is ACL is this year? 
bot: ACL will be september 2nd
user: where?
-where is ACL this year? 
---
user: but speaking of do you know when that band formed?
bot: 1967
user: who was their guitarist?
-who was the guitarist of The Band?
---
user: when did world war 2 end?
bot: September 2, 1945
user: Who was President at that time?
bot: FDR
user: who did he serve with?
-who did President Franklin Delano Roosevelt serve with? 
---
user: what year was chuck berry born?
bot: 1942
user: where?
-where was Chuck Berry born? 
---
user: What year did Woodstock take place?
bot: 1969
user: where was it held?
-where was Woodstock held? 
---
user: who wrote the screenplay for Rocky?
bot: Sylvester Stallone
user: what is the plot?
-what is the plot of Rocky by Sylvester Stallone?
---
user:  What is the GLOBEX Code for Crude Oil Futures on the Chicago Mercantile Exchange?
-
contextual question: What is the GLOBEX Code for Crude Oil Futures on the Chicago Mercantile Exchange? 

https://serpapi.com/search
all search result context: Just getting started with futures? Learn more about futures and the unique advantages of futures trading.
Companies that produce or consume crude oil can manage price risk by using crude oil futures as hedge to lock in the price for the crude oil — the selling price if they produce it or the buying price if they consume it. Investors can also use crude oil futures to hedge against investments in their portfolio that may be sensitive to crude oil price changes.
Crude oil futures provide individual investors with an easy and convenient way to participate in one of the world's most important commodity markets. In addition, a broad cross-section of companies in the energy industry-from those involved in exploration and production to refiners-can use crude oil futures contracts to hedge their price risk. Light, sweet crude is preferred by refiners because of its low sulfur content and relatively high yields of gasoline, diesel fuel, heating oil, and jet fuel. Even companies that are substantial consumers of energy products can use crude oil futures to protect against adverse price fluctuations.
 
Investors and traders can use crude oil futures to speculate on the future price of crude oil and can be used as an alternative to oil and gas stocks. Crude oil prices can change due to a number of factors but primarily from the perceived changes in supply and demand that comes from both overall output worldwide and the economic health of the industry’s major consuming countries.
It is important to understand the benefits and risks involved with crude oil futures before placing a futures trade. Compared to traditional investments, with crude oil futures you can trade nearly 24 hours a day during the trading week and take advantage of trading opportunities regardless of market direction. Crude oil futures also provide the ability to trade with greater leverage and allow a more efficient use of trading capital. However, trading leveraged products like crude oil futures also involves the risk that losses can exceed the amount originally invested and may not be suitable for all investors.
This site is designed for U.S. residents. Non-U.S. residents are subject to country-specific restrictions. Learn more about our services for non-U.S. residents.
The Charles Schwab Corporation provides a full range of brokerage, banking and financial advisory services through its operating subsidiaries. Its broker-dealer subsidiary, Charles Schwab & Co., Inc. (Member SIPC), offers investment services and products, including Schwab brokerage accounts. Its banking subsidiary, Charles Schwab Bank, SSB (member FDIC and an Equal Housing Lender), provides deposit and lending services and products. Access to Electronic Services may be limited or unavailable during periods of peak demand, market volatility, systems upgrade, maintenance, or for other reasons.
Crude oil futures on the New York Mercantile Exchange (NYMEX) are the world's most actively traded futures contract on a physical commodity. Because of its excellent liquidity and price transparency, the contract is used as a principal international pricing benchmark. The NYMEX also offers trading in heating oil futures and gasoline futures.
At Schwab, you get access to specialize trading tools and resources, such as real-time crude oil futures quotes, timely research and education, and other helpful insights. 
Charles Schwab Futures and Forex LLC (NFA Member) and Charles Schwab & Co., Inc. (Member FINRA/SIPC) are separate but affiliated companies and subsidiaries of The Charles Schwab Corporation. 
Considering trading crude oil futures? Here are the crude oil futures contract specifications.
Futures and futures options trading involves substantial risk and is not suitable for all investors. Please read the Risk Disclosure for Futures and Options prior to trading futures products. Futures accounts are not protected by SIPC. Futures and futures options trading services provided by Charles Schwab Futures and Forex LLC. Trading privileges subject to review and approval. Not all clients will qualify. 
Crude oil futures are 1,000 barrels per contract, traded from 6:00 p.m. U.S. until 5:00 p.m. U.S. ET, all months of the year. However, you can trade more than just NYMEX crude oil futures online with Schwab. We also offer Brent crude oil futures as well as E-mini crude oil futures, which are just 50% of the size of a standard futures contract. E-mini crude futures trade exclusively on the Chicago Mercantile Exchange's Globex® platform nearly 24 hours per day. 

    Please visit this URL to
    review a list of supported browsers.

The ICE West Texas Intermediate (WTI) Light Sweet Crude Oil Futures
Contract offers participants the opportunity to trade one of the
world's most liquid oil commodities in an electronic marketplace.
The contract not only brings the benefits of electronic trading a
US light sweet crude marker, but also brings together the world's
most significant crude benchmarks on a single exchange: Brent,
(Platts) Dubai, and WTI, as well as the emerging benchmarks Murban
and Midland WTI AGC. This offers a reduction in collateral
requirements through the offsetting of margins.

Take advantage of $2.25 per contract pricing plus specialized tools, research, and support.
© 2022 Charles Schwab & Co., Inc. All rights reserved. Member SIPC. Unauthorized access is prohibited. Usage will be monitored.
relevant result context: Take advantage of $2.25 per contract pricing plus specialized tools, research, and support.
© 2022 Charles Schwab & Co., Inc. All rights reserved. Member SIPC. Unauthorized access is prohibited. Usage will be monitored.
Some characters could not be decoded, and were replaced with REPLACEMENT CHARACTER.
answer: CL
Source:
https://www.schwab.com/futures/crude-oil
question: 
michaelwechner commented 1 year ago

Maybe you can use my fork https://github.com/wyona/sandbox-grounded-qa/tree/michael_main where I added various log statements, in particular when you run it with verbosity 2

I am working on a PR, but will need to finish some other tasks first :-)