[Feature Request] Custom Prompts

Hi, I guess this may be no surprise, but I've noticed that the current prompts do not work properly with models like Claude.

Example

Formatted prompt:

Given a context, and an answer, analyze each sentence in the answer and classify if the sentence can be attributed to the given context or not.
Think in steps and reason before coming to conclusion. 

context: Albert Einstein (14 March 1879 – 18 April 1955) was a German-born theoretical physicist,widely held to be one of the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century. His mass–energy equivalence formula E = mc2, which arises from relativity theory, has been called "the world's most famous equation". He received the 1921 Nobel Prize in Physics "for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect", a pivotal step in the development of quantum theory. His work is also known for its influence on the philosophy of science. In a 1999 poll of 130 leading physicists worldwide by the British journal Physics World, Einstein was ranked the greatest physicist of all time. His intellectual achievements and originality have made Einstein synonymous with genius.
answer: Albert Einstein born in 14 March 1879 was  German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. He received the 1921 Nobel Prize in Physics "for his services to theoretical physics. He published 4 papers in 1905.  Einstein moved to Switzerland in 1895 
classification
1. Albert Einstein born in 14 March 1879 was  German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. The date of birth of Einstein is mentioned clearly in the context. So [Attributed]
2. He received the 1921 Nobel Prize in Physics "for his services to theoretical physics. The exact sentence is present in the given context. So [Attributed]
3. He published 4 papers in 1905. There is no mention about papers he wrote in given the context. So [Not Attributed]
4. Einstein moved to Switzerland in 1895. There is not supporting evidence for this in the given the context. So [Not Attributed]

context:from 1785 until 1790, and has been the largest U.S. city since 1790. The Statue of Liberty greeted millions of immigrants as they came to the U.S. by ship in the late 19th and early 20th centuries, and is a symbol of the U.S. and its ideals of liberty and peace. In the 21st century, New York City has emerged as a global node of creativity, entrepreneurship, and as a symbol of freedom and cultural diversity. The New York Times has won the most Pulitzer Prizes for journalism and remains the U.S. media's "newspaper of record". In 2019, New York City was voted the greatest city in the world in a survey of over 30,000 people from 48 cities worldwide, citing its cultural diversity.Many districts and monuments in New York City are major landmarks, including three of the world's ten most visited tourist attractions in 2013. A record 66.6 million tourists visited New York City in 2019. Times Square is the brightly illuminated hub of the Broadway Theater District, one of the world's busiest
The Statue of Liberty National Monument and Ellis Island Immigration Museum are managed by the National Park Service and are in both New York and New Jersey. They are joined in the harbor by Governors Island National Monument. Historic sites under federal management on Manhattan Island include Stonewall National Monument; Castle Clinton National Monument; Federal Hall National Memorial; Theodore Roosevelt Birthplace National Historic Site; General Grant National Memorial (Grant's Tomb); African Burial Ground National Monument; and Hamilton Grange National Memorial. Hundreds of properties are listed on the National Register of Historic Places or as a National Historic Landmark.
Tourism is a vital industry for New York City, and NYC & Company represents the city's official bureau of tourism. New York has witnessed a growing combined volume of international and domestic tourists, reflecting over 60 million visitors to the city per year, the world's busiest tourist destination. Approximately 12 million visitors to New York City have been from outside the United States, with the highest numbers from the United Kingdom, Canada, Brazil, and China. Multiple sources have called New York the most photographed city in the world.I Love New York (stylized I ❤ NY) is both a logo and a song that are the basis of an advertising campaign and have been used since 1977 to promote tourism in New York City, and later to promote New York State as well. The trademarked logo, owned by New York State Empire State Development, appears in souvenir shops and brochures throughout the city and state, some licensed, many not. The song is the state song of New York.
pedestrian intersections and a major center of the world's entertainment industry. Many of the city's landmarks, skyscrapers, and parks are known around the world, and the city's fast pace led to the phrase New York minute. The Empire State Building is a global standard of reference to describe the height and length of other structures.Manhattan's real estate market is among the most expensive in the world. Providing continuous 24/7 service and contributing to the nickname The City That Never Sleeps, the New York City Subway is the largest single-operator rapid transit system in the world with 472 passenger rail stations, and Penn Station in Midtown Manhattan is the busiest transportation hub in the Western Hemisphere. The city has over 120 colleges and universities, including Columbia University, an Ivy League university routinely ranked among the world's top universities, New York University, and the City University of New York system, the largest urban public university system in
answer:The Statue of Liberty in New York City holds great significance as a symbol of the United States and its ideals of liberty and peace. It greeted millions of immigrants who arrived in the U.S. by ship in the late 19th and early 20th centuries, representing hope and freedom for those seeking a better life. It has since become an iconic landmark and a global symbol of cultural diversity and freedom.
classification:

Response:

Here are the classifications for each sentence in the answer:

1. The Statue of Liberty in New York City holds great significance as a symbol of the United States and its ideals of liberty and peace. [Attributed] - The context mentions the Statue of Liberty as a symbol of the U.S. and its ideals of liberty and peace.

2. It greeted millions of immigrants who arrived in the U.S. by ship in the late 19th and early 20th centuries, representing hope and freedom for those seeking a better life. [Attributed] - The context mentions the Statue of Liberty greeting immigrants in the late 19th and early 20th centuries.

3. It has since become an iconic landmark and a global symbol of cultural diversity and freedom. [Not Attributed] - The context does not mention the Statue of Liberty becoming an iconic global symbol.

In summary, the first two sentences can be attributed to the given context, while the third sentence cannot be attributed based on the information provided.

Analyzing the response sentence by sentence:

sentence:  Here are the classifications for each sentence in the answer:
sentence: 
sentence: 1. The Statue of Liberty in New York City holds great significance as a symbol of the United States and its ideals of liberty and peace. [Attributed] - The context mentions the Statue of Liberty as a symbol of the U.S. and its ideals of liberty and peace.
sentence: 
sentence: 2. It greeted millions of immigrants who arrived in the U.S. by ship in the late 19th and early 20th centuries, representing hope and freedom for those seeking a better life. [Attributed] - The context mentions the Statue of Liberty greeting immigrants in the late 19th and early 20th centuries.
sentence: 
sentence: 3. It has since become an iconic landmark and a global symbol of cultural diversity and freedom. [Not Attributed] - The context does not mention the Statue of Liberty becoming an iconic global symbol.
sentence: 
sentence: In summary, the first two sentences can be attributed to the given context, while the third sentence cannot be attributed based on the information provided.

denom value: 9
final score: 0.2222222222222222

I've edited and successfully tested new prompts for Claude models and put them here: https://github.com/austinmw/ragas/tree/bedrock

I think it would be good to have a feature to customize the prompts without editing the source code? Thanks! 🙏🏻

explodinggradients / ragas

[Feature Request] Custom Prompts #245

Example