[ ] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug A clear and concise description of what the bug is.

Ragas version:0.1.18 Python version:3.10.9

Code to Reproduce from agent import * from datasets import Dataset

data = { "question": ["How does Qingdao strengthen the coordinated governance of land, coast, and sea in its '14th Five-Year' ecological and environmental protection plan?"], "answer": ["In Qingdao's '14th Five-Year' ecological and environmental protection plan, the following measures are taken to strengthen the coordinated governance of land, coast, and sea:\n\n1. Establish a land-sea coordination mechanism: Strengthen the coordinated governance of coastal, river basin, and sea areas, and promote the convergence of policies and standards across regions.\n\n2. Enhance pollution prevention and ecological protection: Coordinate pollution control in river basins and sea areas to ensure the connection and interaction of ecological protection and restoration responsibilities.\n\n3. Establish a water quality monitoring mechanism: Establish a response mechanism for coastal water quality and river inflows to ensure the organic integration of water quality monitoring and pollution prevention efforts.\n\nThese measures aim to achieve continuous improvement in the marine environment, build a beautiful ocean, and enhance the public's sense of gain and happiness when accessing the sea.\n\nSource: Qingdao Municipal People's Government Notice on Issuing the '14th Five-Year' Ecological and Environmental Protection Plan of Qingdao."], "contexts": [ [ "With the protection and construction of beautiful bays as the guiding principle, we systematically plan land, coast, and sea governance tasks to promote continuous improvement in the ecological environment of coastal waters, strictly abide by the bottom line for marine biological ecology to rest and recuperate, prevent and defuse marine environmental risks, build a 'blue sea, clear sky, clean beaches' beautiful ocean, and enhance the public's sense of gain and happiness when accessing the sea.\n(I) Strengthen the coordinated governance of land, coast, and sea.\n1. Establish a land-sea coordination mechanism. Strengthen the comprehensive governance of coastal, river basin, and sea areas in a coordinated manner, enhance the convergence of ecological and environmental goals, policies, standards, and systems in coastal districts (cities), rivers flowing into the sea, and coastal waters, strengthen the convergence, coordination, and unified supervision of pollution prevention and ecological protection and restoration responsibilities in regions, river basins, and sea areas, promote the coordinated governance of river basins and sea areas, and establish a response mechanism for coastal water quality - river inflow sections - land-based pollution sources.", "Rivers and animal migration corridors, constructing an ecological spatial pattern of 'one bay with two wings, three mountains and one plain, one axis with multiple corridors'. Implement classified control of land and space, and establish an ecological and environmental zoning control system centered on ecological protection red lines, environmental quality baselines, resource utilization upper limits, and ecological and environmental access lists. Improve the management system of nature reserves with 'classified setting, hierarchical management, and zoning control', integrate and optimize various types of nature reserves, and protect biodiversity.\nPromote the construction of beautiful bays. Implement the Jiaozhou Bay Protection Regulations, strengthen the permanent protection of coastlines, strictly prohibit land reclamation, implement wetland protection, ecological restoration, marine biological resource restoration and enhancement projects, raise the threshold for industrial access in the bay area, and strengthen the assessment of water environment and ecological conditions. Establish a linkage mechanism for pollution prevention and control in river basins, rivers, and coastal waters, and implement pollution prevention and control measures in key sea areas from land sources.", "Implement strict control over beach reclamation and sea reclamation, strictly control development and utilization activities in strictly protected areas, restricted development areas, and sea areas with weak ecological vulnerability and self-purification capabilities such as Jiaozhou Bay, Dingzi Bay, Tangdao Bay, Lingshan Bay, Longwan, and other areas. Strengthen maritime law enforcement, establish a normalized working mechanism for joint maritime law enforcement, strengthen the supervision and inspection of sea use. Expand the scope of mortgage loans for sea area use rights, and explore the development of mortgage loan business for island use rights. Promote the integration of port resources and operating resources, comprehensively clean up various small and scattered ports with close locations, overlapping functions, inefficient utilization, and serious pollution, and encourage the conversion of some construction sea space into marine ecological space.\n(II) Strengthen marine ecological protection and restoration\nImplement joint prevention and control of land-sea environmental pollution, strengthen the supervision and management of direct discharge outlets into the sea, and strengthen the environmental protection of river basins and coastal waters." ], ], "ground_truths": [ ["The methods Qingdao adopts in its '14th Five-Year' ecological and environmental protection plan to strengthen the coordinated governance of land, coast, and sea include: establishing a land-sea coordination mechanism, strengthening the comprehensive and coordinated governance of coastal, river basin, and sea areas, enhancing the convergence of ecological and environmental goals, policies, standards, and systems in coastal districts (cities), rivers flowing into the sea, and coastal waters, strengthening the convergence, coordination, and unified supervision of pollution prevention and ecological protection and restoration responsibilities in regions, river basins, and sea areas, promoting the coordinated governance of river basins and sea areas, and establishing a response mechanism for coastal water quality - river inflow sections - land-based pollution sources."] ] } dataset = Dataset.from_dict(data)

使用RAGAs进行评估

from ragas import evaluate from ragas.metrics import ( faithfulness, answer_relevancy, context_recall, context_precision, )

result = evaluate( dataset, metrics=[ context_precision, context_recall, faithfulness, answer_relevancy, ], )

print(result)

Error trace Traceback (most recent call last): File "D:.study\myProject\hh.py", line 32, in result = evaluate( File "F:\Application2022\Anaconda\an\lib\site-packages\ragas_analytics.py", line 129, in wrapper result = func(*args, **kwargs) File "F:\Application2022\Anaconda\an\lib\site-packages\ragas\evaluation.py", line 177, in evaluate validate_required_columns(dataset, metrics) File "F:\Application2022\Anaconda\an\lib\site-packages\ragas\validation.py", line 62, in validate_required_columns raise ValueError( ValueError: The metric [context_precision] that that is used requires the following additional columns ['reference'] to be present in the dataset.

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

I found a similar open issue that might help you resolve the ValueError related to the 'context_precision' metric requiring the 'reference' column. You can check it out here.

To resolve this error, you need to ensure that your dataset includes the 'reference' column. The ContextPrecision class requires the 'reference' column as part of its _required_columns. Here is an example of how your dataset should look:

{
    "question": ["How does Qingdao strengthen the coordinated governance of land, coast, and sea in its '14th Five-Year' ecological and environmental protection plan?"],
    "answer": ["In Qingdao's '14th Five-Year' ecological and environmental protection plan, the following measures are taken to strengthen the coordinated governance of land, coast, and sea:\n\n1. **Establish a land-sea coordination mechanism**: Strengthen the coordinated governance of coastal, river basin, and sea areas, and promote the convergence of policies and standards across regions.\n\n2. **Enhance pollution prevention and ecological protection**: Coordinate pollution control in river basins and sea areas to ensure the connection and interaction of ecological protection and restoration responsibilities.\n\n3. **Establish a water quality monitoring mechanism**: Establish a response mechanism for coastal water quality and river inflows to ensure the organic integration of water quality monitoring and pollution prevention efforts.\n\nThese measures aim to achieve continuous improvement in the marine environment, build a beautiful ocean, and enhance the public's sense of gain and happiness when accessing the sea.\n\nSource: Qingdao Municipal People's Government Notice on Issuing the '14th Five-Year' Ecological and Environmental Protection Plan of Qingdao."],
    "contexts": [
        [
            "With the protection and construction of beautiful bays as the guiding principle, we systematically plan land, coast, and sea governance tasks to promote continuous improvement in the ecological environment of coastal waters, strictly abide by the bottom line for marine biological ecology to rest and recuperate, prevent and defuse marine environmental risks, build a 'blue sea, clear sky, clean beaches' beautiful ocean, and enhance the public's sense of gain and happiness when accessing the sea.\n(I) Strengthen the coordinated governance of land, coast, and sea.\n1. Establish a land-sea coordination mechanism. Strengthen the comprehensive governance of coastal, river basin, and sea areas in a coordinated manner, enhance the convergence of ecological and environmental goals, policies, standards, and systems in coastal districts (cities), rivers flowing into the sea, and coastal waters, strengthen the convergence, coordination, and unified supervision of pollution prevention and ecological protection and restoration responsibilities in regions, river basins, and sea areas, promote the coordinated governance of river basins and sea areas, and establish a response mechanism for coastal water quality - river inflow sections - land-based pollution sources.",
            "Rivers and animal migration corridors, constructing an ecological spatial pattern of 'one bay with two wings, three mountains and one plain, one axis with multiple corridors'. Implement classified control of land and space, and establish an ecological and environmental zoning control system centered on ecological protection red lines, environmental quality baselines, resource utilization upper limits, and ecological and environmental access lists. Improve the management system of nature reserves with 'classified setting, hierarchical management, and zoning control', integrate and optimize various types of nature reserves, and protect biodiversity.\nPromote the construction of beautiful bays. Implement the Jiaozhou Bay Protection Regulations, strengthen the permanent protection of coastlines, strictly prohibit land reclamation, implement wetland protection, ecological restoration, marine biological resource restoration and enhancement projects, raise the threshold for industrial access in the bay area, and strengthen the assessment of water environment and ecological conditions. Establish a linkage mechanism for pollution prevention and control in river basins, rivers, and coastal waters, and implement pollution prevention and control measures in key sea areas from land sources.",
            "Implement strict control over beach reclamation and sea reclamation, strictly control development and utilization activities in strictly protected areas, restricted development areas, and sea areas with weak ecological vulnerability and self-purification capabilities such as Jiaozhou Bay, Dingzi Bay, Tangdao Bay, Lingshan Bay, Longwan, and other areas. Strengthen maritime law enforcement, establish a normalized working mechanism for joint maritime law enforcement, strengthen the supervision and inspection of sea use. Expand the scope of mortgage loans for sea area use rights, and explore the development of mortgage loan business for island use rights. Promote the integration of port resources and operating resources, comprehensively clean up various small and scattered ports with close locations, overlapping functions, inefficient utilization, and serious pollution, and encourage the conversion of some construction sea space into marine ecological space.\n(II) Strengthen marine ecological protection and restoration\nImplement joint prevention and control of land-sea environmental pollution, strengthen the supervision and management of direct discharge outlets into the sea, and strengthen the environmental protection of river basins and coastal waters."
        ]
    ],
    "ground_truths": [
        ["The methods Qingdao adopts in its '14th Five-Year' ecological and environmental protection plan to strengthen the coordinated governance of land, coast, and sea include: establishing a land-sea coordination mechanism, strengthening the comprehensive and coordinated governance of coastal, river basin, and sea areas, enhancing the convergence of ecological and environmental goals, policies, standards, and systems in coastal districts (cities), rivers flowing into the sea, and coastal waters, strengthening the convergence, coordination, and unified supervision of pollution prevention and ecological protection and restoration responsibilities in regions, river basins, and sea areas, promoting the coordinated governance of river basins and sea areas, and establishing a response mechanism for coastal water quality - river inflow sections - land-based pollution sources."]
    ],
    "reference": ["The methods Qingdao adopts in its '14th Five-Year' ecological and environmental protection plan to strengthen the coordinated governance of land, coast, and sea include: establishing a land-sea coordination mechanism, strengthening the comprehensive and coordinated governance of coastal, river basin, and sea areas, enhancing the convergence of ecological and environmental goals, policies, standards, and systems in coastal districts (cities), rivers flowing into the sea, and coastal waters, strengthening the convergence, coordination, and unified supervision of pollution prevention and ecological protection and restoration responsibilities in regions, river basins, and sea areas, promoting the coordinated governance of river basins and sea areas, and establishing a response mechanism for coastal water quality - river inflow sections - land-based pollution sources."]
}

Ensure that each row in your dataset includes these columns: 'question', 'answer', 'contexts', 'ground_truths', and 'reference'. This will satisfy the requirements of the ContextPrecision metric and prevent the ValueError ^[1].

To continue talking to Dosu, mention @dosu.

Please refer to my answer in #1299

explodinggradients / ragas

ValueError: The metric [context_precision] that that is used requires the following additional columns ['reference'] to be present in the dataset. #1310

使用RAGAs进行评估