search

argilla-io / distilabel

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.
https://distilabel.argilla.io
Apache License 2.0
1.45k stars 111 forks source link

UltraFeedback error with TogetherAI #763

Closed sameearif closed 3 months ago

sameearif commented 3 months ago

ultrafeedback = UltraFeedback( name="ultrafeedback", llm=TogetherLLM(model="meta-llama/Llama-3-70b-chat-hf", base_url="https://api.together.xyz/v1", generation_kwargs= {"max_new_tokens": 512, "temperature": 0.7}), aspect="overall-rating", output_mappings={"model_name": "ultrafeedback_model"}, )

ultrafeedback.load()

result = next( ultrafeedback.process( [ { "instruction": instruction "generations": [ out1, out2 ] } ] ) )

Error: BadRequestError: Error code: 400 - {'error': {'message': 'json_object is the only supported response_format type', 'type': 'invalid_request_error', 'param': 'response_format', 'code': None}}

gabrielmbmb commented 3 months ago

Hi @sameearif, thanks for reporting this bug! It has been fixed and we will release a new version.

gabrielmbmb commented 3 months ago

@sameearif we've just released 1.2.1, it should work now :)

sameearif commented 2 months ago

@gabrielmbmb Thank you so much for the fast response!