search

triton-inference-server / client

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
BSD 3-Clause "New" or "Revised" License
551 stars 227 forks source link

Add rankings support #724

Closed debermudez closed 3 months ago

debermudez commented 3 months ago

Add support for the NIM rankings API. The --input-file command is updated to take a directory or file. Directory is only support for rankings currently.

The user should pass a directory using --input-file. In that directory, must be 2 JSONL files named queries.jsonl and passages.jsonl. Genai-Perf will parse those files to generate stimulus payloads.

Sample queries.jsonl:

{"text": "What sport does America's first openly gay, actively playing, professional team sport athlete play?"}
{"text": "What production company co-owned by Kevin Loader and Rodger Michell produced My Cousin Rachel?"}
{"text": "Who served as the 1st Vice President of Colombia under El Libertador?"}

Sample passages.jsonl:

{"text": "Eric Anderson (sociologist) Eric Anderson (born January 18, 1968) is an American sociologist and sexologist specializing in adolescent men's gender and sexualities. He holds the position of Professor of Masculinities, Sexualities and Sport at the University of Winchester, in England. His research has been recognized for excellence by the British Academy of Social Sciences and he is an elected Fellow of the International Academy of Sex Research. Anderson is an advocate for the inclusion of gay men in sport and is America's first openly gay high-school coach coming out at Huntington Beach High School, the same high-school that produced the nation's first openly gay, actively playing, professional team sport athlete, Robbie Rogers who currently plays for LA Galaxy."}
{"text": "Kevin Loader Kevin Loader is a British film and television producer. Since 1996, he and co-owner Roger Michell have run a London-based production company, Free Range Films, through which the pair have made several feature films directed by Michell, including \"The Mother\", \"Enduring Love\", \"Venus\", \"Hyde Park on Hudson\", and \"Le Week-end\". Their most recent film is an adaptation by Michell of Daphne Du Maurier's \"My Cousin Rachel\". The company is also developing and producing film and television projects with other directors. Loader was awarded the Bafta for Best Television Serial in 2015 for \"The Lost Honour of Christopher Jefferies\"."}
debermudez commented 3 months ago

Left some small comments but overall LGTM! 🚀 Do you think we can also add a simple test to validate the rankings input json contents?

Added that test. Great idea.