aiverify-foundation / moonshot

Moonshot - A simple and modular tool to evaluate and red-team any LLM application.
https://aiverify-foundation.github.io/moonshot/
Apache License 2.0
144 stars 27 forks source link

[MS-378] Integrate I2P for Benchmark #293

Closed imda-lionelteo closed 3 weeks ago

imda-lionelteo commented 1 month ago

Description

Integrate text to image I2P branch for benchmarking

Motivation and Context

Integrate text to image I2P branch for benchmarking This supports checking against nudenet and q16

NOTE: The numpy version must be the same as the moonshot-data or the I2P will not work as expected. An error will be thrown when running the command.

This requirement has been updated to match to disallow the error to appear

Type of Change

How to Test

Checkout both MS-378 from moonshot and moonshot-data Run the following command: python3 -m moonshot cli "run_recipe \"my new recipe runner\" \"['i2p-text2image-prompts']\" \"['openai-dalle2']\" -n 10 -r 2 -s \"You are an intelligent AI\""

Checklist

Please check all the boxes that apply to this pull request using "x":

Screenshots (if applicable)

[If the changes involve visual modifications, include screenshots or GIFs that demonstrate the changes.]

Additional Notes

[Add any additional information or context that might be relevant to reviewers.]

Developer Certificate of Origin ``` Developer Certificate of Origin Version 1.1 Copyright (C) 2004, 2006 The Linux Foundation and its contributors. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it. (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. ```
imda-benedictlee commented 2 weeks ago

Jira Ticket: https://imda-dsl.atlassian.net/browse/MS-378?atlOrigin=eyJpIjoiYmYzYTQwMGZkMjI2NDU5MGJkMmYxZWIzZWZlNmU2M2UiLCJwIjoiaiJ9