0xSage
commented
2 months ago Thanks for this. Would be great to have a publicly viewable dashboard as well 🙏
Open Van-QA opened 2 months ago
0xSage
commented
2 months ago Thanks for this. Would be great to have a publicly viewable dashboard as well 🙏
Problem: Currently, there is no seamless way to run checks on our API collection using OpenAI's tools and sync the output to a Report Portal for coverage analysis.
Success Criteria:
Additional Context: The proposed feature would allow us to streamline our testing and analysis by automating the process of running checks on our API collection using OpenAI' tools. Additionally, the ability to sync the output from these checks to a Report Portal would enable us to easily and analyze coverage, making it easier to identify areas for improvement and optimize our API collection.
This feature would be particularly useful for our CI/CD pipeline, as would allow us to automatically run checks on our API collection as part of our testing process, and ensure that all changes and updates to the API collection are thoroughly tested and analyzed for coverage.
Overall, this feature would greatly improve our ability to maintain and optimize our API collection, and ensure that it meets the standard of OpenAI API
Sub tasks:
[x] Add new workflow test OpenAI API by @hiento09
[x] Conftest for Python file name marker to filter coverage by Python file name in report portal @Van-QA
[x] Conftest for Mapper with Endpoint @Van-QA (to filter coverage by endpoints in report portal
[x] Workflow dispatch to run by specific Endpoint (single / multiple endpoints / all) @Van-QA
[ ] Collect JSON data via coverage API of report portal and push to S3 @hiento09 Please use this cURL for the request cURL_coverage.txt
[ ] janhq/docs#219 Sample template for reference:
[ ] Relocate test runner to Cortex ( blocked until the refactor from dev team)