ogre-run / miniogre

Automate the management of software dependencies with AI, to ensure your Python code runs on any computer.
https://docs.ogre.run
Apache License 2.0
43 stars 6 forks source link

[New Feature] Determine the reproducibility score of the repository. Start by evaluating the README quality. #16

Closed andrewssobral closed 5 months ago

andrewssobral commented 5 months ago

How it works for now:

$ ls
README.md  check_convex.py
$ miniogre eval --provider ollama
               _       _
   ____ ___  (_)___  (_)___  ____ _________
  / __ `__ \/ / __ \/ / __ \/ __ `/ ___/ _ \
 / / / / / / / / / / / /_/ / /_/ / /  /  __/
/_/ /_/ /_/_/_/ /_/_/\____/\__, /_/   \___/
                          /____/

miniogre - https://ogre.run

πŸ‘Ή Starting miniogre...
πŸ” Evaluating...
πŸ™ƒ Readme score: 8
βŒ› Done.

It was tested with the following providers: OpenAI, Gemini, Ollama and Groq.

README contents of the above test:

# README for Convexity Checker

The `check_convex.py` script is designed to numerically examine whether a given function is convex. This implementation is based on the definition and principles outlined in convex analysis lectures by S. Boyd at Stanford University.

## Functionality

The script includes a function called `check_convex`, which receives a function and checks its convexity numerically by generating multiple test points and evaluating the convexity condition at these points. This tool is particularly useful for initial assessments before committing to more rigorous analytical proofs.

### How It Works

- The function `check_convex` generates pairs of points along with their convex combinations.
- It then evaluates the provided function at these points to verify the convexity or quasi-convexity condition based on the configuration.
- It outputs either "not convex" with a differentiation value if a counter-example is found, or "could be convex" if no counter-examples are observed.

### Parameters

- `func`: The function to test for convexity.
- `gen`: Specifies how the test points are generated. It can be a tuple representing dimensions, or a function that generates data points.
- `max_iter`: Maximum number of iterations or trials.
- `max_inner`: Maximum number of linearly spaced values between 0 and 1 for convex combinations.
- `quasi`: Flag to indicate whether to check for quasi-convexity instead of strict convexity.
- `random_state`: Seed for the random number generation for reproducibility.
- `eps`: Tolerance level for considering deviations from convexity.
- `verbose`: Control for the level of output verbosity.

## Example Usage

The script can be executed directly to test the square of the norm function (`sqnorm`) for convexity:

`python
def sqnorm(x):
    return np.dot(x, x)

check_convex(sqnorm, gen=(5,), max_iter=10000, max_inner=10,
             random_state=0)
`

## Requirements

- Python 3
- NumPy library

## Installation

Ensure that Python 3 is installed on your system. Then, you can install NumPy using pip:

`bash
pip install numpy
`

## Additional Information

The current implementation is set up for educational and initial exploratory purposes. For formal proofs or extensive testing, further analytical or computational methods should be applied.

Contact the repository maintainers for further details or any questions regarding the script's application or potential collaborations.
wilderlopes commented 5 months ago

First iteration of reproducibility score works as expected. I am approving this merge.

Future tasks (tagging you @andrewssobral and @carolinepacheco):