phasehq / console

Application secrets and configuration management for developers.
https://phase.dev
Other
453 stars 24 forks source link

fix: share corruption test #345

Closed nimish-ks closed 1 month ago

nimish-ks commented 1 month ago

:mag: Overview

Secret sharding test suite was experiencing frequent failures in the "cannot reconstruct secret with a corrupted share" test. This PR fixes it.

:bulb: Proposed Changes

:test_tube: Testing

:sparkles: How to Test the Changes Locally

  1. Check out this branch
  2. Run yarn install to ensure all dependencies are up to date
  3. To thoroughly test the consistency of the changes, you can use the following script:
#!/bin/bash

NUM_RUNS=200
FAILURES=0
FAILURE_TIMES=()
ERROR_LOG="error_log.txt"

# Clear the error log at the start
> "$ERROR_LOG"

start_time=$(date +%s)

for i in $(seq 1 $NUM_RUNS); do
    run_start=$(date +%s)
    OUTPUT=$(yarn test tests/utils/crypto/secretSharding.test.ts 2>&1)

    if echo "$OUTPUT" | grep -q "FAIL"; then
        FAILURES=$((FAILURES + 1))
        FAILURE_TIMES+=($((run_start - start_time)))
        echo "Run $i: FAILED at $((run_start - start_time)) seconds"

        # Log the entire output for failed tests
        echo "Error in Run $i:" >> "$ERROR_LOG"
        echo "$OUTPUT" >> "$ERROR_LOG"
        echo "----------------------------------------" >> "$ERROR_LOG"
    else
        echo "Run $i: PASSED"
    fi
done

FAILURE_RATE=$(echo "scale=4; $FAILURES / $NUM_RUNS" | bc)

echo "Total runs: $NUM_RUNS"
echo "Total failures: $FAILURES"
echo "Failure rate: $FAILURE_RATE"

if [ ${#FAILURE_TIMES[@]} -gt 0 ]; then
    echo "Failure times: ${FAILURE_TIMES[*]}"

    # Calculate average time between failures
    total_time=0
    for time in "${FAILURE_TIMES[@]}"; do
        total_time=$((total_time + time))
    done
    avg_time=$(echo "scale=2; $total_time / $FAILURES" | bc)
    echo "Average time between failures: $avg_time seconds"
    echo "Detailed error messages have been logged to $ERROR_LOG"
else
    echo "No failures occurred."
fi

To use this script: a. Save the script to a file (e.g., run_tests.sh) in your project directory b. Make the script executable: chmod +x run_tests.sh c. Run the script: ./run_tests.sh

  1. The script will run the test 200 times and provide a summary of the results, including:

    • Total number of runs
    • Number of failures
    • Failure rate
    • Average time between failures (if any)
    • Detailed error messages for any failures (logged in error_log.txt)
  2. Verify that the tests pass consistently across these multiple runs. The expected outcome is zero failures.