Document CEP-5

[coderabbitai[bot]]

Introduction

This Convention Enhancement Proposal (CEP-5) seeks to generalize the use of the custom shlock implementation (tool_shlock_helper.sh) across all shell scripts within the project (excluding tool_shlock_helper.sh itself). The goal is to ensure consistent, portable, and reliable file locking mechanisms throughout the project's shell scripts.

Background

In the check_pip script, a custom shlock implementation is used to handle file locking due to inconsistencies and the lack of availability of standard shlock tools across different environments. This custom approach has proven effective in ensuring portability and reliability.

Problem Statement

Multiple shell scripts within the project may require file locking to prevent concurrent execution and ensure data integrity. However, relying on system-specific locking tools can lead to inconsistencies, failures, and increased maintenance overhead due to variations in tool availability and behavior across different environments.

[reactive-firewall]

Here is the draft we discussed

CEP-5: Standardization of Custom Locking Mechanism in Shell Scripts

Introduction

This Convention Enhancement Proposal (CEP-5) seeks to generalize the use of the custom shlock implementation (tool_shlock_helper.sh) across all shell scripts within the project (excluding tool_shlock_helper.sh itself). The goal is to ensure consistent, portable, and reliable file locking mechanisms throughout the project's shell scripts.

Background

In the check_pip script, a custom shlock implementation is used to handle file locking due to inconsistencies and the lack of availability of standard shlock tools across different environments. This custom approach has proven effective in ensuring portability and reliability.

Problem Statement

Multiple shell scripts within the project may require file locking to prevent concurrent execution and ensure data integrity. However, relying on system-specific locking tools can lead to inconsistencies, failures, and increased maintenance overhead due to variations in tool availability and behavior across different environments.

Proposal

• Adopt the Custom shlock Implementation: Standardize the use of tool_shlock_helper.sh for file locking across all shell scripts in the project (excluding tool_shlock_helper.sh itself to avoid circular dependencies).

• Implementation Guidelines:

  • Hashing tool_shlock_helper.sh: Utilize hash -p ./.github/tool_shlock_helper.sh shlock to make the custom shlock available as a shell command within scripts.

    hash -p ./.github/tool_shlock_helper.sh shlock || exit 255

  • Lock Acquisition: Use a consistent locking mechanism that checks the exit status of shlock to determine if the lock was successfully acquired.

    if [[ $(shlock -f "${LOCK_FILE}" -p $$) -eq 0 ]]; then
    # Proceed with the script
    else
    printf "%s\n" "Script already in progress by $(head "${LOCK_FILE}")" >&2
    exit 126
    fi

  • Signal Trapping: Implement trap commands to handle signals and ensure that locks are released appropriately upon script exit or interruption.

    trap 'cleanup; exit 129;' SIGHUP
    trap 'cleanup; exit 143;' SIGTERM
    trap 'cleanup; exit 131;' SIGQUIT
    trap 'cleanup; exit 130;' SIGINT
    trap 'cleanup; exit 137;' SIGABRT
    trap 'cleanup; exit ${EXIT_CODE};' EXIT

  • Lock File Management: Define a standard location and naming convention for lock files (e.g., within the TMPDIR or /tmp directory using a unique identifier).

    LOCK_FILE="${TMPDIR:-/tmp}/org.projectname.scriptname.lock"

• Documentation: Update all relevant documentation to reflect the standardization, providing clear instructions and examples for script authors.

Rationale

• Portability: By using a custom implementation, we eliminate dependencies on external tools that may not be present in all environments, ensuring our scripts run reliably everywhere.
• Consistency: A standardized approach simplifies maintenance and reduces the learning curve for new contributors.
• Reliability: Proper locking prevents race conditions and ensures that scripts do not interfere with each other during execution.

Impact

• Developers: Must update existing shell scripts to adopt the standardized locking mechanism.
• Documentation: Requires updating to include new guidelines and best practices.
• Environment Setup: Minimal impact, as the custom shlock is included in the repository and does not require external dependencies.

Implementation Plan

1. Update tool_shlock_helper.sh: Ensure the custom shlock script is up-to-date and capable of supporting the needs of all shell scripts.
2. Modify Existing Scripts: Refactor current shell scripts to implement the standardized locking mechanism.
3. Provide Examples: Include sample code snippets in the documentation to illustrate proper usage.
4. Code Review: Conduct thorough reviews to ensure compliance with the new standard.
5. Testing: Verify the updated scripts across different environments to confirm portability and reliability.

Alternatives Considered

• Using System shlock: Rejected due to inconsistent availability and differing implementations across environments.
• Employing flock or Other Locking Tools: Not viable as they may not be universally available or could introduce additional dependencies.
• Refactoring the Locking Logic: Adjusting the conditional logic could compromise the intentional design choices made for portability and might introduce bugs.
• Not Standardizing: Would allow inconsistencies and potential reliability issues to persist.

Conclusion

Standardizing the use of the custom shlock implementation across all shell scripts enhances the project's portability, consistency, and reliability. CEP-5 proposes a clear path forward to achieve this standardization, minimizing dependencies on external tools and simplifying the development process.

References

Normative References

(No normative references at this time.)

Informative References

Bash Subshells Understanding Bash Subshells shlock Manual Page PR-123

---

_Originally posted by @coderabbitai[bot] in https://github.com/reactive-firewall/multicast/pull/123#discussion_r1770695801_

[reactive-firewall]

CEP-5 has been posted as a draft gitst at: https://gist.github.com/reactive-firewall/3d2bd3cf37f87974df6f7bee31a05a89