scripts: docs: serialize and deserialize

[johnandersen777]

Please write and return as attachments only refactored to best practices python code for a CLI script which reads markdown and includes all docs within comments and documentation within the module docstring of the file.

Guidance:

• Use stdlib whenever possible
  • Use pathlib rather than os.path
• Use PyDantic rather than dataclasses
• Enable I/O as string or JSON representing classes defined
• We MUST use newline \n instead of blank lines within strings

Tasks:

• Read markdown from string, filepath, or stdin
• Parse markdown and return a list of files and terminal commands to be run, use PyDantic to define a class for the file as a whole. Within that file we want an ordered list of sections. Use PyDantic to define classes for each section. Sections will be either documentation in text form or preformatted blocks which are preceded by their type. Ensure the document can be round tripped (deserialized and re-serialized). Provide methods to round trip with I/O to them as strings. Ensure prefix of preformatted blocks is captured as well to assist with understanding types of each block.
  • Preformatted blocks may be directly preceded (prefixed) by
  • :computer: terminal or 💻 terminal
  • :floppy_disk some/path/to/file.extension or 💾 some/path/to/file.extension
    • some/path/to/file.extension is an example, please ensure the actual path is captured

[johnandersen777]

mistletoe CONTRIBUTING.md --renderer mistletoe.ast_renderer.AstRenderer | jq -r --arg searchString "PATH" --arg excludeString "mistletoe" '.. | strings | select(contains($searchString) and (contains($excludeString) | not))' | bash -xe

[johnandersen777]

For yq extraction of example policies

python -m mistletoe docs/adrs/governance.md --renderer mistletoe.ast_renderer.AstRenderer | jq -r --arg searchString "branch_name Maintainers" --arg excludeString "mistletoe" '.. | strings | select(contains($searchString) and (contains($excludeString) | not))' | yq

To modify policy

export LOCAL_OPERATION_CACHE_SHA="$(head -n 1000 /dev/urandom | sha384sum - | awk '{print $1}')"
export LOCAL_OPERATION_CACHE_DIR="cache/operations/${LOCAL_OPERATION_CACHE_SHA}"
export DATA_PUBLIC_KEY_JSON_PATH="${LOCAL_OPERATION_CACHE_DIR}/data.public_keys.json"
export NEXT_DATA_PUBLIC_KEY_JSON_PATH="${LOCAL_OPERATION_CACHE_DIR}/next.data.public_keys.json"
mkdir -pv "${LOCAL_OPERATION_CACHE_DIR}"
echo '[{}]' > "${DATA_PUBLIC_KEY_JSON_PATH}"
jq --arg owner "$USER" '.[0].owner = $owner' "${DATA_PUBLIC_KEY_JSON_PATH}" | tee "${NEXT_DATA_PUBLIC_KEY_JSON_PATH}"
cat "${NEXT_DATA_PUBLIC_KEY_JSON_PATH}" | tee "${DATA_PUBLIC_KEY_JSON_PATH}" | jq
jq --arg public_key "$(gpg --export --armor $(git config user.email))" '.[0].public_key = $public_key' "${DATA_PUBLIC_KEY_JSON_PATH}" | tee "${NEXT_DATA_PUBLIC_KEY_JSON_PATH}"
cat "${NEXT_DATA_PUBLIC_KEY_JSON_PATH}" | tee "${DATA_PUBLIC_KEY_JSON_PATH}" | jq
python -m mistletoe docs/adrs/governance.md --renderer mistletoe.ast_renderer.AstRenderer | jq -r --arg searchString "branch_name Maintainers" --arg excludeString "mistletoe" '.. | strings | select(contains($searchString) and (contains($excludeString) | not))' | yq --indent 2 --prettyPrint '.data.public_keys = load(strenv(DATA_PUBLIC_KEY_JSON_PATH))'

Related

• 8

• https://mikefarah.gitbook.io/yq/usage/output-format