pyinfra-dev / pyinfra

pyinfra turns Python code into shell commands and runs them on your servers. Execute ad-hoc commands and write declarative operations. Target SSH servers, local machine and Docker containers. Fast and scales from one server to thousands.
https://pyinfra.com
MIT License
3.91k stars 382 forks source link

show diff of files which will be modified with --dry #985

Open khimaros opened 1 year ago

khimaros commented 1 year ago

Is your feature request related to a problem? Please describe

it would be helpful to see specifically what changes will be made to a file before doing it. the only tool i've seen do this well is slack, but i miss it every time.

Describe the solution you'd like

when running in --dry mode (or perhaps with a new flag --diff), show the files which would be changed by a deploy and the differences between the old and new file.

Fizzadar commented 1 year ago

Love this idea. As part of tackling #805 I am going to overhaul the diff output to be more verbose (think terraform).

xvello commented 3 months ago

I managed to hack a prototype by wrapping files.put and running the diff there. It's inefficient because it needs to always pull the remote file (even if the sha1 matches), implementing it within files.put would remove this issue.

image

Here's my current code, feel free to reuse under MIT (click to unfurl). I'd be happy to try cleaning it up and turning it into a PR after we decide how the output should look and how to turn it on.   ```python import difflib from io import BytesIO from typing import IO, Any, Generator, Optional import click from pyinfra import host, logger from pyinfra.api.operation import OperationMeta from pyinfra.api.util import get_file_io from pyinfra.facts.server import Hostname from pyinfra.operations import files def put( src: str | IO[Any], dest: str, name: Optional[str] = None, present: bool = True, **kwargs: Any ) -> OperationMeta: if not present: return files.file(path=dest, present=False, name=(name or f"Delete {dest}")) current_contents = BytesIO() current_lines: list[str] = [] try: if host.get_file(dest, current_contents): current_lines = current_contents.getvalue().decode("utf-8").splitlines(keepends=True) except FileNotFoundError: pass result = files.put(src, dest, name=name, **kwargs) if result.changed: hostname = host.get_fact(Hostname) if current_lines: logger.info(f"\n Will modify {click.style(dest, bold=True)} on {hostname}:") else: logger.info(f"\n Will create {click.style(dest, bold=True)} on {hostname}:") with get_file_io(src, "r") as f: desired_lines = f.readlines() for line in generate_color_diff(current_lines, desired_lines): logger.info(f" {line}") logger.info("") return result # Customized copy of difflib.unified_diff, added color, removed diff header def generate_color_diff(current_lines: list[str], desired_lines: list[str]) -> Generator[str, None, None]: def _format_range_unified(start: int, stop: int) -> str: beginning = start + 1 # lines start numbering with one length = stop - start if length == 1: return "{}".format(beginning) if not length: beginning -= 1 # empty ranges begin at line just before the range return "{},{}".format(beginning, length) for group in difflib.SequenceMatcher(None, current_lines, desired_lines).get_grouped_opcodes(2): first, last = group[0], group[-1] file1_range = _format_range_unified(first[1], last[2]) file2_range = _format_range_unified(first[3], last[4]) yield "@@ -{} +{} @@".format(file1_range, file2_range) for tag, i1, i2, j1, j2 in group: if tag == "equal": for line in current_lines[i1:i2]: yield " " + line.rstrip() continue if tag in {"replace", "delete"}: for line in current_lines[i1:i2]: yield click.style("- " + line.rstrip(), "red") if tag in {"replace", "insert"}: for line in desired_lines[j1:j2]: yield click.style("+ " + line.rstrip(), "green") ```
olfway commented 3 months ago

I think it's pretty important feature, in ansible I always run it with --check first to review changes It's dangerous to overwrite sshd_config without checking the difference first

Also I think it should be a standard way to report differences from operations, so for example postgresql.role operation can report what changes it will do to existing role