search

pyinfra-dev / pyinfra

pyinfra turns Python code into shell commands and runs them on your servers. Execute ad-hoc commands and write declarative operations. Target SSH servers, local machine and Docker containers. Fast and scales from one server to thousands.
https://pyinfra.com
MIT License
3.85k stars 374 forks source link

CycleError is still here #911

Closed julienlavergne closed 1 year ago

julienlavergne commented 1 year ago

Describe the bug

The is still some CycleErrors when using embedded loops. Especially, it looks like the host.loop_position is shared across all hosts.

To Reproduce

I can consistently reproduce the issue with following simple script and 2 hosts.

import random
import time

from pyinfra import host
from pyinfra.operations import server

for i in host.loop(range(0, 3)):
    for j in host.loop(range(0, 3)):
        print(f"{host.name} {host.loop_position}")
        server.shell(name=f"ls A", commands="ls")

        if random.randint(0, 1) == 0:
            time.sleep(1)
            server.shell(name=f"ls B", commands="ls")

        server.shell(name=f"ls C", commands="ls")

The print will show something going wrong in the handling for host.loop_position.

Meta

v2.5.2

Output of previous example shows issues in host.loop_position.

--> Preparing Operations...
    Loading: tasks/loop_test.py
host08 [0, 0]
host08 [0, 1]
host08 [0, 2]
host07 [0, 2, 0, 0]
host07 [0, 2, 0, 1]
host08 [0, 2, 1, 0]
host08 [0, 2, 1, 1]
host07 [0, 2, 1, 2]
host07 [0, 2, 1, 0]
host07 [0, 2, 1, 1]
host08 [0, 2, 1, 2]
host07 [0, 2, 1, 2]
host08 [0, 2, 2, 0]
host08 [0, 2, 2, 1]
host07 [0, 2, 2, 0]
host07 [0, 2, 2, 1]
host08 [0, 2, 2, 2]
    [host08] Ready: tasks/loop_test.py
host07 [0, 2]
    [host07] Ready: tasks/loop_test.py

--> Proposed changes:
    Groups: * / *
    [host08]   Operations: 22   Change: 22   No change: 0
    [host07]   Operations: 23   Change: 23   No change: 0

--> Beginning operation run...
Cycle detected in operation ordering DAG.
    Error: ('nodes are in a cycle', ['16914cca7f66c3611e9d2a416a7d621a277ee2c3', 'f4a17e92c61121c85c77d34bc15544d0679d5c50', '30096631874118bdb1cfbeaa427c57b569521c25', '7fa4d2f28e0ae70166e7b1a885755610166d49f2', '1552cc4ed1d48843b07f6e66ac114c6dfee0febe-0', '1e668c25af2946343637a2a0ee48d1b3dbec151d-0', '14a0639245fafec20a20e3c702021c28baca8a49', '5746348464227d74adbd59e0b377451a397f00b8', 'bbb49c1789858e4aafb6f0e471a1c2658512f809', '8eeaf6fd0106f688e69270ab5110c006b9dfa894', '75605df4dcce58cd84e134dd1fbbcc4f87c782ba', '67d0504000be158e31955ca4433465ff6fbb549c', '6d7d88a24e3b87c7d02003429c70be1a8f3a15ba', '12a41289448f557d0cd296011d41a36e3fd73bb2', '47155dc5a19d4758dac71657aac9458106fa8ecc', '5c96e4db6ad35ebd092fb183629e0752f9211fce', '16914cca7f66c3611e9d2a416a7d621a277ee2c3'])

    This can happen when using loops in operation code, please see: https://docs.pyinfra.com/en/latest/deploy-process.html#loops-cycle-errors
Fizzadar commented 1 year ago

Gah this is a mutable class attribute 🤦 - https://github.com/Fizzadar/pyinfra/commit/6dc9fb8cee693e73a2c077be70fd0d7fcd983193. Will roll fix in 2.5.3 shortly.