saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Install Salt from the Salt package repositories here:
https://docs.saltproject.io/salt/install-guide/en/latest/
Apache License 2.0
14.21k stars 5.48k forks source link

IndexError: list index out of range when using file.blockreplace with jinja template #46492

Closed spyd3r closed 5 years ago

spyd3r commented 6 years ago

Description of Issue/Question

An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/site-packages/salt/state.py", line 1851, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1795, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/site-packages/salt/states/file.py", line 4258, in blockreplace
                  show_changes=show_changes
                File "/usr/lib/python2.7/site-packages/salt/modules/file.py", line 2404, in blockreplace
                  log.debug("SPLIT_CONTENT[-1]".format(split_content[-1]))
              IndexError: list index out of range

Setup

SLS:

testing:
  file.blockreplace:
    - name: /tmp/testing.txt
    - marker_start: "# [START] Salt managed searches."
    - marker_end: "[END Salt managed searches - DO NOT REMOVE]"
    - append_if_not_found: True
    - backup: '.bak'
    - source: salt://template
    - template: jinja

TEMPLATE:

{%- if name is defined -%}
{{ name.get('first', '') }}
{{ name.get('last', '') }}
More
random
lines
{%- endif -%}

/TMP/TESTING.TXT:

# [START] Salt managed searches.
[END Salt managed searches - DO NOT REMOVE]

Steps to Reproduce Issue

If no defaults are passed in through testing.sls, the first run results in "More\nrandom\nlines" being added to /tmp/testing.txt. Any subsequent runs result in the index out of range error. Passing in the following defaults avoids the error message: defaults: name: first: bob last: dobbs

Versions Report

Salt Version: Salt: 2017.7.4

Dependency Versions: cffi: Not Installed cherrypy: Not Installed dateutil: Not Installed docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed ioflo: Not Installed Jinja2: 2.7.2 libgit2: Not Installed libnacl: Not Installed M2Crypto: 0.21.1 Mako: Not Installed msgpack-pure: Not Installed msgpack-python: 0.4.6 mysql-python: Not Installed pycparser: Not Installed pycrypto: 2.6.1 pycryptodome: Not Installed pygit2: Not Installed Python: 2.7.5 (default, Nov 20 2015, 02:00:19) python-gnupg: Not Installed PyYAML: 3.10 PyZMQ: 14.3.1 RAET: Not Installed smmap: Not Installed timelib: Not Installed Tornado: 4.2.1 ZMQ: 3.2.5

System Versions: dist: centos 7.2.1511 Core locale: ANSI_X3.4-1968 machine: x86_64 system: Linux version: CentOS Linux 7.2.1511 Core

garethgreenaway commented 6 years ago

@spyd3r Thanks for the report. I think there might be something missing in the issue as I'm unable to reproduce.

spyd3r commented 6 years ago

Here's my step-by-step on how I triggered the error. Running on a default CentOS7 vagrant VM.


[vagrant@ip-10-0-2-15 ~]$ sudo -i
[root@ip-10-0-2-15 ~]# yum install https://repo.saltstack.com/yum/redhat/salt-repo-latest-2.el7.noarch.rpm -y
Loaded plugins: fastestmirror
salt-repo-latest-2.el7.noarch.rpm                                                                                                                             | 5.6 kB  00:00:00
Examining /var/tmp/yum-root-aUc3Ud/salt-repo-latest-2.el7.noarch.rpm: salt-repo-latest-2.el7.noarch
Marking /var/tmp/yum-root-aUc3Ud/salt-repo-latest-2.el7.noarch.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package salt-repo.noarch 0:latest-2.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================================================================================================================
 Package                               Arch                               Version                                   Repository                                                  Size
=====================================================================================================================================================================================
Installing:
 salt-repo                             noarch                             latest-2.el7                              /salt-repo-latest-2.el7.noarch                             3.6 k

Transaction Summary
=====================================================================================================================================================================================
Install  1 Package

Total size: 3.6 k
Installed size: 3.6 k
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : salt-repo-latest-2.el7.noarch                                                                                                                                     1/1
  Verifying  : salt-repo-latest-2.el7.noarch                                                                                                                                     1/1

Installed:
  salt-repo.noarch 0:latest-2.el7

Complete!
[root@ip-10-0-2-15 ~]# yum install -y salt-minion
Loaded plugins: fastestmirror
salt-latest                                                                                                                                                   | 2.9 kB  00:00:00
salt-latest/x86_64/primary_db                                                                                                                                 |  38 kB  00:00:01
Loading mirror speeds from cached hostfile
 * base: mirrors.xmission.com
 * extras: mirrors.sonic.net
 * updates: mirror.compevo.com
Resolving Dependencies
--> Running transaction check
---> Package salt-minion.noarch 0:2017.7.4-1.el7 will be installed
--> Processing Dependency: salt = 2017.7.4-1.el7 for package: salt-minion-2017.7.4-1.el7.noarch
--> Running transaction check
---> Package salt.noarch 0:2017.7.4-1.el7 will be installed
--> Processing Dependency: python-tornado < 5.0 for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-msgpack > 0.3 for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-tornado >= 4.2.1 for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-requests >= 1.0.0 for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-futures >= 2.0 for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-crypto >= 2.6.1 for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: systemd-python for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-zmq for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-six for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-psutil for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-markupsafe for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: python-jinja2 for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: pciutils for package: salt-2017.7.4-1.el7.noarch
--> Processing Dependency: PyYAML for package: salt-2017.7.4-1.el7.noarch
--> Running transaction check
---> Package PyYAML.x86_64 0:3.11-1.el7 will be installed
--> Processing Dependency: libyaml-0.so.2()(64bit) for package: PyYAML-3.11-1.el7.x86_64
---> Package pciutils.x86_64 0:3.5.1-2.el7 will be installed
---> Package python-futures.noarch 0:3.0.3-1.el7 will be installed
---> Package python-jinja2.noarch 0:2.7.2-2.el7 will be installed
--> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.7.2-2.el7.noarch
---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed
---> Package python-msgpack.x86_64 0:0.4.6-1.el7 will be installed
---> Package python-psutil.x86_64 0:2.2.1-1.el7 will be installed
---> Package python-requests.noarch 0:2.6.0-1.el7_1 will be installed
--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-1.el7_1.noarch
---> Package python-six.noarch 0:1.9.0-2.el7 will be installed
---> Package python-tornado.x86_64 0:4.2.1-1.el7 will be installed
--> Processing Dependency: python-backports-ssl_match_hostname for package: python-tornado-4.2.1-1.el7.x86_64
---> Package python-zmq.x86_64 0:15.3.0-2.el7 will be installed
--> Processing Dependency: libzmq.so.5()(64bit) for package: python-zmq-15.3.0-2.el7.x86_64
---> Package python2-crypto.x86_64 0:2.6.1-15.el7 will be installed
--> Processing Dependency: libtomcrypt.so.0()(64bit) for package: python2-crypto-2.6.1-15.el7.x86_64
---> Package systemd-python.x86_64 0:219-42.el7_4.10 will be installed
--> Processing Dependency: systemd-libs = 219-42.el7_4.10 for package: systemd-python-219-42.el7_4.10.x86_64
--> Processing Dependency: systemd = 219-42.el7_4.10 for package: systemd-python-219-42.el7_4.10.x86_64
--> Running transaction check
---> Package libtomcrypt.x86_64 0:1.17-26.el7 will be installed
--> Processing Dependency: libtommath >= 0.42.0 for package: libtomcrypt-1.17-26.el7.x86_64
--> Processing Dependency: libtommath.so.0()(64bit) for package: libtomcrypt-1.17-26.el7.x86_64
---> Package libyaml.x86_64 0:0.1.4-11.el7_0 will be installed
---> Package python-babel.noarch 0:0.9.6-8.el7 will be installed
---> Package python-backports-ssl_match_hostname.noarch 0:3.4.0.2-4.el7 will be installed
--> Processing Dependency: python-backports for package: python-backports-ssl_match_hostname-3.4.0.2-4.el7.noarch
---> Package python-urllib3.noarch 0:1.10.2-3.el7 will be installed
---> Package systemd.x86_64 0:219-42.el7_4.4 will be updated
--> Processing Dependency: systemd = 219-42.el7_4.4 for package: systemd-sysv-219-42.el7_4.4.x86_64
---> Package systemd.x86_64 0:219-42.el7_4.10 will be an update
---> Package systemd-libs.x86_64 0:219-42.el7_4.4 will be updated
---> Package systemd-libs.x86_64 0:219-42.el7_4.10 will be an update
---> Package zeromq.x86_64 0:4.1.4-6.el7 will be installed
--> Processing Dependency: libsodium.so.13()(64bit) for package: zeromq-4.1.4-6.el7.x86_64
--> Processing Dependency: libpgm-5.2.so.0()(64bit) for package: zeromq-4.1.4-6.el7.x86_64
--> Running transaction check
---> Package libsodium.x86_64 0:1.0.5-1.el7 will be installed
---> Package libtommath.x86_64 0:0.42.0-6.el7 will be installed
---> Package openpgm.x86_64 0:5.2.122-2.el7 will be installed
---> Package python-backports.x86_64 0:1.0-8.el7 will be installed
---> Package systemd-sysv.x86_64 0:219-42.el7_4.4 will be updated
---> Package systemd-sysv.x86_64 0:219-42.el7_4.10 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================================================================================================================
 Package                                                      Arch                            Version                                     Repository                            Size
=====================================================================================================================================================================================
Installing:
 salt-minion                                                  noarch                          2017.7.4-1.el7                              salt-latest                           35 k
Installing for dependencies:
 PyYAML                                                       x86_64                          3.11-1.el7                                  salt-latest                          160 k
 libsodium                                                    x86_64                          1.0.5-1.el7                                 salt-latest                          143 k
 libtomcrypt                                                  x86_64                          1.17-26.el7                                 extras                               224 k
 libtommath                                                   x86_64                          0.42.0-6.el7                                extras                                36 k
 libyaml                                                      x86_64                          0.1.4-11.el7_0                              base                                  55 k
 openpgm                                                      x86_64                          5.2.122-2.el7                               salt-latest                          172 k
 pciutils                                                     x86_64                          3.5.1-2.el7                                 base                                  93 k
 python-babel                                                 noarch                          0.9.6-8.el7                                 base                                 1.4 M
 python-backports                                             x86_64                          1.0-8.el7                                   base                                 5.8 k
 python-backports-ssl_match_hostname                          noarch                          3.4.0.2-4.el7                               base                                  12 k
 python-futures                                               noarch                          3.0.3-1.el7                                 salt-latest                           26 k
 python-jinja2                                                noarch                          2.7.2-2.el7                                 base                                 515 k
 python-markupsafe                                            x86_64                          0.11-10.el7                                 base                                  25 k
 python-msgpack                                               x86_64                          0.4.6-1.el7                                 salt-latest                           73 k
 python-psutil                                                x86_64                          2.2.1-1.el7                                 salt-latest                          114 k
 python-requests                                              noarch                          2.6.0-1.el7_1                               base                                  94 k
 python-six                                                   noarch                          1.9.0-2.el7                                 base                                  29 k
 python-tornado                                               x86_64                          4.2.1-1.el7                                 base                                 636 k
 python-urllib3                                               noarch                          1.10.2-3.el7                                base                                 101 k
 python-zmq                                                   x86_64                          15.3.0-2.el7                                salt-latest                          520 k
 python2-crypto                                               x86_64                          2.6.1-15.el7                                extras                               477 k
 salt                                                         noarch                          2017.7.4-1.el7                              salt-latest                          7.9 M
 systemd-python                                               x86_64                          219-42.el7_4.10                             updates                              118 k
 zeromq                                                       x86_64                          4.1.4-6.el7                                 salt-latest                          555 k
Updating for dependencies:
 systemd                                                      x86_64                          219-42.el7_4.10                             updates                              5.2 M
 systemd-libs                                                 x86_64                          219-42.el7_4.10                             updates                              378 k
 systemd-sysv                                                 x86_64                          219-42.el7_4.10                             updates                               72 k

Transaction Summary
=====================================================================================================================================================================================
Install  1 Package  (+24 Dependent packages)
Upgrade             (  3 Dependent packages)

Total download size: 19 M
Downloading packages:
Delta RPMs reduced 5.5 M of updates to 2.3 M (58% saved)
(1/28): libtomcrypt-1.17-26.el7.x86_64.rpm                                                                                                                    | 224 kB  00:00:00
(2/28): libtommath-0.42.0-6.el7.x86_64.rpm                                                                                                                    |  36 kB  00:00:00
(3/28): libyaml-0.1.4-11.el7_0.x86_64.rpm                                                                                                                     |  55 kB  00:00:00
(4/28): systemd-219-42.el7_4.4_219-42.el7_4.10.x86_64.drpm                                                                                                    | 2.2 MB  00:00:00
/usr/share/man/man8/udevadm.8.gz: No such file or directory
cannot reconstruct rpm from disk files
warning: /var/cache/yum/x86_64/7/salt-latest/packages/libsodium-1.0.5-1.el7.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID de57bfbe: NOKEY       ] 1.5 MB/s | 2.5 MB  00:00:08 ETA
Public key for libsodium-1.0.5-1.el7.x86_64.rpm is not installed
(5/28): libsodium-1.0.5-1.el7.x86_64.rpm                                                                                                                      | 143 kB  00:00:01
(6/28): pciutils-3.5.1-2.el7.x86_64.rpm                                                                                                                       |  93 kB  00:00:00
(7/28): python-backports-1.0-8.el7.x86_64.rpm                                                                                                                 | 5.8 kB  00:00:00
(8/28): PyYAML-3.11-1.el7.x86_64.rpm                                                                                                                          | 160 kB  00:00:01
(9/28): python-backports-ssl_match_hostname-3.4.0.2-4.el7.noarch.rpm                                                                                          |  12 kB  00:00:00
(10/28): python-futures-3.0.3-1.el7.noarch.rpm                                                                                                                |  26 kB  00:00:00
(11/28): python-jinja2-2.7.2-2.el7.noarch.rpm                                                                                                                 | 515 kB  00:00:00
(12/28): python-markupsafe-0.11-10.el7.x86_64.rpm                                                                                                             |  25 kB  00:00:00
(13/28): python-babel-0.9.6-8.el7.noarch.rpm                                                                                                                  | 1.4 MB  00:00:00
(14/28): python-msgpack-0.4.6-1.el7.x86_64.rpm                                                                                                                |  73 kB  00:00:00
(15/28): python-requests-2.6.0-1.el7_1.noarch.rpm                                                                                                             |  94 kB  00:00:00
(16/28): openpgm-5.2.122-2.el7.x86_64.rpm                                                                                                                     | 172 kB  00:00:00
(17/28): python-tornado-4.2.1-1.el7.x86_64.rpm                                                                                                                | 636 kB  00:00:00
(18/28): python-six-1.9.0-2.el7.noarch.rpm                                                                                                                    |  29 kB  00:00:00
(19/28): python-urllib3-1.10.2-3.el7.noarch.rpm                                                                                                               | 101 kB  00:00:00
(20/28): python2-crypto-2.6.1-15.el7.x86_64.rpm                                                                                                               | 477 kB  00:00:00
(21/28): python-psutil-2.2.1-1.el7.x86_64.rpm                                                                                                                 | 114 kB  00:00:00
(22/28): python-zmq-15.3.0-2.el7.x86_64.rpm                                                                                                                   | 520 kB  00:00:01
(23/28): salt-minion-2017.7.4-1.el7.noarch.rpm                                                                                                                |  35 kB  00:00:00
(24/28): systemd-python-219-42.el7_4.10.x86_64.rpm                                                                                                            | 118 kB  00:00:00
(25/28): systemd-sysv-219-42.el7_4.10.x86_64.rpm                                                                                                              |  72 kB  00:00:00
(26/28): zeromq-4.1.4-6.el7.x86_64.rpm                                                                                                                        | 555 kB  00:00:00
(27/28): salt-2017.7.4-1.el7.noarch.rpm                                                                                                                       | 7.9 MB  00:00:04
systemd-libs-219-42.el7_4.4_21 FAILED
http://mirrors.cat.pdx.edu/centos/7.4.1708/updates/x86_64/drpms/systemd-libs-219-42.el7_4.4_219-42.el7_4.10.x86_64.drpm: [Errno 14] curl#7 - "Failed to connect to 2610:10:20:208::20: Network is unreachable"
Trying other mirror.
(28/28): systemd-libs-219-42.el7_4.4_219-42.el7_4.10.x86_64.drpm                                                                                              | 143 kB  00:00:00
Finishing delta rebuilds of 1 package(s) (378 k)
Some delta RPMs failed to download or rebuild. Retrying..
systemd-219-42.el7_4.10.x86_64.rpm                                                                                                                            | 5.2 MB  00:00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                1.3 MB/s |  21 MB  00:00:16
Retrieving key from file:///etc/pki/rpm-gpg/saltstack-signing-key
Importing GPG key 0xDE57BFBE:
 Userid     : "SaltStack Packaging Team <packaging@saltstack.com>"
 Fingerprint: 754a 1a7a e731 f165 d5e6 d4bd 0e08 a149 de57 bfbe
 Package    : salt-repo-latest-2.el7.noarch (installed)
 From       : /etc/pki/rpm-gpg/saltstack-signing-key
Running transaction check
Running transaction test
##### Primary configuration settings #####
##########################################
# This configuration file is used to manage the behavior of the Salt Minion.
# With the exception of the location of the Salt Master Server, values that are
# commented out but have an empty line after the comment are defaults that need
# not be set in the config. If there is no blank line after the comment, the
# value is presented as an example and is not the default.

# Per default the minion will automatically include all config files
# from minion.d/*.conf (minion.d is a directory in the same directory
# as the main minion config file).
#default_include: minion.d/*.conf

# Set the location of the salt master server. If the master server cannot be
# resolved, then the minion will fail to start.
#master: salt

# Set http proxy information for the minion when doing requests
#proxy_host:
#proxy_port:
#proxy_username:
#proxy_password:

# If multiple masters are specified in the 'master' setting, the default behavior
# is to always try to connect to them in the order they are listed. If random_master is
# set to True, the order will be randomized instead. This can be helpful in distributing
# the load of many minions executing salt-call requests, for example, from a cron job.
# If only one master is listed, this setting is ignored and a warning will be logged.
# NOTE: If master_type is set to failover, use master_shuffle instead.
#random_master: False

# Use if master_type is set to failover.
#master_shuffle: False

# Minions can connect to multiple masters simultaneously (all masters
# are "hot"), or can be configured to failover if a master becomes
# unavailable.  Multiple hot masters are configured by setting this
# value to "str".  Failover masters can be requested by setting
# to "failover".  MAKE SURE TO SET master_alive_interval if you are
# using failover.
# Setting master_type to 'disable' let's you have a running minion (with engines and
# beacons) without a master connection
# master_type: str

# Poll interval in seconds for checking if the master is still there.  Only
# respected if master_type above is "failover". To disable the interval entirely,
# set the value to -1. (This may be necessary on machines which have high numbers
# of TCP connections, such as load balancers.)
# master_alive_interval: 30

# If the minion is in multi-master mode and the master_type configuration option
# is set to "failover", this setting can be set to "True" to force the minion
# to fail back to the first master in the list if the first master is back online.
#master_failback: False

# If the minion is in multi-master mode, the "master_type" configuration is set to
# "failover", and the "master_failback" option is enabled, the master failback
# interval can be set to ping the top master with this interval, in seconds.
#master_failback_interval: 0

# Set whether the minion should connect to the master via IPv6:
#ipv6: False

# Set the number of seconds to wait before attempting to resolve
# the master hostname if name resolution fails. Defaults to 30 seconds.
# Set to zero if the minion should shutdown and not retry.
# retry_dns: 30

# Set the port used by the master reply and authentication server.
#master_port: 4506

# The user to run salt.
#user: root

# The user to run salt remote execution commands as via sudo. If this option is
# enabled then sudo will be used to change the active user executing the remote
# command. If enabled the user will need to be allowed access via the sudoers
# file for the user that the salt minion is configured to run as. The most
# common option would be to use the root user. If this option is set the user
# option should also be set to a non-root user. If migrating from a root minion
# to a non root minion the minion cache should be cleared and the minion pki
# directory will need to be changed to the ownership of the new user.
/file_client
# attempt to retrieve a named value from pillar fails. When this option is set
Transaction test succeeded
Running transaction
  Updating   : systemd-libs-219-42.el7_4.10.x86_64                                                                                                                              1/31
  Updating   : systemd-219-42.el7_4.10.x86_64                                                                                                                                   2/31
  Installing : python-markupsafe-0.11-10.el7.x86_64                                                                                                                             3/31
testing:
  Installing : python-six-1.9.0-2.el7.noarch                                                                                                                                    4/31
{%- if name is defined -%}
  Installing : systemd-python-219-42.el7_4.10.x86_64                                                                                                                            5/31
# [START] Salt managed searches.
  Installing : libyaml-0.1.4-11.el7_0.x86_64                                                                                                                                    6/31
  Installing : PyYAML-3.11-1.el7.x86_64                                                                                                                                         7/31
  Installing : libtommath-0.42.0-6.el7.x86_64                                                                                                                                   8/31
  Installing : libtomcrypt-1.17-26.el7.x86_64                                                                                                                                   9/31
  Installing : python2-crypto-2.6.1-15.el7.x86_64                                                                                                                              10/31
  Installing : python-backports-1.0-8.el7.x86_64                                                                                                                               11/31
  Installing : python-backports-ssl_match_hostname-3.4.0.2-4.el7.noarch                                                                                                        12/31
  Installing : python-urllib3-1.10.2-3.el7.noarch                                                                                                                              13/31
  Installing : python-requests-2.6.0-1.el7_1.noarch                                                                                                                            14/31
  Installing : python-tornado-4.2.1-1.el7.x86_64                                                                                                                               15/31
  Installing : python-msgpack-0.4.6-1.el7.x86_64                                                                                                                               16/31
  Installing : python-babel-0.9.6-8.el7.noarch                                                                                                                                 17/31
  Installing : python-jinja2-2.7.2-2.el7.noarch                                                                                                                                18/31
  Installing : pciutils-3.5.1-2.el7.x86_64                                                                                                                                     19/31
  Installing : openpgm-5.2.122-2.el7.x86_64                                                                                                                                    20/31
  Installing : python-psutil-2.2.1-1.el7.x86_64                                                                                                                                21/31
  Installing : python-futures-3.0.3-1.el7.noarch                                                                                                                               22/31
  Installing : libsodium-1.0.5-1.el7.x86_64                                                                                                                                    23/31
  Installing : zeromq-4.1.4-6.el7.x86_64                                                                                                                                       24/31
  Installing : python-zmq-15.3.0-2.el7.x86_64                                                                                                                                  25/31
  Installing : salt-2017.7.4-1.el7.noarch                                                                                                                                      26/31
  Installing : salt-minion-2017.7.4-1.el7.noarch                                                                                                                               27/31
  Updating   : systemd-sysv-219-42.el7_4.10.x86_64                                                                                                                             28/31
  Cleanup    : systemd-sysv-219-42.el7_4.4.x86_64                                                                                                                              29/31
  Cleanup    : systemd-219-42.el7_4.4.x86_64                                                                                                                                   30/31
  Cleanup    : systemd-libs-219-42.el7_4.4.x86_64                                                                                                                              31/31
  Verifying  : libsodium-1.0.5-1.el7.x86_64                                                                                                                                     1/31
  Verifying  : python-jinja2-2.7.2-2.el7.noarch                                                                                                                                 2/31
  Verifying  : python-backports-ssl_match_hostname-3.4.0.2-4.el7.noarch                                                                                                         3/31
  Verifying  : python-futures-3.0.3-1.el7.noarch                                                                                                                                4/31
  Verifying  : systemd-python-219-42.el7_4.10.x86_64                                                                                                                            5/31
  Verifying  : systemd-sysv-219-42.el7_4.10.x86_64                                                                                                                              6/31
  Verifying  : salt-2017.7.4-1.el7.noarch                                                                                                                                       7/31
  Verifying  : python-urllib3-1.10.2-3.el7.noarch                                                                                                                               8/31
  Verifying  : python-psutil-2.2.1-1.el7.x86_64                                                                                                                                 9/31
  Verifying  : openpgm-5.2.122-2.el7.x86_64                                                                                                                                    10/31
  Verifying  : PyYAML-3.11-1.el7.x86_64                                                                                                                                        11/31
  Verifying  : zeromq-4.1.4-6.el7.x86_64                                                                                                                                       12/31
  Verifying  : python-six-1.9.0-2.el7.noarch                                                                                                                                   13/31
  Verifying  : salt-minion-2017.7.4-1.el7.noarch                                                                                                                               14/31
  Verifying  : pciutils-3.5.1-2.el7.x86_64                                                                                                                                     15/31
  Verifying  : python-markupsafe-0.11-10.el7.x86_64                                                                                                                            16/31
  Verifying  : systemd-219-42.el7_4.10.x86_64                                                                                                                                  17/31
  Verifying  : python-babel-0.9.6-8.el7.noarch                                                                                                                                 18/31
  Verifying  : python-msgpack-0.4.6-1.el7.x86_64                                                                                                                               19/31
  Verifying  : python-backports-1.0-8.el7.x86_64                                                                                                                               20/31
  Verifying  : libtommath-0.42.0-6.el7.x86_64                                                                                                                                  21/31
  Verifying  : python-tornado-4.2.1-1.el7.x86_64                                                                                                                               22/31
  Verifying  : libyaml-0.1.4-11.el7_0.x86_64                                                                                                                                   23/31
  Verifying  : python2-crypto-2.6.1-15.el7.x86_64                                                                                                                              24/31
  Verifying  : python-requests-2.6.0-1.el7_1.noarch                                                                                                                            25/31
  Verifying  : python-zmq-15.3.0-2.el7.x86_64                                                                                                                                  26/31
  Verifying  : systemd-libs-219-42.el7_4.10.x86_64                                                                                                                             27/31
  Verifying  : libtomcrypt-1.17-26.el7.x86_64                                                                                                                                  28/31
  Verifying  : systemd-libs-219-42.el7_4.4.x86_64                                                                                                                              29/31
  Verifying  : systemd-219-42.el7_4.4.x86_64                                                                                                                                   30/31
  Verifying  : systemd-sysv-219-42.el7_4.4.x86_64                                                                                                                              31/31

Installed:
  salt-minion.noarch 0:2017.7.4-1.el7

Dependency Installed:
  PyYAML.x86_64 0:3.11-1.el7             libsodium.x86_64 0:1.0.5-1.el7                             libtomcrypt.x86_64 0:1.17-26.el7        libtommath.x86_64 0:0.42.0-6.el7
  libyaml.x86_64 0:0.1.4-11.el7_0        openpgm.x86_64 0:5.2.122-2.el7                             pciutils.x86_64 0:3.5.1-2.el7           python-babel.noarch 0:0.9.6-8.el7
  python-backports.x86_64 0:1.0-8.el7    python-backports-ssl_match_hostname.noarch 0:3.4.0.2-4.el7 python-futures.noarch 0:3.0.3-1.el7     python-jinja2.noarch 0:2.7.2-2.el7
  python-markupsafe.x86_64 0:0.11-10.el7 python-msgpack.x86_64 0:0.4.6-1.el7                        python-psutil.x86_64 0:2.2.1-1.el7      python-requests.noarch 0:2.6.0-1.el7_1
  python-six.noarch 0:1.9.0-2.el7        python-tornado.x86_64 0:4.2.1-1.el7                        python-urllib3.noarch 0:1.10.2-3.el7    python-zmq.x86_64 0:15.3.0-2.el7
  python2-crypto.x86_64 0:2.6.1-15.el7   salt.noarch 0:2017.7.4-1.el7                               systemd-python.x86_64 0:219-42.el7_4.10 zeromq.x86_64 0:4.1.4-6.el7

Dependency Updated:
  systemd.x86_64 0:219-42.el7_4.10                        systemd-libs.x86_64 0:219-42.el7_4.10                        systemd-sysv.x86_64 0:219-42.el7_4.10

Complete!
[root@ip-10-0-2-15 ~]# # set file_client: local
[root@ip-10-0-2-15 ~]# vi /etc/salt/minion
[root@ip-10-0-2-15 ~]# service salt-minion restart
Redirecting to /bin/systemctl restart salt-minion.service
[root@ip-10-0-2-15 ~]# mkdir /srv/salt
[root@ip-10-0-2-15 ~]# cd /srv/salt/
[root@ip-10-0-2-15 salt]# vi testing.sls
[root@ip-10-0-2-15 salt]# vi template
[root@ip-10-0-2-15 salt]# vi /tmp/testing.txt
[root@ip-10-0-2-15 salt]# cat testing.sls
testing:
  file.blockreplace:
    - name: /tmp/testing.txt
    - marker_start: "# [START] Salt managed searches."
    - marker_end: "[END Salt managed searches - DO NOT REMOVE]"
    - append_if_not_found: True
    - backup: '.bak'
    - source: salt://template
    - template: jinja
[root@ip-10-0-2-15 salt]# cat template
{%- if name is defined -%}
{{ name.get('first', '') }}
{{ name.get('last', '') }}
More
random
lines
{%- endif -%}
[root@ip-10-0-2-15 salt]# cat /tmp/testing.txt
# [START] Salt managed searches.
[END Salt managed searches - DO NOT REMOVE]
[root@ip-10-0-2-15 salt]# salt-call state.apply testing -l debug
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: ip-10-0-2-15.us-west-2.compute.internal
[DEBUG   ] Configuration file path: /etc/salt/minion
[WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Determining pillar cache
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] LazyLoaded state.apply
[DEBUG   ] LazyLoaded saltutil.is_running
[DEBUG   ] LazyLoaded grains.get
[DEBUG   ] LazyLoaded roots.envs
[DEBUG   ] Could not LazyLoad roots.init: 'roots.init' is not available.
[DEBUG   ] Updating roots fileserver cache
[DEBUG   ] Determining pillar cache
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[INFO    ] Loading fresh modules for state activity
[DEBUG   ] LazyLoaded jinja.render
[DEBUG   ] LazyLoaded yaml.render
[DEBUG   ] In saltenv 'base', looking at rel_path 'testing.sls' to resolve 'salt://testing.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/testing.sls' to resolve 'salt://testing.sls'
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://testing.sls'
[DEBUG   ] No dest file found
[INFO    ] Fetching file from saltenv 'base', ** done ** 'testing.sls'
[DEBUG   ] compile template: /var/cache/salt/minion/files/base/testing.sls
[DEBUG   ] Jinja search path: ['/var/cache/salt/minion/files/base']
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/testing.sls' using 'jinja' renderer: 0.00355982780457
[DEBUG   ] Rendered data from file: /var/cache/salt/minion/files/base/testing.sls:
testing:
  file.blockreplace:
    - name: /tmp/testing.txt
    - marker_start: "# [START] Salt managed searches."
    - marker_end: "[END Salt managed searches - DO NOT REMOVE]"
    - append_if_not_found: True
    - backup: '.bak'
    - source: salt://template
    - template: jinja

[DEBUG   ] LazyLoaded config.get
[DEBUG   ] Results of YAML rendering:
OrderedDict([('testing', OrderedDict([('file.blockreplace', [OrderedDict([('name', '/tmp/testing.txt')]), OrderedDict([('marker_start', '# [START] Salt managed searches.')]), OrderedDict([('marker_end', '[END Salt managed searches - DO NOT REMOVE]')]), OrderedDict([('append_if_not_found', True)]), OrderedDict([('backup', '.bak')]), OrderedDict([('source', 'salt://template')]), OrderedDict([('template', 'jinja')])])]))])
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/testing.sls' using 'yaml' renderer: 0.00537800788879
[DEBUG   ] LazyLoaded file.blockreplace
[INFO    ] Running state [/tmp/testing.txt] at time 21:10:27.064231
[INFO    ] Executing state file.blockreplace for [/tmp/testing.txt]
[DEBUG   ] LazyLoaded cp.get_template
[DEBUG   ] LazyLoaded roots.envs
[DEBUG   ] Could not LazyLoad roots.init: 'roots.init' is not available.
[DEBUG   ] In saltenv 'base', looking at rel_path 'template' to resolve 'salt://template'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/minion/files/base/template' to resolve 'salt://template'
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://template'
[DEBUG   ] No dest file found
[INFO    ] Fetching file from saltenv 'base', ** done ** 'template'
[DEBUG   ] Jinja search path: ['/var/cache/salt/minion/files/base']
[DEBUG   ] cp.get_template returned /var/cache/salt/minion/extrn_files/base/template (Called with: salt://template)
[DEBUG   ] LazyLoaded file.blockreplace
[ERROR   ] An exception occurred in this state: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/state.py", line 1851, in call
    **cdata['kwargs'])
  File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1795, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/salt/states/file.py", line 4258, in blockreplace
    show_changes=show_changes
  File "/usr/lib/python2.7/site-packages/salt/modules/file.py", line 2400, in blockreplace
    while not split_content[-1]:
IndexError: list index out of range

[INFO    ] Completed state [/tmp/testing.txt] at time 21:10:27.095293 duration_in_ms=31.062
[DEBUG   ] File /var/cache/salt/minion/accumulator/39755408 does not exist, no need to cleanup.
[DEBUG   ] LazyLoaded config.option
[DEBUG   ] LazyLoaded highstate.output
local:
----------
          ID: testing
    Function: file.blockreplace
        Name: /tmp/testing.txt
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/site-packages/salt/state.py", line 1851, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1795, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/site-packages/salt/states/file.py", line 4258, in blockreplace
                  show_changes=show_changes
                File "/usr/lib/python2.7/site-packages/salt/modules/file.py", line 2400, in blockreplace
                  while not split_content[-1]:
              IndexError: list index out of range
     Started: 21:10:27.064231
    Duration: 31.062 ms
     Changes:

Summary for local
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time:  31.062 m```
spyd3r commented 6 years ago

I set file_client to local in my setup

spyd3r commented 6 years ago

Any progress here?

angeloudy commented 6 years ago

@spyd3r I was having the same issue. Fixed it with the following code. I think the latest salt already fixed this issue. I will double check.

@@ -2397,7 +2397,7 @@

                         # Trim any trailing new lines to avoid unwanted
                         # additional new lines
-                        while not split_content[-1]:
+                        while split_content and not split_content[-1]:
                             split_content.pop()

                         # push new block content in file
stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.