Closed cruscio closed 6 years ago
cruscio
commented
6 years ago - enforce_toplevel: False and it extracted the files from the zip into the folder specified by -name - as if the top level folder they're nested in didn't exist. cruscio
commented
6 years ago ... ... ... ... ... is the file cached somewhere?
An earlier copy of the ZIP didn't have a top level folder - and also didn't have a couple files that are suspiciously absent from the minion after extracting with enforce_toplevel: False ... the missing files are in a copy of the same zip transferred (from the same -source) to the minion by a file.manage state
winlogbeat archive:
archive.extracted:
- name: c:/myproject/winlogbeat/
- source: salt://mystate/files/winlogbeat.zip
- force: True
- overwrite: True
- enforce_toplevel: False
blagraphonious:
file.managed:
- name: c:/mystate/winlogbeat.zip
- source: salt://mystate/files/winlogbeat.zipSame source, but archive.extracted is using an older cached copy of the archive, not the more recent one that file.managed is pulling down
cruscio
commented
6 years ago enforce_toplevel: Falsestate.apply Whatever the issue was, it's okay now, but I'm going to leave this open for review because it seems like something isn't quite right with how the state was being applied. Can't think of any way I could have ended up with an older timestamp on the newer file.
Ch3LL
commented
6 years ago I know @terminalmage has done some work here so maybe he has some insight for you here
terminalmage
commented
6 years ago Please post the output from salt <minion-id> archive.list salt://mystate/files/winlogbeat.zip, replacing <minion-id> with the appropriate minion ID.
cruscio
commented
6 years ago Unfortunately at this point, whatever condition caused the issue has been cleared up in my environment. I've updated the zip package a couple times now, state.apply runs without error, and archive.list shows the newer file manifest with top level directory and all files.
terminalmage
commented
6 years ago OK. Unfortunately, without that information I can't really troubleshoot this. The way the code works is that it uses archive.list to get the contents of the archive (see here). It then uses the result of that list to try to detect whether or not there is a single top-level directory in the archive (see here). The only thing that causes that error you saw is if there is more than one top-level directory, or if there are any files in the top level of the archive.
Given this information, one of two things happened; there was an extra directory or some files in the top level, or Salt did not properly detect the contents of the file. Without the output from archive.list (with verbose=True, I forgot to mention that earlier), I can't possibly know which is the case.
cruscio
commented
6 years ago After some confusing troubleshooting, it seemed archive.list and archive.extracted behaved as expected, just not on the expected file.
The minion was using two different versions of the same archive file when applying the state.
archive.extracted was operating on an older version than the one on the master file.managed was using the version currently on the masterI'm not sure there's anything more to do here, particularly since I can't reproduce it, I just wanted to leave the issue open until someone more knowledgeable could make that determination, in case it was worth pursuing how the minion was refreshing its cache.
terminalmage
commented
6 years ago @cruscio in that case, I'm going to close this since the way archive.extracted handles caching the source archives was overhauled in https://github.com/saltstack/salt/pull/43681, and the result of that work was released in 2017.7.3. If the problem was file caching, the offending code almost certainly no longer exists. If this issue crops up again, please let me know and I can revisit it.
kiemlicz
commented
9 months ago Still happens
state sls:
extract_idea:
archive.extracted:
- name: /opt
- source: https://download.jetbrains.com/idea/ideaIU-2023.3.2.tar.gz
- user: johnsnow
- group: johnsnow
- enforce_ownership_on: /opt/idea-IU-233.13135.103
- skip_verify: Trueroot@ambassador:/# salt thewall archive.list https://download.jetbrains.com/idea/ideaIU-2023.3.2.tar.gz
thewall:
- idea-IU-233.13135.103/
- idea-IU-233.13135.103/
- idea-IU-233.13135.103/
- idea-IU-233.13135.103/
- idea-IU-233.13135.103/Install-Linux-tar.txt
- idea-IU-233.13135.103/bin/
- idea-IU-233.13135.103/bin/
- idea-IU-233.13135.103/bin/appletviewer.policy
- idea-IU-233.13135.103/bin/brokenPlugins.db
- idea-IU-233.13135.103/bin/format.sh
- idea-IU-233.13135.103/bin/fsnotifier
- idea-IU-233.13135.103/bin/idea.png
- idea-IU-233.13135.103/bin/idea.properties
- idea-IU-233.13135.103/bin/idea.sh
- idea-IU-233.13135.103/bin/idea.svg
- idea-IU-233.13135.103/bin/idea64.vmoptions
- idea-IU-233.13135.103/bin/inspect.sh
- idea-IU-233.13135.103/bin/jetbrains_client.sh
- idea-IU-233.13135.103/bin/jetbrains_client64.vmoptions
....
- idea-IU-233.13135.103/plugins/xpath/lib/
- idea-IU-233.13135.103/plugins/xpath/lib/rt/
- idea-IU-233.13135.103/plugins/xpath/lib/rt/xslt-rt.jar
- idea-IU-233.13135.103/plugins/xpath/lib/xpath.jar
- idea-IU-233.13135.103/plugins/yaml/
- idea-IU-233.13135.103/plugins/yaml/lib/
- idea-IU-233.13135.103/plugins/yaml/lib/yaml.jar
- idea-IU-233.13135.103/product-info.json
Description of Issue/Question
Error extracting an archive. It contains a single top level directory, but state.apply raises an error indicating it doesn't
state.apply return:
Note: also tried both
-name: c:/myproject/winlogbeatand-name: c:/myproject/winlogbeat/with similar resultsSetup
init.sls file
zip file
contents available here: https://www.elastic.co/downloads/beats/winlogbeat I just repackaged them in a folder without the version number, using command line zip on Ubuntu 16.04
Steps to Reproduce Issue
salt '*' state.apply mystateVersions Report
Master
Minion