search

mitre / inspec_tools

A command-line and ruby API of utilities, converters and tools for creating, converting and processing security baseline formats, results and data
https://inspec-tools.mitre.org/
Other
91 stars 30 forks source link

Docker Image does not contain files stored in git LFS: v2.0.6 #197

Closed deric4 closed 4 years ago

deric4 commented 4 years ago

I don't think the xlsx files stored in git lfs are getting pulled correctly by whatever CI is being used to build/push the image to docker hub. It looks like only the refs are being added (note file size of xlsx files):

$ docker run -it -v$(pwd):/share --entrypoint="" mitre/inspec_tools:2.0.6  /bin/sh -c 'ls -la $(dirname $(gem which inspec_tools))/data'
total 2428
drwxr-xr-x    2 root     root          4096 Jul  1 16:21 .
drwxr-xr-x    8 root     root          4096 Jul  1 16:21 ..
-rw-r--r--    1 root     root           131 Jul  1 16:21 NIST_Map_02052020_CIS_Controls_Version_7.1_Implementation_Groups_1.2.xlsx
-rw-r--r--    1 root     root           130 Jul  1 16:21 NIST_Map_09212017B_CSC-CIS_Critical_Security_Controls_VER_6.1_Excel_9.1.2016.xlsx
-rw-r--r--    1 root     root          1039 Jul  1 16:21 README.TXT
-rw-r--r--    1 root     root       2376821 Jul  1 16:21 U_CCI_List.xml
-rw-r--r--    1 root     root          1081 Jul  1 16:21 attributes.yml
-rw-r--r--    1 root     root          5936 Jul  1 16:21 cci2html.xsl
-rw-r--r--    1 root     root           320 Jul  1 16:21 mapping.yml
-rw-r--r--    1 root     root            73 Jul  1 16:21 rubocop.yml
-rw-r--r--    1 root     root         57349 Jul  1 16:21 stig.csv
-rw-r--r--    1 root     root           904 Jul  1 16:21 threshold.yaml

This is causing the xlsx2inspec command to fail in the following way:

$ docker run -it -v$(pwd):/share  mitre/inspec_tools xlsx2inspec  -m <mappings> -p <prefix>l -x <benchmark>.xlsx
Traceback (most recent call last):
        25: from /usr/local/bundle/bin/inspec_tools:23:in `<main>'
        24: from /usr/local/bundle/bin/inspec_tools:23:in `load'
        23: from /usr/local/bundle/gems/inspec_tools-2.0.6/exe/inspec_tools:14:in `<top (required)>'
        22: from /usr/local/bundle/gems/inspec-core-4.21.1/lib/inspec/base_cli.rb:35:in `start'
        21: from /usr/local/bundle/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
        20: from /usr/local/bundle/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
        19: from /usr/local/bundle/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
        18: from /usr/local/bundle/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
        17: from /usr/local/bundle/gems/inspec_tools-2.0.6/lib/inspec_tools/plugin_cli.rb:95:in `xlsx2inspec'
        16: from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
        15: from /usr/local/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
        14: from /usr/local/bundle/gems/inspec_tools-2.0.6/lib/inspec_tools/xlsx_tool.rb:14:in `<top (required)>'
        13: from /usr/local/bundle/gems/inspec_tools-2.0.6/lib/inspec_tools/xlsx_tool.rb:16:in `<module:InspecTools>'
        12: from /usr/local/bundle/gems/inspec_tools-2.0.6/lib/inspec_tools/xlsx_tool.rb:17:in `<class:XLSXTool>'
        11: from /usr/local/bundle/gems/roo-2.8.3/lib/roo/spreadsheet.rb:11:in `open'
        10: from /usr/local/bundle/gems/roo-2.8.3/lib/roo/spreadsheet.rb:11:in `new'
         9: from /usr/local/bundle/gems/roo-2.8.3/lib/roo/excelx.rb:61:in `initialize'
         8: from /usr/local/bundle/gems/roo-2.8.3/lib/roo/excelx.rb:398:in `process_zipfile'
         7: from /usr/local/bundle/gems/rubyzip-1.3.0/lib/zip/file.rb:111:in `open'
         6: from /usr/local/bundle/gems/rubyzip-1.3.0/lib/zip/file.rb:111:in `new'
         5: from /usr/local/bundle/gems/rubyzip-1.3.0/lib/zip/file.rb:81:in `initialize'
         4: from /usr/local/bundle/gems/rubyzip-1.3.0/lib/zip/file.rb:81:in `open'
         3: from /usr/local/bundle/gems/rubyzip-1.3.0/lib/zip/file.rb:82:in `block in initialize'
         2: from /usr/local/bundle/gems/rubyzip-1.3.0/lib/zip/central_directory.rb:136:in `read_from_stream'
         1: from /usr/local/bundle/gems/rubyzip-1.3.0/lib/zip/central_directory.rb:103:in `read_e_o_c_d'
/usr/local/bundle/gems/rubyzip-1.3.0/lib/zip/central_directory.rb:143:in `get_e_o_c_d': Zip end of central directory signature not found (Zip::Error)

Downloading the NIST_Map_02052020_CIS_Controls_Version_7.1_Implementation_Groups_1.2.xlsx file and copying into the container works, but def a big speed bump in figuring out how to use a new project correctly 🙃

I'm guessing this is the same problem causing #180

Thanks!

Bialogs commented 4 years ago

Instead of implementing a hack in the build process to get this file I say we just push up the delivery of #134

Related https://github.com/docker/hub-feedback/issues/500

Bialogs commented 4 years ago

Can you please check out 2.0.7 on https://hub.docker.com/r/mitre/inspec_tools/tags and let me know if you have any issues?

deric4 commented 4 years ago

@Bialogs its working! Thanks! I'm getting a correction to stdout for each control now though, which I don't think was happening before 🤔

Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols

Bialogs commented 4 years ago

Great I will close this issue.

The correction you are describing is a new feature that runs the output Ruby through Rubocop so that you start with a linted profile (mostly to replace double quote strings with single quote strings) which is the preferred style.