Fix unit tests broken by #31

[zeroSteiner]

PR #31 added obfuscation for string literals which broke powershell decompression which relied on the contents being a static literal base64 value instead of an obfuscated string. The decompression would then break when the regular expression failed to extract the base64'ed string literal. This adds a new method to reverse the string obfuscation and adds it into the decompression logic to automatically handle cases where the contents are obfuscated. This allows the processing performed by this library to be done both ways (compression/decompression and obfuscate/deobfuscate) which is necessary for a lot of the unit tests to run.

I also added some basic tests for both obfuscation methods.

Once this is landed a new gem should be automatically released.

Testing

• [ ] Run the unit tests, see that everything passes

[adfoster-r7]

Looks like there's intermittent failures if I run the test suite a few times a row:

Failures:

  1) Rex::Powershell::Command::cmd_psh_payload when method is msil should generate a command line
     Failure/Error: @code = Rex::Text.ungzip(unencoded) || Rex::Text.zlib_inflate(unencoded)

     Zlib::DataError:
       invalid distance too far back
     # /Users/user/.rvm/gems/ruby-2.6.6@recog/gems/rex-text-0.2.35/lib/rex/text/compress.rb:99:in `initialize'
     # /Users/user/.rvm/gems/ruby-2.6.6@recog/gems/rex-text-0.2.35/lib/rex/text/compress.rb:99:in `new'
     # /Users/user/.rvm/gems/ruby-2.6.6@recog/gems/rex-text-0.2.35/lib/rex/text/compress.rb:99:in `ungzip'
     # ./lib/rex/powershell/output.rb:155:in `decompress_code'
     # ./spec/rex/powershell/command_spec.rb:10:in `decompress'
     # ./spec/rex/powershell/command_spec.rb:245:in `block (5 levels) in <top (required)>'

Finished in 0.28595 seconds (files took 0.24328 seconds to load)
777 examples, 1 failure

Failed examples:

rspec ./spec/rex/powershell/command_spec.rb:243 # Rex::Powershell::Command::cmd_psh_payload when method is msil should generate a command line

Failures:

  1) Rex::Powershell::Command::cmd_psh_payload when method is old should generate a command line
     Failure/Error: @code = Rex::Text.ungzip(unencoded) || Rex::Text.zlib_inflate(unencoded)

     Zlib::DataError:
       invalid literal/lengths set
     # /Users/user/.rvm/gems/ruby-2.6.6@recog/gems/rex-text-0.2.35/lib/rex/text/compress.rb:99:in `initialize'
     # /Users/user/.rvm/gems/ruby-2.6.6@recog/gems/rex-text-0.2.35/lib/rex/text/compress.rb:99:in `new'
     # /Users/user/.rvm/gems/ruby-2.6.6@recog/gems/rex-text-0.2.35/lib/rex/text/compress.rb:99:in `ungzip'
     # ./lib/rex/powershell/output.rb:155:in `decompress_code'
     # ./spec/rex/powershell/command_spec.rb:10:in `decompress'
     # ./spec/rex/powershell/command_spec.rb:216:in `block (5 levels) in <top (required)>'

Finished in 0.29658 seconds (files took 0.24332 seconds to load)
777 examples, 1 failure

Failed examples:

rspec ./spec/rex/powershell/command_spec.rb:214 # Rex::Powershell::Command::cmd_psh_payload when method is old should generate a command line

Let me know if i can debug anything further / find out what the edge case is if you're not able to replicate on your side :+1:

[zeroSteiner]

Thanks @adfoster-r7 ! This turned out to be due to a bug in the obfuscation logic where if the randomly selected character to be replaced happened to be a digit that was within the range of the length of replaced characters, the format specification in the string could be replaced twice. It should be fixed now, I was able to run the unit tests 30 times without a single failure.

[timwr]

Looks good to me

Before

Finished in 0.24093 seconds (files took 0.20986 seconds to load)
769 examples, 19 failures

Failed examples:

rspec ./spec/rex/powershell/command_spec.rb:62 # Rex::Powershell::Command::compress_script when strip_whitespace is true should strip whitespace
rspec ./spec/rex/powershell/command_spec.rb:70 # Rex::Powershell::Command::compress_script when strip_whitespace is false shouldnt strip whitespace
rspec ./spec/rex/powershell/command_spec.rb:78 # Rex::Powershell::Command::compress_script when sub_vars is true should substitute variables
rspec ./spec/rex/powershell/command_spec.rb:86 # Rex::Powershell::Command::compress_script when sub_vars is false shouldnt substitute variables
rspec ./spec/rex/powershell/command_spec.rb:94 # Rex::Powershell::Command::compress_script when sub_funcs is true should substitute functions
rspec ./spec/rex/powershell/command_spec.rb:102 # Rex::Powershell::Command::compress_script when sub_funcs is false shouldnt substitute variables
rspec ./spec/rex/powershell/command_spec.rb:174 # Rex::Powershell::Command::cmd_psh_payload when persist is true should add a persistance loop
rspec ./spec/rex/powershell/command_spec.rb:181 # Rex::Powershell::Command::cmd_psh_payload when persist is false shouldnt add a persistance loop
rspec ./spec/rex/powershell/command_spec.rb:188 # Rex::Powershell::Command::cmd_psh_payload when prepend_sleep is set should prepend sleep
rspec ./spec/rex/powershell/command_spec.rb:195 # Rex::Powershell::Command::cmd_psh_payload when prepend_sleep isnt set shouldnt prepend sleep
rspec ./spec/rex/powershell/command_spec.rb:202 # Rex::Powershell::Command::cmd_psh_payload when prepend_sleep is 0 shouldnt prepend sleep
rspec ./spec/rex/powershell/command_spec.rb:209 # Rex::Powershell::Command::cmd_psh_payload when method is old should generate a command line
rspec ./spec/rex/powershell/command_spec.rb:224 # Rex::Powershell::Command::cmd_psh_payload when method is net should generate a command line
rspec ./spec/rex/powershell/command_spec.rb:231 # Rex::Powershell::Command::cmd_psh_payload when method is reflection should generate a command line
rspec ./spec/rex/powershell/command_spec.rb:238 # Rex::Powershell::Command::cmd_psh_payload when method is msil should generate a command line
rspec ./spec/rex/powershell/output_spec.rb:39 # Rex::Powershell::Output::deflate_code should zlib the code and wrap in powershell in uncompression stub
rspec ./spec/rex/powershell/output_spec.rb:63 # Rex::Powershell::Output::gzip_code should gzip the code and wrap in powershell in uncompression stub
rspec ./spec/rex/powershell/output_spec.rb:95 # Rex::Powershell::Output::decompress_code should locate the base64 string and decompress it when deflate is used
rspec ./spec/rex/powershell/output_spec.rb:101 # Rex::Powershell::Output::decompress_code should locate the base64 string and decompress it when gzip is used

After

Finished in 0.27445 seconds (files took 0.19672 seconds to load)
777 examples, 0 failures

[timwr]

I'll go ahead and land this to build the gem. I'll do some final testing before landing the gem with https://github.com/rapid7/metasploit-framework/pull/15254