yujinakayama / transpec

The RSpec syntax converter
http://yujinakayama.me/transpec/
MIT License
1.01k stars 52 forks source link

transpec tries to process .AppleDouble folders during dynamic analysis and fails with "invalid byte sequence in US-ASCII" or "invalid byte sequence in UTF-8" #136

Closed nicobrenner closed 2 years ago

nicobrenner commented 2 years ago

Using transpec 1.0.0 with ruby 2.0 on a project that has rails 3.0.6 and rspec 2.99.

Ran transpec with no arguments from the root folder of the rails project.

It seems like transpec is trying to process MacOS' .AppleDouble hidden files/folders. After adding some additional error messages in the libraries used, I'm getting this:

/home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/base_rewriter.rb:13:in `rescue in rewrite_file!': "invalid byte sequence in UTF-8 \u0000\u0005\u0016\a\u0000\u0002\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\r\u0000\u0000\u0000\u0002\u0000\u0000\u0002\xE5\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0003\u0000\u0000\u0000\xB6\u0000\u0000\u0000\u000F\u0000\u0000\u0000\u0004\u0000\u0000\u0001\xB5\u0000\u0000\u0000\u0000\u0000\u0000\u0000\b\u0000\u0000\u0002}\u0000\u0000\u0000\u0010\u0000\u0000\u0000\t\u0000\u0000\u0002\x8D\u0000\u0000\u0000 \u0000\u0000\u0000\v\u0000\u0000\u0002\xC1\u0000\u0000\u0000\b\u0000\u0000\u0000\r\u0000\u0000\u0002\xB5\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u000E\u0000\u0000\u0002\xB1\u0000\u0000\u0000\u0004\u0000\u0000\u0000\u000F\u0000\u0000\u0002\xAD\u0000\u0000\u0000\u0004\x80DEV\u0000\u0000\u0002\xC9\u0000\u0000\u0000\b\x80INO\u0000\u0000\u0002\xD1\u0000\u0000\u0000\b\x80SYN\u0000\u0000\u0002\xD9\u0000\u0000\u0000\b\x80SV~\u0000\u0000\u0002\xE1\u0000\u0000\u0000\u0004rails_helper.rb\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000)X\xC5r)X\xC5r\x80\u0000\u0000\u0000)X\xC5r\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0002\xB5\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0004\b\u0000\u0000\u0000\u0000\u0000\u0000\x8C#\u0018\u0000\u0000\u0000\u0000\u0000f\u0002\x93`\u0000\u0000\u0000\u0000\u0000\u0000\x84\xEC spec/.AppleDouble/rails_helper.rb" (RuntimeError)
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/base_rewriter.rb:10:in `rewrite_file!'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:98:in `block (2 levels) in analyze'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:96:in `each'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:96:in `block in analyze'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:116:in `in_copied_project'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:93:in `analyze'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/cli.rb:50:in `block in process'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:79:in `block in initialize'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:124:in `block (2 levels) in in_copied_project'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:123:in `chdir'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:123:in `block in in_copied_project'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/2.0.0/tmpdir.rb:88:in `mktmpdir'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:120:in `in_copied_project'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/dynamic_analyzer.rb:78:in `initialize'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/cli.rb:48:in `new'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/cli.rb:48:in `process'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/cli.rb:34:in `run'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/cli.rb:17:in `run'
        from /home/username/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/bin/transpec:7:in `<top (required)>'
        from /home/username/.rbenv/versions/2.0.0-p647/bin/transpec:23:in `load'
        from /home/username/.rbenv/versions/2.0.0-p647/bin/transpec:23:in `<main>'

The important part is the end of the first line of the message, which shows the file path: spec/.AppleDouble/rails_helper.rb. So it looks like transpec is trying to process MacOS' .AppleDouble hidden files/folders.

The file path part comes from these bits of modified code that include the text and the source file in the error:

# ~/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/base_rewriter.rb:6
module Transpec
  class BaseRewriter
    def rewrite_file!(file_path)
      source = File.read(file_path)
      begin
        rewritten_source = rewrite(source, file_path)
      rescue => e
              raise "#{e.message} #{file_path}".inspect
      end
      return if source == rewritten_source
      File.write(file_path, rewritten_source)
    end
# ~/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/parser-2.7.2.0/lib/parser/source/buffer.rb:163
    def source=(input)
        input = input.dup if input.frozen?
        input = self.class.reencode_string(input)

        unless input.valid_encoding?
          raise EncodingError, "invalid byte sequence in #{input.encoding.name} #{input}"
        end

        self.raw_source = input
      end
nicobrenner commented 2 years ago

Was able to get around it by modifying lib/transpec/file_finder.rb (~/.rbenv/versions/2.0.0-p647/lib/ruby/gems/2.0.0/gems/transpec-1.0.0/lib/transpec/file_finder.rb) to skip over paths containing AppleDouble:

def ruby_files_in_directory(directory_path)
  ruby_file_paths = []

  Find.find(directory_path) do |path|
    next unless path !~ /AppleDouble/ # Files within AppleDouble dirs are excluded
    next unless File.file?(path)
    next unless File.extname(path) == '.rb'
    ruby_file_paths << path
  end

  ruby_file_paths
end

Leaving it here in case it helps someone else.