search

librariesio / bibliothecary

:notebook_with_decorative_cover: Libraries.io Package Manager Manifest Parsers
https://libraries.io/rubygems/bibliothecary
GNU Affero General Public License v3.0
89 stars 36 forks source link

invalid byte sequence in UTF-8 (ArgumentError) #590

Closed sneko closed 6 months ago

sneko commented 6 months ago

Hi @tiegz ,

I'm trying to analyse a third-party repository (among others) but this specific one triggers an error with bibliothecary.

To reproduce:

git clone https://github.com/betagouv/signalconso-website
bibliothecary list --trace

I get this error output:

/.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary.rb:83:in `sub': invalid byte sequence in UTF-8 (ArgumentError)

      .sub(/^\xEF\xBB\xBF/, "") # remove any Byte Order Marks so JSON, etc don't fail while parsing them.
           ^^^^^^^^^^^^^^^^^^^
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary.rb:83:in `utf8_string'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/file_info.rb:20:in `contents'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/analyser/matchers.rb:35:in `mapping_entry_match?'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/analyser/matchers.rb:59:in `block in first_matching_mapping_details'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/analyser/matchers.rb:59:in `each'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/analyser/matchers.rb:59:in `find'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/analyser/matchers.rb:59:in `first_matching_mapping_details'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/analyser/matchers.rb:52:in `match_info?'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/runner.rb:41:in `block in applicable_package_managers'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/runner.rb:41:in `select'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/runner.rb:41:in `applicable_package_managers'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/runner.rb:171:in `add_matching_package_managers_for_file_to_list'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/runner.rb:91:in `block in load_file_info_list'
    from /.../ruby/3.2.2/lib/ruby/3.2.0/find.rb:49:in `block (2 levels) in find'
    from /.../ruby/3.2.2/lib/ruby/3.2.0/find.rb:48:in `catch'
    from /.../ruby/3.2.2/lib/ruby/3.2.0/find.rb:48:in `block in find'
    from /.../ruby/3.2.2/lib/ruby/3.2.0/find.rb:43:in `each'
    from /.../ruby/3.2.2/lib/ruby/3.2.0/find.rb:43:in `find'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/runner.rb:84:in `load_file_info_list'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/runner.rb:14:in `analyse'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary.rb:23:in `analyse'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/cli.rb:20:in `block (2 levels) in run'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/commander-5.0.0/lib/commander/command.rb:187:in `call'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/commander-5.0.0/lib/commander/command.rb:157:in `run'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/commander-5.0.0/lib/commander/runner.rb:444:in `run_active_command'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/commander-5.0.0/lib/commander/runner.rb:58:in `run!'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/commander-5.0.0/lib/commander/delegates.rb:18:in `run!'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/lib/bibliothecary/cli.rb:35:in `run'
    from /.../ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bibliothecary-8.7.6/bin/bibliothecary:8:in `<top (required)>'
    from /.../ruby/3.2.2/bin/bibliothecary:25:in `load'
    from /.../ruby/3.2.2/bin/bibliothecary:25:in `<main>'

Any idea if this an issue with the repository itself, or if bibliothecary is not managing a specific case?

Thank you,

tiegz commented 6 months ago

thanks @sneko , this is now fixed in 8.7.7