Error running the mime-types suite

[halostatue]

Running qa against the mime-types/ruby-mime-types minitest suite, I get two errors:

‼  <internal error: Unexpected begin test filter: MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved. Expected one of []. Saw [MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved]> [112/143] 0
‼  <internal error: Unexpected begin test filter: MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved. Expected one of []. Saw [MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved]> [112/143] 0

When I run with -capture-standard-fds=false, I got the following result (I have trimmed a lot of vertical whitespace presented):

‼  <internal error: Unexpected begin test filter: MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes. Expected one of [MIME::Type::#add_extensions#test_0002_remains deduplicated with duplicate values MIME::Type::#add_extensions#test_0001_does not modify extensions when provided nil MIME::Type::#preferred_extension, #preferred_extension=#test_0001_is nil when not set and there are no extensions MIME::Type::#preferred_extension, #preferred_extension=#test_0002_is the first extension when not set but there are extensions MIME::Type::#preferred_extension, #preferred_extension=#test_0003_is the extension provided when set MIME::Type::#preferred_extension, #preferred_extension=#test_0004_is adds the preferred extension if it does not exist MIME::Type::.i18n_key#test_0002_does not remove x-prefixes MIME::Type::.i18n_key#test_0004_returns nil when the value provided is not a valid content type MIME::Type::.i18n_key#test_0001_converts text/plain to text.plain MIME::Type::.i18n_key#test_0003_converts text/vCard to text.vcard MIME::Type::#encoding, #encoding=#test_0001_returns #default_encoding if not set explicitly MIME::Type::#encoding, #encoding=#test_0003_resets to the default encoding when set to nil or :default MIME::Type::#encoding, #encoding=#test_0004_raises a MIME::Type::InvalidEncoding for an invalid encoding MIME::Type::#encoding, #encoding=#test_0002_returns the set value when set MIME::Type::.simplified#test_0001_leaves normal types alone MIME::Type::.simplified#test_0002_does not remove x- prefixes by default MIME::Type::.simplified#test_0004_lowercases mixed-case types MIME::Type::.simplified#test_0003_removes x- prefixes when requested MIME::Type::.simplified#test_0005_returns nil when the value provided is not a valid content type MIME::Type::#to_s, #to_str#test_0003_acts like a string for other purposes MIME::Type::#to_s, #to_str#test_0002_acts like a string of the canonical content_type for comparison MIME::Type::#to_s, #to_str#test_0001_represents itself as a string of the canonical content_type MIME::Type::#extensions, #extensions=#test_0002_sets a single extension when provided a single value MIME::Type::#extensions, #extensions=#test_0003_deduplicates extensions MIME::Type::#extensions, #extensions=#test_0001_returns an array of extensions MIME::Type::#xref_urls#test_0001_translates according to given rules MIME::Type::#default_encoding#test_0001_is quoted-printable for text/* types MIME::Type::#default_encoding#test_0002_is base64 for non-text/* types MIME::Type::#to_h#test_0005_has the obsolete key if set MIME::Type::#to_h#test_0007_has the signature key if set MIME::Type::#to_h#test_0003_has the extensions key if set MIME::Type::#to_h#test_0006_has the obsolete and use-instead keys if set MIME::Type::#to_h#test_0004_has the preferred-extension key if set MIME::Type::#to_h#test_0001_has the required keys (content-type, registered, encoding) MIME::Type::#to_h#test_0002_has the docs key if there are documents MIME::Type::#ascii?#test_0001_defaults to true for text/* types MIME::Type::#ascii?#test_0002_defaults to false for non-text/* types]. Saw [MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes]> [31/143] 0
   Unexpected begin test filter: MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes. Expected one of [MIME::Type::#add_extensions#test_0002_remains deduplicated with duplicate values MIME::Type::#add_extensions#test_0001_does not modify extensions when provided nil MIME::Type::#preferred_extension, #preferred_extension=#test_0001_is nil when not set and there are no extensions MIME::Type::#preferred_extension, #preferred_extension=#test_0002_is the first extension when not set but there are extensions MIME::Type::#preferred_extension, #preferred_extension=#test_0003_is the extension provided when set MIME::Type::#preferred_extension, #preferred_extension=#test_0004_is adds the preferred extension if it does not exist MIME::Type::.i18n_key#test_0002_does not remove x-prefixes MIME::Type::.i18n_key#test_0004_returns nil when the value provided is not a valid content type MIME::Type::.i18n_key#test_0001_converts text/plain to text.plain MIME::Type::.i18n_key#test_0003_converts text/vCard to text.vcard MIME::Type::#encoding, #encoding=#test_0001_returns #default_encoding if not set explicitly MIME::Type::#encoding, #encoding=#test_0003_resets to the default encoding when set to nil or :default MIME::Type::#encoding, #encoding=#test_0004_raises a MIME::Type::InvalidEncoding for an invalid encoding MIME::Type::#encoding, #encoding=#test_0002_returns the set value when set MIME::Type::.simplified#test_0001_leaves normal types alone MIME::Type::.simplified#test_0002_does not remove x- prefixes by default MIME::Type::.simplified#test_0004_lowercases mixed-case types MIME::Type::.simplified#test_0003_removes x- prefixes when requested MIME::Type::.simplified#test_0005_returns nil when the value provided is not a valid content type MIME::Type::#to_s, #to_str#test_0003_acts like a string for other purposes MIME::Type::#to_s, #to_str#test_0002_acts like a string of the canonical content_type for comparison MIME::Type::#to_s, #to_str#test_0001_represents itself as a string of the canonical content_type MIME::Type::#extensions, #extensions=#test_0002_sets a single extension when provided a single value MIME::Type::#extensions, #extensions=#test_0003_deduplicates extensions MIME::Type::#extensions, #extensions=#test_0001_returns an array of extensions MIME::Type::#xref_urls#test_0001_translates according to given rules MIME::Type::#default_encoding#test_0001_is quoted-printable for text/* types MIME::Type::#default_encoding#test_0002_is base64 for non-text/* types MIME::Type::#to_h#test_0005_has the obsolete key if set MIME::Type::#to_h#test_0007_has the signature key if set MIME::Type::#to_h#test_0003_has the extensions key if set MIME::Type::#to_h#test_0006_has the obsolete and use-instead keys if set MIME::Type::#to_h#test_0004_has the preferred-extension key if set MIME::Type::#to_h#test_0001_has the required keys (content-type, registered, encoding) MIME::Type::#to_h#test_0002_has the docs key if there are documents MIME::Type::#ascii?#test_0001_defaults to true for text/* types MIME::Type::#ascii?#test_0002_defaults to false for non-text/* types]. Saw [MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes]

Ran 21% in 195ms (12ms of job time): 30 passes, 1 error, with 110 remaining and 2 running:
  MIME::Types::registry::.[] ▸ test
  MIME::Types::#[] ▸ test
-e:884:in `write': Broken pipe (Errno::EPIPE)
    from -e:884:in `block (2 levels) in flush'
    from -e:881:in `each'
    from -e:881:in `block in flush'
    from -e:880:in `synchronize'
    from -e:880:in `flush'
    from -e:1005:in `flush'
    from -e:1527:in `ensure in block in accept_client'
    from -e:1528:in `block in accept_client'
    from -e:1459:in `fork'
    from -e:1459:in `accept_client'

‼  <internal error: Unexpected begin test filter: MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes. Expected one of []. Saw [MIME::Type::#to_json#test_0001_converts to JSON when requested MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes]> [88/143] 0
   Unexpected begin test filter: MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes. Expected one of []. Saw [MIME::Type::#to_json#test_0001_converts to JSON when requested MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes MIME::Type::#raw_media_type#test_0001_extracts the media type as case-preserved MIME::Type::#raw_media_type#test_0002_does not remove x- prefixes]

Ran 61% in 1.868s (5.418s of job time): 86 passes, 2 errors, with 52 remaining and 3 running:
  MIME::Types::registry::.[] ▸ test
  MIME::Types::registry::.type_for ▸ test
  MIME::Types::registry::is enumerable ▸ test
-e:884:in `write': Broken pipe (Errno::EPIPE)
    from -e:884:in `block (2 levels) in flush'
    from -e:881:in `each'
    from -e:881:in `block in flush'
    from -e:880:in `synchronize'
    from -e:880:in `flush'
    from -e:1005:in `flush'
    from -e:1527:in `ensure in block in accept_client'
    from -e:1528:in `block in accept_client'
    from -e:1459:in `fork'

🏁  Ran 108 tests in 4.924s (11.384s of job time): 106 passes, 2 errors.
Test(s) failed.

The speed is somewhat faster (5s vs 8.5s), but the number of tests run does not match the number of tests run by rake test (I use hoe, but provide a usable gemspec-powered Gemfile so you can just use bundle with mime-types to verify yourself).

Finished in 8.552907s, 16.7195 runs/s, 32.0359 assertions/s.

143 runs, 274 assertions, 0 failures, 0 errors, 0 skips

[halostatue]

When run with bundle exec qa run minitest and I get a different set of runs (123 tests, instead of 108 or 143).

[ajbouh]

@halostatue Thanks for the bug report. I've figured out the problem and have a fix in progress. The issue is that qa doesn't handle two different test methods with the same name. It appears that there are two different test methods in your test suite with the same name.

This is because of how Minitest::Spec determines whether or not to make a new anonymous class for each describe (always) or reuse a previous one (never). Merging the identically named "#raw_media_type" describe blocks should temporarily address the issue while I work out a more sustainable fix.

[halostatue]

I’ll fix the mime-types test suite for that; I have a release that I am supposed to put out soon, but won’t get to until I’m back from vacation. Thanks.

[ajbouh]

I just released a new build. I believe it fixes the issue you were having.

Please reopen if not!

[halostatue]

I haven’t been able to get the gem (qa-tool) to work, so I downloaded all of the releases. Version 0.16 is the last version that works for me on mime-types:

% qa-0.16 run 'minitest:test/**/test*.rb'
Listening on unix /tmp/qa
Will run 141 tests using 4 jobs and seed 1298498081. Will hide names of passing tests lacking output.

🏁  Ran 141 tests in 4.236s (10.502s of job time): 141 passes.

The 10 slowest tests took 4.387s or 42% of job time (the 131 others each took ≤ 378ms):
🐌  MIME::Types::lazy loading ▸ test_0001_loads lazily when RUBY_MIME_TYPES_LAZY_LOAD is set 523ms
🐌  MIME::Types::Cache::.load ▸ test_0004_outputs an error when there is a marshal file incompatibility 509ms
🐌  MIME::Types::registry::.[] ▸ test_0005_can be searched with the complete flag 473ms
🐌  MIME::Types::Cache::.load ▸ test_0003_outputs an error when there is an invalid version 458ms
🐌  MIME::Types::registry::.[] ▸ test_0001_can be searched with a MIME::Type 446ms
🐌  MIME::Types::registry::.type_for ▸ test_0002_separates the extension from filenames 409ms
🐌  MIME::Types::registry::.add ▸ test_0001_successfully adds a new type 403ms
🐌  MIME::Types::registry::.[] ▸ test_0004_can be searched with a string 396ms
🐌  MIME::Types::lazy loading::.lazy_load? ▸ test_0003_is false when RUBY_MIME_TYPES_LAZY_LOAD is false 386ms
🐌  MIME::Types::registry::.type_for ▸ test_0005_modifying type extensions causes reindexing 379ms

No matter how I run 0.17, this is what I get:

% qa-0.17 minitest test/**/test*.rb
Will run 0 tests using 4 jobs and seed 19542. Will only show passing/omitted tests with output...
🏁  Ran 0 tests in 1.786s (0s of job time): .

[ajbouh]

What OS? Linux?

[ajbouh]

Very sorry to hear that the latest qa is misbehaving for you, @halostatue. 😞

I have tried (and failed) to reproduce this on both Linux and macOS. I tried both using the qa-tool gem and downloading the 0.17 release from github. I've also tried it with the glob in single quotes and without. All of these combinations work for me.

I can work on a debug build that will emit a lot more logging so we can narrow things down. Can you confirm the folder structure and working directory? Is there a way you can share access to an environment that reproduces the problem? I've tried Ruby 2.3.1, Bundler 1.13.1 on macOS 10.11.6 and Ubuntu 14.04.2 LTS.

[halostatue]

Sorry—had a tech night out tonight. macOS 10.11.6, bundler 1.12.5, pretty much the same as yours…except that it’s absolutely not working for me.

I’ve pushed up the (minor) changes to ruby-mime-types under the qa-tool branch.

I’m not quite sure how to share access to this as its on my personal laptop. Let me see if I can reproduce somewhere I can give you login credentials.

[ajbouh]

Great, thanks! I'm very eager to fix the issue. If you would like me to spin up an ec2 instance and share it with you, I can do that.

On Sep 13, 2016 9:59 PM, "Austin Ziegler" notifications@github.com wrote:

Sorry—had a tech night out tonight. macOS 10.11.6, bundler 1.12.5, pretty much the same as yours…except that it’s absolutely not working for me.

I’ve pushed up the (minor) changes to ruby-mime-types under the qa-tool branch.

I’m not quite sure how to share access to this as its on my personal laptop. Let me see if I can reproduce somewhere I can give you login credentials.

-a

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/ajbouh/qa/issues/3#issuecomment-246906871, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAcna2Yc1fJLNkvDxBKF7OHXf8JFigDks5qp384gaJpZM4JuwXn .

[halostatue]

I just tried it on a fresh Ubuntu 16.04 and I can’t get it failing. I suspect it’s related to my environment, but not sure how. I’ll see if I can figure out why it’s failing.

[halostatue]

So…something is happening, but I’m not sure what. I had a syntax error on the branch (I am also fixing some documentation as part of using inch as well) and it’s clear that something is being read, because everything failed because of the syntax error, but somehow tests are not being found when there are no errors.

I have run a memdump and heapdump and trace and see absolutely nothing useful.

I can set up a teleconsole to see if there is anything that you can observe. Since we’ve emailed and both use gmail, we could arrange that relatively securely.