Suite `title` should be a "string" but "undefined" was given instead

[idoshamun]

I am encountering this error message randomly in my Jenkins output logs: Tests failed: Error thrown outside of test function: Suitetitleshould be a "string" but "undefined" was given instead.

Most the time just rerunning the job fixes the issue but I want to fix it completely. I don't know how to reproduce as it happens randomly.

I am using BrowserStack with Jenkins as my CI platform. Important to say that this error doesn't occur locally.

Versions: polymer 2.0.0-rc.5 wct 5.0.1 polymer-cli 0.18.0

[cadwmaster]

Have the same problem, found out wct is looking in all the directories of the component and if finds another html file before that's the one executed, and if that file doesn't have a suite command then you'll get that error.

[Bubbit]

I'm experiencing the same issue, haven't done a deep dive yet like cadwmaster, but I'm experiencing it while using selenium, so not just browser-stack (also with wct 6.0.0)

[chronicIntrovert]

Also experiencing this issue... but unlike @idoshamun, we are experiencing it locally.

Noticing that we are getting this error when running more than 48 tests, but less will give us a success.

Versions: polymer 1.2.3 wct 5.0.1

[motss]

Not sure if this log is helpful. This thing is so annoying and no way to figure out why it happened! 😢 Everything already has had the <title> tag and really so frustrating!

Error: Error thrown outside of test function: Suite `title` should be a "string" but "undefined" was given instead.
, Timed out
error:   cli runtime exception: Error: Error thrown outside of test function: Suite `title` should be a "string" but "undefined" was given instead.
, Timed out
error:   Error: Error thrown outside of test function: Suite `title` should be a "string" but "undefined" was given instead.
, Timed out
    at /home/travis/.nvm/versions/node/v7.10.0/lib/node_modules/polymer-cli/node_modules/web-component-tester/runner/steps.js:170:27
    at Generator.next (<anonymous>)
    at fulfilled (/home/travis/.nvm/versions/node/v7.10.0/lib/node_modules/polymer-cli/node_modules/web-component-tester/runner/steps.js:4:58)
    at process._tickCallback (internal/process/next_tick.js:109:7)

[Bubbit]

Not sure if other people are experiencing the same, but it always seems to happen on a certain test. I still haven't dived into which test precisely as there are 500+ tests in the repo. So before I do can someone confirm or deny this?

[emilbillberg]

I upgraded from v5.0.1 to v6.0.0 and are now experiencing the same issue. Just like Cadwmaster said it is when you have another html file in the same folder as your test file you get this error. I like to keep my test in the same folder as the component I'm testing. Any progress on this issue?

[kaspnilsson]

We on the PolyGerrit team are dealing with this issue too.

[notwaldorf]

@justinfagnani 👀

[beckysiegel]

It seems at least part of our (PolyGerrit's) problem was having incorrect bower paths in some of our test files. Why these passed before (and sometimes intermittently) I'm not sure. Something happened and they went from intermittently passing to failing 100% of the time. Updating the bower component path seems to get us passing again (we'll see if intermittent or not shortly!)

[justinfagnani]

Does anyone have a github repo that experiences this problem? Even if it is an import pats problem, I'd like to find why the error message is so cryptic.

[beckysiegel]

@justinfagnani not github, but this is the CL that fixed our CI (at least for now) https://gerrit-review.googlesource.com/c/107733/

[beckysiegel]

If you clone the repo and check out the commit prior to it, it's should be reproducible . git clone https://gerrit.googlesource.com/gerrit

[emilbillberg]

After looking in to the issue I found that my paths to web-component-tester/browser.js was wrong and that is what caused the error. I encourage everyone with this issue to take an extra look at all the paths to your elements.

[sp1rs]

Is anyone still facing this issue?

[bennypowers]

yeah I've got it. wrong paths doesn't seem to be the issue. I have a folder structure like this;

/test/sometests.html
/src/eve-behaviors/eve-fire-custom-behavior.html
/src/eve-behaviors/test/eve-fire-custom-behavior.html

/src/eve-behaviors/test/eve-fire-custom-behavior.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, minimum-scale=1, initial-scale=1, user-scalable=yes">

    <title>eve-fire-custom-behavior</title>

    <script src="../../../bower_components/webcomponentsjs/webcomponents-lite.js"></script>
    <script src="../../../bower_components/web-component-tester/browser.js"></script>

    <link rel="import" href="../../../bower_components/polymer/polymer.html">
    <link rel="import" href="../eve-fire-custom-behavior.html">
  </head>
  <body>
    <test-fixture id="basic">
      <template>
        <x-eve-fire-custom-behavior></x-eve-fire-custom-behavior>
      </template>
    </test-fixture>

    <script>
      chai.should();

      Polymer({
        is: 'x-eve-fire-custom-behavior',
        behaviors: [
          Eve.FireCustomBehavior,
        ],
      });

      suite('eve-fire-custom-behavior basic tests', () => {
        let testView;

        setup(() => testView = fixture('basic'));

        test('event should have provided name and default options', (done) => {
          // do test
        });
    </script>

  </body>
</html>

It could be that because i'm importing the same file from different relative paths that things are a-breaking

[ageblade]

After two days of trial and error I managed to solve this problem by removing a reference to iron-resizable-behavior on my element source code (not even the test file). Maybe this will help someone along the way... 😪

[itaiterner]

For me it's fails after calling an HTML attribute with the following convention: this.$['some-id']. After changing it to this.$.someId using camel case convention, it's works.

[visionscaper]

I found out that I got this error because some test files loaded with loadSuites did not exist any more.

[justinfagnani]

We have some changes to path resolution coming up that might fix some of the hard-to-track-down errors like this one. I'm not sure this.$ would interact here, that one's interesting. @visionscaper seems like loadSuites needs some better error handling.

[visionscaper]

@justinfagnani that sounds like a good plan, thanks.

[xscheiner]

We have also had this issue and I just spent the better part of 2 days tracking down the source on a single component that has about 150 tests. We run that in Browserstack against Edge, IE11 and Chrome. The tests consistently passed running IE and Edge in Browserstack and Chrome locally, but every time we ran them in Chrome on Browserstack, we got this error.

The component is fairly large and has two main elements that consume some combination of 5 sub-element that they import. The error happened on one single sub-element when it was imported into the larger of the two main elements.

In addition, we have another component we call js-imports that we use to import third party libraries. One page in that imports ramdajs. This is the entire source of the ramda.html page:

<script src="./js/ramda.min.js"></script>

That was being imported in both the main element and the sub-element that the main element imported. When I removed the ramda.html import from the sub-element, the problem went away.

Now we also have a page in js-imports for momentjs and that was also used in both elements and there was and is no issue. The difference is that the moment one references bower_components rather than from a folder inside itself as the ramda one does. That appears to be the issue.

This is a damn painful bug to track down but I think given it only affects html imports (which are being deprecated) and happens in Polymer 1, I'd be surprised if there were a fix, if one is even possible. I'm only posting here to help people with their own issues if this arises.

Unfortunately, the only hope is to do more or less what I did: keep commenting out huge amounts of code and slowly narrowing till you find the offending line.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.