has_text? and invisible content

jonleighton commented 11 years ago

Capybara master has a new has_text? matcher which excludes invisible content such as <title> elements and display: none elements.

Its implementation uses the Node#text method, so the implication is that the behaviour of Node#text has now changed from returning all text to only returning visible text. This presents two problems:

Users will have trouble upgrading if they are using Node#text directly, or has_content? (which is now just an alias of has_text?)
There is no API for getting e.g. the text of the <title> element. This could be a valid user requirement and was previously supported by the #text method.

I propose adding a #visible_text method and using that to implement has_text?, leaving the implementations of #text and #has_content? as they are in 1.1. (#has_content? can be deprecated but I believe #text remains useful for checking e.g. <title> text.) WDYT?

Related: #522

cc @mkdynamic

ugisozols commented 11 years ago

+1 for not being able to query <title> after upgrading to 2.0.

find("title").should have_content("some text") now fails.

demetrios commented 11 years ago

+1 for troubles with <title> in 2.0

it { should have_selector('title', text: 'Some Title') }

will fail on <head><title>Some Title</title></head>

jnicklas commented 11 years ago

Wouldn't you guys rather have an explicit API for testing the page title, rather than changing the behaviour of #text just for this? That seems much more sensible to me.

demetrios commented 11 years ago

OK with me...just point me to it.

jnicklas commented 11 years ago

There isn't one, because no one has bothered to build one.

demetrios commented 11 years ago

Wish I could help...but I've never worked on a gem and would only cause more problems...

Mic92 commented 11 years ago

Maybe there should be a way to get content of every tag, even if it is not visible.

Mic92 commented 11 years ago

I think the problem is that the railstutorial makes heavily use of this feature (check the title) and people tend to copy these examples.

demetrios commented 11 years ago

Yup, that's my problem. I learned TDD & RSpec from railstutorial.org and as such most of my projects carry with them some basic structure gleaned from there. After upgrading projects to 2.0 my title specs are failing everywhere.

FWIW, it's a little confusing to a relative newb like me when this is happening:

  it { should have_selector('title', text: 'Some Title') } # <= fails
  it { should have_selector('title') }                     # <= passes
  it { should have_text('Some Title') }                    # <= passes

…and according to http://rubydoc.info/github/jnicklas/capybara/master/Capybara/Node/Matchers#has_selector%3F-instance_method we should be able to find elements containing text:. Now maybe those docs are for 1.x…but that's not clear either...to me anyway.

If there's an alternative way to easily test the title in a feature spec let me know and I'll go away…leaving you all to continue your great work!

anilali commented 11 years ago

+1 for not being able to query after upgrading to 2.0.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/jonleighton"><img src="https://avatars.githubusercontent.com/u/1979?v=4" />jonleighton</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>Adding a <code>#has_title?</code> matcher sounds like a decent solution to me. I'll tweet about in and see if I can get someone to step up to do the work.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/gucki"><img src="https://avatars.githubusercontent.com/u/175494?v=4" />gucki</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>IMO the behavior of 2.0 is completely broken. In 1.1.4 <code>page.find("title").text</code> returns the text, in 2.0 it returns an empty string. Returning an empty string is just wrong when the element does contain text.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mkdynamic"><img src="https://avatars.githubusercontent.com/u/4312?v=4" />mkdynamic</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>My original reason for wanting the <code>has_text</code> matcher (#522) was to be able to test specifically for 'text visible to the user'.</p> <p>The title tag <strong>is</strong> visible text, it's shown in the browser chrome.</p> <p>Given that, I think it would be ideal if the title text was included.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/jnicklas"><img src="https://avatars.githubusercontent.com/u/134?v=4" />jnicklas</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>@gucki: the title tag <em>is</em> an element and it is decidedly <em>not</em> visible by default. Try adding this CSS selector to a page:</p> <pre><code class="language-css">head, title { display: block }</code></pre> <p>Boom, there is the title <em>element</em> and it is quite distinct from the title shown in the browser chrome. This element is not visible, and if we follow the logic that the text of invisible elements is not shown, then it makes sense that the title <em>element</em> has no text. I agree that that's a matter of debate and discussion, but I wouldn't go so far as to say that the current behaviour is broken. It's just a different interpretation.</p> <p>I think the solution to this problem is two-fold:</p> <ol> <li>Add some kind of other method which returns the text regardless of if it's visible or not. This could be an option to <code>#text</code> or a separate method.</li> <li>Add an official API for retrieving the title of the page and querying it. Some work on this has already been done on this and I will take a closer look as soon as I have the time.</li> </ol> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mkdynamic"><img src="https://avatars.githubusercontent.com/u/4312?v=4" />mkdynamic</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>@jnicklas Well, it is visible :) <img src="http://cl.ly/image/0g2E462s3Z3U/Screen%20Shot%202012-12-15%20at%202.34.51%20PM.png" alt="" /></p> <p>I do think it makes more semantic sense for <code>has_text</code> to include text from the title element. Since the whole idea behind <code>has_text</code> is to move away from the logical model of XML's notion of text and towards the notion of human readable text.</p> <p>I think your suggested solution for #1 and #2 are a good idea, regardless.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/gucki"><img src="https://avatars.githubusercontent.com/u/175494?v=4" />gucki</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>@jnicklas As you say, the elemenI itself is not visible - not only its content. So think it'd be more intuitive if <code>page.find(element)</code> would return <code>nil</code> if the element is not visible. But when <code>page.find(element)</code> returns an element, it should properly return its attributes and content as they are - no matter if it's visible or not.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/joliss"><img src="https://avatars.githubusercontent.com/u/524783?v=4" />joliss</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>It seems a bit unintuitive to me too that page.find("title").text shouldn't work, but if it's architecturally messy to make it work, then we may have to do without it. Jonas is the judge. :)</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/wearethefoos"><img src="https://avatars.githubusercontent.com/u/309331?v=4" />wearethefoos</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>+1 for this problem, and also:</p> <pre><code class="language-html"><html> <head> <title>Title Text</title> </head> <body> <h1>Some heading</h1> <p>Some paragraph</p> </body> </html></code></pre> <pre><code class="language-ruby">page.has_xpath? "//title" # => true page.has_xpath? "//title", text: "Title Text" # => false page.should have_xpath "//title" #=> true page.should have_xpath "//title", text: "Title Text" #=> Capybara::ExpectationNotMet Exception: expected to find xpath "//title" with text "Title Text." but there were no matches. Also found "", which matched the selector but not all filters.</code></pre> <p>Which indicates the title tag is found, but the text inside it remains invisible.</p> <p>Although a <code>page.title</code> would be a nice shorthand to have, it will only provide support for that one tag. How about meta tags, canonical links, etc.? In my humble opinion, <code>#find</code> etc. can return nothing when content is invisible, but xpaths should return visible and invisible content, so it can be used as a more "advanced" fallback for niche situations..</p> <p>I'd be happy to help code it, once there is agreed on what to do with this btw!</p> <p>Thanks.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/agileapplications"><img src="https://avatars.githubusercontent.com/u/235900?v=4" />agileapplications</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>Would be great to have a method that retrieves the text of an element independent of it's visibility. That method could also be used to retrieve the page title. So in my opinion no need for an extra page-title-method but wouldn't hurt of course.</p> <p>If you keep the behaviour as earlier I would having #text returning always the text and e.g. #visible_text returning only text that is visible.</p> <p>Best regards Tobias</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/murdoch"><img src="https://avatars.githubusercontent.com/u/56636?v=4" />murdoch</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>+1 can't query title text, but the following works ok:</p> <pre><code><head><title>marflar</title></head> should have_xpath("//title[contains(.,'marflar')]")</code></pre> <p>Is there anything wrong with just doing that?</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/meredrica"><img src="https://avatars.githubusercontent.com/u/450950?v=4" />meredrica</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>The content of the title tag is definatively visible, so it should also be possible to check it.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/molfar"><img src="https://avatars.githubusercontent.com/u/235048?v=4" />molfar</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>completely agree, the page title is the first thing checked in requests specs.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/john-patrik"><img src="https://avatars.githubusercontent.com/u/489888?v=4" />john-patrik</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>I also ran into this roadblock today and stumbled upon this solution; <a href="http://stackoverflow.com/a/13755730">http://stackoverflow.com/a/13755730</a></p> <p>Seems to work out pretty well for me, so lots of kudos to the creator (not me)!</p> <p>I'm copy/pasting the solution because I want to save you guys a click.</p> <hr /> <p>I had the same issues when I upgraded to Capybara 2.0, and managed to solve them by creating the following custom RSpec matcher using Capybara.string:</p> <p><strong>spec/support/utilities.rb</strong></p> <pre><code class="language-ruby">RSpec::Matchers::define :have_title do |text| match do |page| Capybara.string(page.body).has_selector?('title', text: text) end end</code></pre> <p>Now, in a spec file where subject { page }, I can use:</p> <pre><code class="language-ruby">it { should have_title("My Title") } it { should_not have_title("My Title") }</code></pre> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/jsuwo"><img src="https://avatars.githubusercontent.com/u/898045?v=4" />jsuwo</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>+1 for not being able to query <code><title></code> after upgrading to 2.0.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/leahcim"><img src="https://avatars.githubusercontent.com/u/595918?v=4" />leahcim</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>Thanks @john-patrik. That's a great find and just what I needed. Something like this could be in Capybara by default as per @jnicklas's 2nd proposed solution above: <a href="https://github.com/jnicklas/capybara/issues/844#issuecomment-11411003">https://github.com/jnicklas/capybara/issues/844#issuecomment-11411003</a></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/beaubrewer"><img src="https://avatars.githubusercontent.com/u/249089?v=4" />beaubrewer</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>Has there been an official decision or progress made here? Both solutions would work and I feel a solution (even if it proves to be less than perfect) is better than none at all. -I hate not being able to get through a tutorial :)</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/jnicklas"><img src="https://avatars.githubusercontent.com/u/134?v=4" />jnicklas</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>Closed by #960</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mhartl"><img src="https://avatars.githubusercontent.com/u/6232?v=4" />mhartl</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>I'd like to express my frustration about the breaking of backwards compatibility on the behavior of <code>have_selector('title', text: ...)</code>. As others on this thread have noted, many people use <code>have_selector</code> because of its use in the Ruby on Rails Tutorial. As the author that tutorial, it is my job to keep it up to date, and the lack of support for the previous behavior of <code>have_selector</code> is both frustrating for me and (because the necessary fix adds extra code) increases the complexity for beginning users. It is my hope that future iterations of Capybara will take more care to preserve current (and, in this case, logical) behavior.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/jnicklas"><img src="https://avatars.githubusercontent.com/u/134?v=4" />jnicklas</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>@mhartl we've gone out of our way to make this more intuitive for Capybara 2.1. We've added more control over whether to return only visible text or all text and we've added a new api for querying and asserting on the page title.</p> <p>I'm sorry that behaviour broke this significantly for you, we didn't anticipate that this change would affect users. We had a lengthy beta period for Capybara 2.0, and this issue was not reported during that time. If it had, we could have done something about it sooner.</p> <p>However, I don't agree that the way you are suggesting to query for the page title is a good idea. Once Capybara 2.1 is released, you should update the tutorial to suggest the use of <code>have_title</code>, which will be the official API from now on. <code>have_selector("title", text: "something")</code> might work or it might not, it depends on the driver. Imo, it's a perfectly valid interpretation for the title <em>element</em> not to have any text.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/jnicklas"><img src="https://avatars.githubusercontent.com/u/134?v=4" />jnicklas</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>@mhartl it just occured to me that since Capybara 2.1 will default <code>Capybara.ignore_hidden_elements</code> to <code>true</code>, and the title element is invisible, it won't be possible to query its content this way, visible text or not.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mhartl"><img src="https://avatars.githubusercontent.com/u/6232?v=4" />mhartl</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>@jnicklas I actually never liked using <code>have_selector</code> in this context; in the original Rails 2.3 version of the book I used Webrat's <code>have_tag</code>, but that disappeared when Webrat was deprecated. Using <code>have_selector</code> was the closest, if imperfect, substitute. I agree that <code>have_title</code> is much better, and I'll plan to update the tutorial accordingly. Thanks!</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/topfunky"><img src="https://avatars.githubusercontent.com/u/26?v=4" />topfunky</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>For reference, Capybara now has <code>have_title</code> in 2.1.0.beta1 as described here:</p> <p><a href="http://www.elabs.se/blog/60-introducing-capybara-2-1#asserting_against_the_page_title">http://www.elabs.se/blog/60-introducing-capybara-2-1#asserting_against_the_page_title</a></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/mhartl"><img src="https://avatars.githubusercontent.com/u/6232?v=4" />mhartl</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>Awesome. I can confirm that this works with the Rails Tutorial sample app and have updated the beta book accordingly.</p> <p>I noticed that the selenium-webdriver gem dependency didn't install automatically. Is that intentional? To get it to work, I had to add it explicitly to my Gemfile:</p> <pre><code class="language-ruby">group :test do gem 'selenium-webdriver', '2.0' gem 'capybara', '2.1.0.beta1' end</code></pre> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/topfunky"><img src="https://avatars.githubusercontent.com/u/26?v=4" />topfunky</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>@mhartl See issue #1018 for the Selenium fix.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/husnainAshraf"><img src="https://avatars.githubusercontent.com/u/3715370?v=4" />husnainAshraf</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>This work for me</p> <p>get :home assert_select "title", "Ruby on Rails Tutorial Sample App | Home"</p> <p>can see <a href="http://stackoverflow.com/questions/3971449/verifying-page-title-with-rspec">http://stackoverflow.com/questions/3971449/verifying-page-title-with-rspec</a></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/carlad"><img src="https://avatars.githubusercontent.com/u/1754975?v=4" />carlad</a> commented <strong> 11 years ago</strong> </div> <div class="markdown-body"> <p>the solution @john-patrik posted above worked for me <a href="http://stackoverflow.com/questions/13573525/rspec-capybara-2-0-tripping-up-my-have-selector-tests/13755730#13755730">http://stackoverflow.com/questions/13573525/rspec-capybara-2-0-tripping-up-my-have-selector-tests/13755730#13755730</a></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/andresprogrammer"><img src="https://avatars.githubusercontent.com/u/12407407?v=4" />andresprogrammer</a> commented <strong> 9 years ago</strong> </div> <div class="markdown-body"> <p>I have one afternoon and half of the morning only with the title problem. Finally I found a solution and I just sign up to write it:</p> <pre><code>expect(page.title).to eq("My title")</code></pre> <p>Good luck</p> </div> </div> <div class="page-bar-simple"> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>

teamcapybara / capybara

has_text? and invisible content #844