Port all the text researches to tree researches

[atimmer]

The following list of researches exist now, I will put after the research what needs to be done for them.

In general I want to:

• Drop all the verbs from the research. A research is not something you do, it is an objective fact about the text.
• Change count research to be plain research. The count can easily be retrieved by doing .length whenever the count is necessary. And the actual results are far more useful than the count.

List of current research

• urlLength, unchanged
• wordCountInText, change to words research. Count can easily be retrieved by doing .length in the assessment.
• findKeywordInPageTitle, rename to keywordInPageTitle.
• calculateFleschReading, rename to fleschReadingEase, change it so it uses the new syllableCount research.
• getLinkStatistics, drop in favor of links research
• getLinks, rename to links, change so it returns a completely object instead of only an URL.
• linkCount, drop in favor of links research
• imageCount, change to images research. Count can easily be retrieved by doing .length in the assessment. The research should return image objects with all the data for an image so it is a lot more useful
• altTagCount, drop in favor of images research. Logic for counting how many images with/without tags should be in the assessment.
• matchKeywordInSubheadings, drop in favor of a combination of the headings and keywords research. The assessment can first get references to all the headings in the text and then ask if the keyword has been matched in these headings by calling getResearchForNode( 'keywords', heading ).
• keywordCount, change to keywords research. Count can easily be retrieved by doing .length in the assessment. The research should return keyword objects with keyword, startOffset, endOffset, isExactMatch properties.
• getKeywordDensity, rename to keywordDensity, change to use the keywords and words research.
• stopWordsInKeyword, unchanged
• stopWordsInUrl, unchanged
• metaDescriptionLength, unchanged
• keyphraseLength, unchanged
• keywordCountInUrl, unchanged
• findKeywordInFirstParagraph, drop in favor of a combination of the paragraphs and keywords research. The assessment can first get a reference to the first paragraph and the ask if the keyword has been matched in this paragraph by calling getResearchForNode( 'keywords', firstParagraph ).
• metaDescriptionKeyword, unchanged
• pageTitleWidth, unchanged
• getWordComplexity, drop in favor of the words research, a word object should include the amount of syllables in the word
• getParagraphLength, **drop in favor of a combination of the paragraphs and words research. An assessment can get all the paragraphs with the paragraph research and then retrieve the words for this specific paragraph by calling getResearchForNode( 'words', paragraph )
• countSentencesFromText, change to sentences research.
• countSentencesFromDescription, rename to descriptionSentences
• getSubheadingTextLengths, drop in favor of a combination of the headings and words research
• findTransitionWords, rename to `transitionWords
• passiveVoice, rename research file to passiveVoice
• getSentenceBeginnings, we should probably not change this until we implement the linguistic tree
• relevantWords, adapt for the tree, but this is going to require a very specifically tuned recursion strategy. We can also choose to refactor this later and only make this work on the root node for now.
• readingTime, unchanged, but adapt for the tree
• getTopicDensity, drop in favor of a combination of matchedTopics and words
• topicCount, rename to matchedTopics, should return an object with properties matchedTopic, startOffset, endOffset.
• sentences, unchanged, but works for the tree.
• keyphraseDistribution, refactor to use the new words/keywords/matchedTopics research
• morphology/buildKeywordsForms, rename to keywordForms
• functionWordsInKeyphrase, unchanged
• h1s, drop in favor of the headings research. That one can then easily be filtered based on the level (in this case 1)

As a result of the above changes we also need to introduce some new research:

• words, returns all the words for a specific node. Every word has a
• keywords, returns all the keywords for a specific node.
• subheadings, returns a reference to all the Heading nodes in the tree.
• paragraph, returns a reference to all the Paragraph nodes in the tree.

[igorschoester]

I checked the current issues with the label requires structured data / html parser to match them to these researches. The goal is to make it easier to double check if the issues are indeed fixed after/while creating the tree researches.

This is a work in progress, at this time there are 6 issues left to match.

To match still

• https://github.com/Yoast/YoastSEO.js/issues/1418
• https://github.com/Yoast/YoastSEO.js/issues/701
• https://github.com/Yoast/YoastSEO.js/issues/744
• https://github.com/Yoast/YoastSEO.js/issues/1087
• https://github.com/Yoast/YoastSEO.js/issues/1082
• https://github.com/Yoast/YoastSEO.js/issues/1108

getSentenceBeginnings

• Ignore lists for this research https://github.com/Yoast/YoastSEO.js/issues/1190

headings/subheadings

• Match paragraphs between headings https://github.com/Yoast/YoastSEO.js/issues/463
• Mark the subheading (text included) currently disabled https://github.com/Yoast/YoastSEO.js/issues/1151
• Mark headings with attributes properly https://github.com/Yoast/YoastSEO.js/issues/1998
• Do not mark the first H1 when it is the first content (probably a same text as the second H1 issue that should be resolved after implementing indices) https://github.com/Yoast/YoastSEO.js/issues/2034

getParagraphLength

• Mark the paragraph (text included) https://github.com/Yoast/YoastSEO.js/issues/1151

getSubheadingTextLengths

• Re-enable marking and check (the connected assessment has a unclear name SubheadingsDistributionTooLong; distribution can not be too long) https://github.com/Yoast/YoastSEO.js/issues/1195

getKeywordDensity

• Not all occurrences are marked + possibly mark sentences instead of the words to be less confusing (see the comment) https://github.com/Yoast/YoastSEO.js/issues/2029

keyphraseDistribution

• Mark sentence/paragraph with links (text included) https://github.com/Yoast/YoastSEO.js/issues/1907
• Can get picked up after marking keywords works again https://github.com/Yoast/YoastSEO.js/issues/1498

links

• Mark links currently disabled https://github.com/Yoast/YoastSEO.js/issues/1155

images

• Check correctly counting cat's as keyphrase when used in the image alt tag https://github.com/Yoast/YoastSEO.js/issues/1209

[atimmer]

List with priority, based on a deliberation with @moorscode.

Sub Headings Keyword

• Match paragraphs between headings #463
• Mark the subheading (text included) currently disabled #1151
• Mark headings with attributes properly #1998
• Do not mark the first H1 when it is the first content (probably a same text as the second H1 issue that should be resolved after implementing indices) #2034
• https://github.com/Yoast/wordpress-seo/issues/11660
• https://github.com/Yoast/YoastSEO.js/issues/2155

Internal Links

• Mark links currently disabled #1155

Text Competing Links ( currently disabled)

• Mark links currently disabled #1155

Keyphrase Distribution

• Mark sentence/paragraph with links (text included) #1907
• Can get picked up after marking keywords works again #1498

Keyword Density

• Not all occurrences are marked + possibly mark sentences instead of the words to be less confusing (see the comment) #2029
• https://github.com/Yoast/YoastSEO.js/issues/2155

sentence Beginnings

• Ignore lists for this research #1190

• https://github.com/Yoast/YoastSEO.js/issues/785 paragraph Too Long

• Mark the paragraph (text included) #1151

subheading Distribution Too Long (currently disabled)

• Re-enable marking and check (the connected assessment has a unclear name SubheadingsDistributionTooLong; distribution can not be too long) #1195

Text Images

• Check correctly counting cat's as keyphrase when used in the image alt tag #1209

Outbound Links passive Voice sentence Length In Text transition Words Keyphrase Length Single H1

Geen marking

Introduction Keyword keyword Stop Words Function Words In Keyphrase text Presence sentence Length In Description flesch Reading Ease Meta Description Keyword Meta Description Length Page Title Width taxonomy Text Length Url Keyword Url Length url Stop Words Text Length Title Keyword

Assessment disabled

word Complexity (assessment disabled)

[manuelaugustin]

Other issues that should be solved after the implementation of the tree (not specific to a given assessment, but possibly researches):

• Exclude tables https://github.com/Yoast/YoastSEO.js/issues/820
• If Keyword contains an ending punctuation like period exclamation point or question mark do not count it as a sentence ending https://github.com/Yoast/YoastSEO.js/issues/2218
• If Keyword contains an ending punctuation like period exclamation point or question mark be able to find it for Keyword Density https://github.com/Yoast/wordpress-seo/issues/13726
• SentenceTokenizer incorrectly processes punctuation marks within words #402

[omarreiss]

Closing all parse tree issues.