Closed jhoneill closed 5 months ago
Using Pester 5.02 / Powershell 5.1.18362.752 and happy to help with this too, though it's a pretty broad issue to create examples for each command and duplicate the effort both online and in the offline help. I wonder what other module projects have done to automate / synch this?
I see this coming up in PSScriptAnalyzer recurse scan.
ProvideCommentHelp Information Pester.Run 78 The cmdlet 'Find-Test' does not have a help comment.
PSProvideCommentHelp Information Pester.Run 746 The cmdlet 'Is-Discovery' does not have a help comment.
PSProvideCommentHelp Information Pester.Run 1000 The cmdlet 'Invoke-PluginStep' does not have a help comment.
PSProvideCommentHelp Information Pester.Run 1680 The cmdlet 'Invoke-Test' does not have a help comment.
PSProvideCommentHelp Information Pester.Run 2372 The cmdlet 'Recurse-Up' does not have a help comment.
PSProvideCommentHelp Information Pester.Moc 345 The cmdlet 'Public' does not have a help comment.
PSProvideCommentHelp Information InModuleSc 31 The cmdlet 'PublicFunction' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 458 The cmdlet 'PublicFunction' does not have a help comment.
.ps1 not have a help comment.
.ps1 have a help comment.
PSProvideCommentHelp Information Mock.Tests 481 The cmdlet 'PublicFunction' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 484 The cmdlet 'PublicFunction2' does not have a help comment.
.ps1 have a help comment.
PSProvideCommentHelp Information Mock.Tests 494 The cmdlet 'ScopeTest' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 1046 The cmdlet 'PublicFunction' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 1340 The cmdlet 'PublicFunction' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 1352 The cmdlet 'PublicFunction' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 2203 The cmdlet 'f' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 2252 The cmdlet 'f' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 2265 The cmdlet 'f' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 2441 The cmdlet 'f' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 2504 The cmdlet 'f' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 2555 The cmdlet 'f' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 2565 The cmdlet 'f' does not have a help comment.
PSProvideCommentHelp Information Mock.Tests 2581 The cmdlet 'f' does not have a help comment.
Help works for Invoke-Pester on this machine
get-help invoke-pester
NAME
Invoke-Pester
SYNTAX
Invoke-Pester [[-Path] <string[]>] [-ExcludePath <string[]>] [-TagFilter <string[]>] [-ExcludeTagFilter <string[]>] [-FullNameFilter <string[]>] [-CI] [-Output {Diagnostic | Detailed
| Normal | Minimal | None}] [-PassThru] [<CommonParameters>]
Invoke-Pester [[-Path] <string[]>] [[-FullNameFilter] <string[]>] [[-EnableExit]] [[-TagFilter] <string[]>] [-ExcludeTagFilter <string[]>] [-PassThru] [-CodeCoverage <Object[]>]
[-CodeCoverageOutputFile <string>] [-CodeCoverageOutputFileEncoding <string>] [-CodeCoverageOutputFileFormat {JaCoCo}] [-Strict] [-OutputFile <string>] [-OutputFormat {NUnitXml |
JUnitXml}] [-Quiet] [-PesterOption <Object>] [-Show {None | Default | Passed | Failed | Pending | Skipped | Inconclusive | Describe | Context | Summary | Header | Fails | All}]
[<CommonParameters>]
Invoke-Pester [-Configuration <PesterConfiguration>] [<CommonParameters>]
ALIASES
None
REMARKS
None
Should -?
NAME
Should
SYNOPSIS
Should is a keyword what is used to define an assertion inside It block.
SYNTAX
Should [[-ActualValue] <Object>] [<CommonParameters>]
DESCRIPTION
Should is a keyword what is used to define an assertion inside the It block.
Should provides assertion methods for verify assertion e.g. comparing objects.
If assertion is not met the test fails and an exception is throwed up.
Should can be used more than once in the It block if more than one assertion
need to be verified. Each Should keywords need to be located in a new line.
Test will be passed only when all assertion will be met (logical conjuction).
RELATED LINKS
https://github.com/pester/Pester/wiki/Should
about_Should
about_Pester
REMARKS
To see the examples, type: "get-help Should -examples".
For more information, type: "get-help Should -detailed".
For technical information, type: "get-help Should -full".
For online help, type: "get-help Should -online"
Examples should show examples, but currently doesn't. The online help command redirects to another page and could have the link fixed to direct to correct link.
Some more details on the Pester 5 configuration object, discovery, code coverage and example scripts and documentation for Devops pipelines would be super useful.
Using Pester 5.02 / Powershell 5.1.18362.752 and happy to help with this too, though it's a pretty broad issue to create examples for each command and duplicate the effort both online and in the offline help. I wonder what other module projects have done to automate / synch this?
It's not trivial, for sure
Help works for Invoke-Pester on this machine That's the base "system help" but there is nothing written to explanation the parameters, give examples, etc just what Get-command returns, formatted in help style.
The online docs are mostly generated based on the comment based help, we just need to set it up to generate also from v5.0 branch.
I plan to start on the docs soon, are there any areas that you'd be interested in helping with?
With some guidance would like to help with Mocking documentation and some of the examples in the code. I think the most powerful feature of Pester is its mocking functionality and its understanding of underlying Powershell modules. This helps a lot when things like Azure Powershell change API signatures.
I think it would be hard to update the documentation without a code scanner for the examples, to identify legacy syntax. Could a list of keywords be provided for legacy syntax, their Pester 5 mappings, and list of APIs to document/currently documented as a text file in the repo? This could then be used to search the codebase and generate a report, similar to PsScriptAnalyzer warnings.
How do we automatically "test" (perhaps using Pester?) that the documentation is up to standard? What is the standard for Powershell Get-Help?
A few ideas for tests.
Perhaps pulling the code within documentation out as separate PS1 files then including afterwards would make it easier to debug/test on each build. And external URLs outside of the Pester wiki could be moved to the repo or documentation web site so they can be scanned for 404's / security and curated for relevance.
eg. get-help about_Mocking shows legacy code & legacy examples & blog posts. These are still helpful but probably shouldn't be in the main offline help and should be scanned regularly for security & 404s.
An ideas for managing this as part of the Github workflow
There are a few "quick hits" here for Migration Guides - Pester v4 to v5 - can reference some materials from issues on the project? Adding a label to tag these issues and surface as changes to document within Pester 5 would make sense?
Test Results, Report Generator & Code Coverage are probably priority items to document further to ease experience implementing in Azure Devops and other CI tools.
I can also help with the documentation. This would be a good time to use PlatyPS to be able to ship updatable help.
I made some progress on the PR, updating synopsis and pulling in some configuration properties, tagging some parameters as deprecated.
Will need some assistance implementing PlatyPS in the build. Probably separate PR.
I don't think we need PlatyPS, there was a PR for it open (or maybe still is) for few years, but it did not go anywhere. We already generate all the command related content for pester.dev from the comment based help. What more would we get from platyps?
https://github.com/pester/docs/pull/54#issuecomment-683792791 will add v5 docs (and a dropdown to switch to v4) and then I will move the stuff from readme there, and revert back to the normal readme.
Then we can finally start grinding on the new docs. It's been a long time overdue.
See #720 for platyps comments. I guess you removed all the markdown /docs previously. I think they're useful for browsing the documentation from Github and would fit well in currently almost empty /docs folder. Or restructure the docs folder itself since it doesn't really contain docs.
If there is already a pattern / tool in place, is there a document identifying the steps to update the comment docs as part of the PR merge / release?
Including some docs outside of comment-based help would also be beneficial - for common setup in Azure Devops / GitHub pipelines for example... There's a lot of info which could be surfaced from the PRs and Issues.
I don't think we need PlatyPS, there was a PR for it open (or maybe still is) for few years, but it did not go anywhere. We already generate all the command related content for pester.dev from the comment based help. What more would we get from platyps?
Two or three things.
None of those is overwhelming but big projects tend to outgrow comment based help (which makes much more sense when starting).
Folks, we are currently using https://docusaurus-powershell.netlify.app/docs/faq/multi-line-examples. This basically is PlatyPS on steroids for multi-line code-fence support.
It auto-generates the online command reference with Get-Help pages for all Pester functions.
Seems much better than PlatyPS. Is it possible to generate markdown to /docs folder as part of the release process to get the markdown help back? Assuming it would be generated in synch with the web docs.
Can the docusaurus step be documented in the repo with steps to generate the docs? I didn't see anything in the devops pipeline. Have started this in #1647 Build.md
Is there a template or set of standards for the about_help in Pester or should put one together?
@asears in all honesty it's still only a PlatyPS post-processor, even though it is pretty good.
Let me try to give an overview:
there is a PR open at pester/docs that implements the v4 and v5 version (PR link, site preview link). This will put in place the required docusaurus structure with "frozen" documentation for version4 and "current" documentation for v5. See the version dropdown in the preview, top-left
That v5 preview has a "command reference" specific to Pester v5, produced by parsing the Pester v5 Get-Help.
There is already a file describing the steps to generate above command reference, see this file. You can use that file to give it a go locally.
We have always wanted to integrate it with the Pester build pipeline. However, nobody ever picked up that task, we need help there. See this ticket.
Edit this Page
button)Thanks guys, yet another time I am the slow link here, wanted to work on docs this weekend but got caught up into adding profiler to powershell
Will try to review and get the v4/v5 PR merged by tomorrow.
Folks, the pester.dev website has been updated and now services the v5 documentation.
Requires some love and help to e.g.:
I have a draft PR #1647 for the second issue and can help with this. Should add another issue to move current readme information (and other github issue info) into the v5 documentation? There are some rather large github issue threads containing lots of detail which could be of use in the docs.
Some items I noticed so far.
new-pesteroption
BeforeDiscovery
In
It
BeforeEach-and-AfterEach
BeforeEach
AfterEach
BeforeAll
AfterAll
SYNOPSIS Missing .LINK property (some may be internal)
Add-ShouldOperator
BeforeDiscovery
In
InModuleScope
Set-DynamicParameterVariable
New-MockObject
Should-InvokeVerifiable
Should-Invoke
Should-InvokeVerifiable
Set-ItResult
Should-Be (and most of the assertions)
Missing SYNOPSIS
Get-AllMockBehaviors
Get-VerifiableBehaviors
As part of PR for new features or contributor guidelines, should a checklist be put together, should it include documentation updates?
Should wiki links be removed from all code after replacement with pester.dev links?
Should some custom tests be put in for missing .LINK, missing .SYNOPSIS etc as part of the code PR? I see Pester.Tests.ps1 already contains some of this. Is there a list reproducing above lists in the build?
I see commit comments regarding .LINK needing to be 1 per link. Should multiple links be separated into 1 per .LINK? eg. should be
.LINK
about_should
.LINK
about_pester
@asears thanks for taking the time to look into this with so much detail. I cannot answer all questions but hope this is a good enough start to get you going. Any contribution is more than welcomed. ❤️
Turning content in "large" github issue conversations into a web page is a very good idea. Feel free to submit a PR.
I will need to look into this to confirm but I think Docusaurus is generating fresh maps with every release.
The LINK
element is optional. It can be left out of the source code without any issue, pages will still render as expected.
Synopsis should never be missing IMO.
Links still pointing to the deprecated wiki should definitely be replaced with links pointing to the pester.dev website
This sounds like a great idea to me.
Get-help expects one single link/reference per .LINK
entry. Stick to that convention and the correlation web pages will be rendered correctly. I would personally insert a whiteline separator in the example you gave above.
Not sure if this belongs here, but might add the about_Pester
help file is very outdated (v3?). E.g. it mentions New-Fixture which is not available atm in Pester v5 (found while looking at #1723 )
How can we best document the assertion operators? The private functions like Should-BeOfType
are usually documented, but they are hidden behind the public Should
which only include a few examples. Moving/copying all examples to Should would make that page huge. Tips? Document them as static content and add links in Should?
Did the move to Docusaurus.Powershell stop the plan for markdown-first help, https://github.com/pester/Pester/issues/720#issuecomment-457475603?
@fflaten I'm not unsure I understand what that markdown-first
comment means but IMO Docusaurus.Powershell is markdown-first. You can put any markdown element like links, bold, italic, etc. in the Get-Help and Docusaurus.Powershell will render it as markdown on the function-page.
I read the mentioned issue and comment as a plan to replace the current comment based help with external markdown files as source and use PlatyPS to create MAML xml help for the module.
Could includes of .ps1 files be used with examples or linked to using Docusaurus?
Would it make sense to reference longer examples in the Pester/docs/Examples folder as another canonical example like planets?
It could be useful if the examples were analyzed, testable and visible outside or via links from the documentation, perhaps as a separately cloneable playground or community script repo. This probably just involves some parsing of the synopsis header and extracting to files?
Will see if there's other ways to expose the nuggets in help of private functions and comments in code.
I found this which may help provide some ideas. https://get-powershellblog.blogspot.com/2017/05/the-classy-platyps-automated-class-enum.html
@asears could you please open a new ticket for this one and then add a concrete example of what you are aiming to achieve so I can understand better?
Closing as most items are fixed and the issues has grown too large. Opening new issues for any remaining work.
Could includes of .ps1 files be used with examples or linked to using Docusaurus?
Would it make sense to reference longer examples in the Pester/docs/Examples folder as another canonical example like planets?
I believe this is covered by pester/docs#268. We already use mdx partials to reuse samples but it could easily be a docusaurus component referencing files when a larger example project is created. 🙂
1. General summary of the issue
Pester's documentation is not up to the same standard as the rest of the module.
2. Describe Your Environment
PowerShell 5, 7.0.3, Windows Terminal, VS Code,
3. Expected Behavior
Should -?
gives valid / useful help.get-help about_Should
worksInvoke-pester -?
returns parameter help.etc.
4.Current Behavior
The remarks are added by the help system but there are no examples. The URL is no longer current. Help for a command shouldn't be in an about topic
This may be a consequence of how about_ help is processed and having multiple pester versions present. Because of syntax changes there may be old and new versions present on a system, VS code installs a version in parallel with any the user has etc.
Invoke-Pester
seems to be missing help entirely.5. Possible Solution
It needs input from more than one person (and I'm happy to be one of the people) to fill in the gaps, make corrections, and generally bring the help up the same standard as a the code. As with PowerShell itself this should include people who wouldn't consider adding to the code.