Incorrect display of some elements on documentation pages. Broken links.

sabina-talipova commented 5 months ago

Description

On some documentation pages, elements such as [notes] or [hint] are displayed as plain text instead of being properly styled.
It is necessary to check all documentation for problems with styles.

Some examples:

https://docs.silverstripe.org/en/5/developer_guides/templates/syntax/. Search for [note]
https://docs.silverstripe.org/en/5/developer_guides/forms/introduction/. Search for [how_tos/handle_nested_data].
https://docs.silverstripe.org/en/5/developer_guides/forms/field_types/htmleditorfield/ Search for [hint].

Related issue

https://github.com/silverstripe/doc.silverstripe.org/issues/280
- specifically https://github.com/silverstripe/doc.silverstripe.org/issues/280#issuecomment-1834597389

Updates

There is another related issue with links. The relative links to the articles on https://docs.silverstripe.org/ is not rendered correctly If they are in content of callout blocks. If link is part of plain text of paragraph everything works. E.g. : [Environment Types](/developer_guides/debugging/environment_types) will be processed in https://docs.silverstripe.org/developer_guides/debugging/environment_types instead of https://docs.silverstripe.org/en/5/developer_guides/debugging/environment_types/.

Acceptance criteria

Our MD parser is updated to use GitHub block quotes for all call outs.
All pre-existing call outs are converted to the new format.
- All changes are applied to CMS 4 branches and to the User help.
Our doc article about updating docs is updated to use the new call out format.

Notes

Call outs in the CMS 3 doc will definitely break. We've decided not to care.

PRs

Assign back to Guy after merging so they can double check everything is working correctly

emteknetnz commented 5 months ago

Talked about internally, essentially we need to either update a regex which is liable to just create issues elsewhere, or switch to a different system such as:

github blockquotes
some other way to parse the markdown

GuySartorelli commented 5 months ago

Callout blocks are converted as follows:

old syntax	new syntax
info	NOTE
note	NOTE
hint	TIP
warning	WARNING
notice	WARNING
alert	CAUTION

GuySartorelli commented 5 months ago

I used the below script for catching automatic changes (ran it separately over developer-docs, this copy is for userhelp but the only difference is the $modules array):

Click to see code

```php [ 'branch' => '4', 'remote' => 'git@github.com:creative-commoners/silverstripe-userhelp-content.git', ], 'symbiote/silverstripe-advancedworkflow' => [ 'branch' => '5', 'remote' => 'git@github.com:creative-commoners/silverstripe-advancedworkflow', ], 'silverstripe/registry' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-registry', ], 'silverstripe/contentreview' => [ 'branch' => '4', 'remote' => 'git@github.com:creative-commoners/silverstripe-contentreview', ], 'silverstripe/blog' => [ 'branch' => '3', 'remote' => 'git@github.com:creative-commoners/silverstripe-blog', ], 'silverstripe/userforms' => [ 'branch' => '5', 'remote' => 'git@github.com:creative-commoners/silverstripe-userforms', ], 'silverstripe/subsites' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-subsites', ], 'silverstripe/taxonomy' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-taxonomy', ], 'silverstripe/iframe' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-iframe', ], 'silverstripe/versionfeed' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-versionfeed', ], 'dnadesign/silverstripe-elemental' => [ 'branch' => '4', 'remote' => 'git@github.com:creative-commoners/silverstripe-elemental', ], 'bringyourownideas/silverstripe-maintenance' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-maintenance', ], 'silverstripe/sharedraftcontent' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-sharedraftcontent', ], 'silverstripe/documentconverter' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-documentconverter', ], 'silverstripe/ckan-registry' => [ 'branch' => '1', 'remote' => 'git@github.com:creative-commoners/silverstripe-ckan-registry', ], 'silverstripe/mfa' => [ 'branch' => '4', 'remote' => 'git@github.com:creative-commoners/silverstripe-mfa', ], 'silverstripe/securityreport' => [ 'branch' => '2', 'remote' => 'git@github.com:creative-commoners/silverstripe-securityreport', ], 'silverstripe/sitewidecontent-report' => [ 'branch' => '3', 'remote' => 'git@github.com:creative-commoners/silverstripe-sitewidecontent-report', ], 'silverstripe/session-manager' => [ 'branch' => '1', 'remote' => 'git@github.com:creative-commoners/silverstripe-session-manager', ], ]; $vendorDir = __DIR__ . '/vendor/'; // Execute a command, throw a wobbly if something goes wrong, and return the result otherwise. function cmd(string $command): string { $output = []; exec($command, $output, $res); $out = implode("\n", $output); if ($res !== 0) { throw new RuntimeException("Failed to execute command: $out"); } return $out; } // Makes a request to the github API function github_api($url, $data = [], $httpMethod = '') { // silverstripe-themes has a kind of weird redirect only for api requests $url = str_replace('/silverstripe-themes/silverstripe-simple', '/silverstripe/silverstripe-simple', $url); // info("Making curl request to $url"); $token = ''; // PUT YOUR TOKEN HERE $jsonStr = empty($data) ? '' : json_encode($data, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE); $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_POST, !empty($data)); if ($httpMethod) { curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $httpMethod); } curl_setopt($ch, CURLOPT_HTTPHEADER, [ 'User-Agent: silverstripe-module-standardiser', 'Accept: application/vnd.github+json', "Authorization: Bearer $token", 'X-GitHub-Api-Version: 2022-11-28' ]); if ($jsonStr) { curl_setopt($ch, CURLOPT_POSTFIELDS, $jsonStr); } $response = curl_exec($ch); $httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE); curl_close($ch); if ($httpcode >= 300) { echo "HTTP code $httpcode returned from GitHub API\n"; echo $response . "\n"; echo "Failure calling github api: $url\n"; } return json_decode($response, true); } // -_____- EXECUTION STARTS HERE $prsCreated = []; foreach ($modules as $moduleDir => $data) { $branch = $data['branch']; $remote = $data['remote']; $org = dirname($moduleDir); preg_match('#^git@github\.com:creative-commoners/(?.*)\.git$#', $remote, $matches); $repo = $org . '/' . $matches['repo']; $fullPath = $vendorDir . $moduleDir; if (!is_dir($fullPath)) { throw new LogicException("$moduleDir is missing!"); } echo "------------------\n"; echo "Starting on $moduleDir\n"; // Swap to correct branch, and create a new PR branch chdir($fullPath); cmd('git checkout ' . $branch); $prBranch = "pulls/$branch/update-callout-syntax"; cmd("git checkout -b $prBranch"); //// UNCOMMENT THIS WHEN YOU DO IT!!!! // Find and replace for all markdown files $iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($fullPath)); $mdIterator = new RegexIterator($iterator, '/^.+\.md$/i', RecursiveRegexIterator::GET_MATCH); foreach ($mdIterator as $filePath => $_) { // echo "File: $filePath\n"; $fileContent = file_get_contents($filePath); $regex = '/\h*\[(?hint|warning|info|alert|notice|note)\](?.*?)\[\/\1\]\h*/s'; $callback = function(array $matches): string { // Find out what type of callout this is $realType = match ($matches['type']) { 'info' => 'NOTE', 'note' => 'NOTE', 'hint' => 'TIP', 'warning' => 'WARNING', 'notice' => 'WARNING', 'alert' => 'CAUTION', default => throw new RuntimeException('unexpected type: ' . $matches['type']) }; $content = $matches['content']; //skip this - it may be nested if (str_starts_with($content, ' ')) { return $matches[0]; } // trim and add callout marker $content = trim($content); $result = ["> [!$realType]"]; // Loop over each line and blockquote-ify it foreach (explode("\n", $content) as $line) { $result[] = rtrim('> ' . $line); $line = strtok("\n"); } return implode("\n", $result); }; // This is the line that actually triggers the content getting replaced $newFileContent = preg_replace_callback($regex, $callback, $fileContent); if ($newFileContent === null) { throw new RuntimeException('Failed to replace content!'); } file_put_contents($filePath, $newFileContent); } // If there's no changes, move on to the next repo $diff = trim(cmd('git diff --name-only')); if (!$diff) { echo "No diff - no PR needed for this repo.\n"; continue; } // If there have been changes, commit them and make a PR cmd('git add . && git commit -m "DOC Update syntax for callout blocks"'); cmd("git remote add cc $remote"); cmd("git push cc $prBranch"); $prDesc = << 'DOC Update syntax for callout blocks', 'body' => $prDesc, 'head' => "creative-commoners:$prBranch", 'base' => $branch, ]); $prsCreated[] = $responseJson['html_url']; } echo "---------------\n"; if (empty($prsCreated)) { echo "No PRs created:\n"; } else { echo "PRs created:\n"; foreach ($prsCreated as $url) { echo "- $url\n"; } } ```

The regex is based off the regex that's currently used to render callout blocks. Since we know that doesn't perfectly catch all callout blocks I also manually ran a text search to look for any that the regex missed.

Note that most of the userhelp docs didn't have any changes, so there's no PR for them.

emteknetnz commented 5 months ago

@GuySartorelli Assigned back to you to do follow up PR

emteknetnz commented 5 months ago

@GuySartorelli have merged PRs, assigning back to you to check things work as expected

GuySartorelli commented 5 months ago

Things render correctly now! 🎉

silverstripe / doc.silverstripe.org