Open karmeye opened 1 year ago
pjkaufman
commented
1 year ago Thanks for reporting this @karmeye . Could you provide your data.json so I can try to reproduce this.
The Linter is not able to garuantee that comments will not be removed, but it will try to preserve them as best it can.
karmeye
commented
1 year ago {
"ruleConfigs": {
"Escape YAML Special Characters": {
"Escapes colons with a space after them (: ), single quotes ('), and double quotes (\") in YAML.": false,
"Try to Escape Single Line Arrays": false
},
"Force YAML Escape": {
"Escapes the values for the specified YAML keys.": false,
"Force YAML Escape on Keys": ""
},
"Format Tags in YAML": {
"Remove Hashtags from tags in the YAML frontmatter, as they make the tags there invalid.": true
},
"Format Yaml Array": {
"Allows for the formatting of regular yaml arrays as either multi-line or single-line and `tags` and `aliases` are allowed to have some Obsidian specific yaml formats. Note that single string to single-line goes from a single string entry to a single-line array if more than 1 entry is present. The same is true for single string to multi-line except it becomes a multi-line array.": false,
"Format yaml aliases section": false,
"Format yaml tags section": true,
"Default yaml array section style": "multi-line",
"Format yaml array sections": true,
"Force key values to be single-line arrays": "",
"Force key values to be multi-line arrays": "tags"
},
"Insert YAML attributes": {
"Inserts the given YAML attributes into the YAML frontmatter. Put each attribute on a single line.": true,
"Text to insert": "tags: "
},
"Move Tags to Yaml": {
"Move all tags to Yaml frontmatter of the document.": false,
"Body tag operation": "Nothing",
"Tags to ignore": ""
},
"Remove YAML Keys": {
"Removes the YAML keys specified": false,
"YAML Keys to Remove": ""
},
"YAML Key Sort": {
"Sorts the YAML keys based on the order and priority specified. Note: may remove blank lines as well.": true,
"YAML Key Priority Sort Order": "↑",
"Priority Keys at Start of YAML": true,
"YAML Sort Order for Other Keys": "Ascending Alphabetical"
},
"YAML Timestamp": {
"Keep track of the date the file was last edited in the YAML front matter. Gets dates from file metadata.": true,
"Date Created": true,
"Date Created Key": "created",
"Date Modified": true,
"Date Modified Key": "modified",
"Format": "YYYY-MM-DDTHH:mm:ssZ"
},
"YAML Title": {
"Inserts the title of the file into the YAML frontmatter. Gets the title from the first H1 or filename if there is no H1.": true,
"Title Key": "title"
},
"YAML Title Alias": {
"Inserts the title of the file into the YAML frontmatter's aliases section. Gets the title from the first H1 or filename.": false,
"Preserve existing aliases section style": true,
"Keep alias that matches the filename": false,
"Use the YAML key `linter-yaml-title-alias` to help with filename and heading changes": false
},
"Capitalize Headings": {
"Headings should be formatted with capitalization": true,
"Style": "Title Case",
"Ignore Cased Words": true,
"Ignore Words": "macOS, iOS, iPhone, iPad, JavaScript, TypeScript, AppleScript",
"Lowercase Words": "via, a, an, the, and, or, but, for, nor, so, yet, at, by, in, of, on, to, up, as, is, if, it, for, to, with, without, into, onto, per, är, det, att, så, för, med"
},
"File Name Heading": {
"Inserts the file name as a H1 heading if no H1 heading exists.": true
},
"Header Increment": {
"Heading levels should only increment by one level at a time": false
},
"Footnote after Punctuation": {
"Ensures that footnote references are placed after punctuation, not before.": false
},
"Move Footnotes to the bottom": {
"Move all footnotes to the bottom of the document.": false
},
"Re-Index Footnotes": {
"Re-indexes footnote keys and footnote, based on the order of occurrence (NOTE: This rule deliberately does *not* preserve the relation between key and footnote, to be able to re-index duplicate keys.)": false
},
"Convert Bullet List Markers": {
"Converts common bullet list marker symbols to markdown list markers.": true
},
"Emphasis Style": {
"Makes sure the emphasis style is consistent.": true,
"Style": "underscore"
},
"No Bare URLs": {
"Encloses bare URLs with angle brackets except when enclosed in back ticks, square braces, or single or double quotes.": false
},
"Ordered List Style": {
"Makes sure that ordered lists follow the style specified. Note that 2 spaces or 1 tab is considered to be an indentation level.": false,
"Number Style": "ascending",
"Ordered List Indicator End Style": "."
},
"Proper Ellipsis": {
"Replaces three consecutive dots with an ellipsis.": true
},
"Remove Consecutive List Markers": {
"Removes consecutive list markers. Useful when copy-pasting list items.": true
},
"Remove Empty List Markers": {
"Removes empty list markers, i.e. list items without content.": true
},
"Remove Hyphenated Line Breaks": {
"Removes hyphenated line breaks. Useful when pasting text from textbooks.": false
},
"Remove Multiple Spaces": {
"Removes two or more consecutive spaces. Ignores spaces at the beginning and ending of the line. ": false
},
"Strong Style": {
"Makes sure the strong style is consistent.": true,
"Style": "asterisk"
},
"Two Spaces Between Lines with Content": {
"Makes sure that two spaces are added to the ends of lines with content continued on the next line for paragraphs, blockquotes, and list items": true
},
"Unordered List Style": {
"Makes sure that unordered lists follow the style specified.": false,
"List item style": "consistent"
},
"Compact YAML": {
"Removes leading and trailing blank lines in the YAML front matter.": true,
"Inner New Lines": true
},
"Consecutive blank lines": {
"There should be at most one consecutive blank line.": false
},
"Convert Spaces to Tabs": {
"Converts leading spaces to tabs.": false,
"Tabsize": 4
},
"Empty Line Around Blockquotes": {
"Ensures that there is an empty line around blockquotes unless they start or end a document. **Note that an empty line is either one less level of nesting for blockquotes or a newline character.**": false
},
"Empty Line Around Code Fences": {
"Ensures that there is an empty line around code fences unless they start or end a document.": true
},
"Empty Line Around Math Blocks": {
"Ensures that there is an empty line around math blocks using `Number of Dollar Signs to Indicate a Math Block` to determine how many dollar signs indicates a math block for single-line math.": false
},
"Empty Line Around Tables": {
"Ensures that there is an empty line around github flavored tables unless they start or end a document.": true
},
"Heading blank lines": {
"All headings have a blank line both before and after (except where the heading is at the beginning or end of the document).": true,
"Bottom": true,
"Empty Line Between Yaml and Header": true
},
"Line Break at Document End": {
"Ensures that there is exactly one line break at the end of a document.": true
},
"Move Math Block Indicators to Their Own Line": {
"Move all starting and ending math block indicators to their own lines using `Number of Dollar Signs to Indicate a Math Block` to determine how many dollar signs indicates a math block for single-line math.": false
},
"Paragraph blank lines": {
"All paragraphs should have exactly one blank line both before and after.": false
},
"Remove Empty Lines Between List Markers and Checklists": {
"There should not be any empty lines between list markers and checklists.": false
},
"Remove link spacing": {
"Removes spacing around link text.": true
},
"Remove Space around Fullwidth Characters": {
"Ensures that fullwidth characters are not followed by whitespace (either single spaces or a tab). Note that this may causes issues with markdown format in some cases.": false
},
"Space after list markers": {
"There should be a single space after list markers and checkboxes.": true
},
"Trailing spaces": {
"Removes extra spaces after every line.": true,
"Two Space Linebreak": true
},
"Add Blockquote Indentation on Paste": {
"Adds blockquotes to all but the first line, when the cursor is in a blockquote/callout line during pasting": false
},
"Prevent Double Checklist Indicator on Paste": {
"Removes starting checklist indicator from the text to paste if the line the cursor is on in the file has a checklist indicator": true
},
"Prevent Double List Item Indicator on Paste": {
"Removes starting list indicator from the text to paste if the line the cursor is on in the file has a list indicator": true
},
"Proper Ellipsis on Paste": {
"Replaces three consecutive dots with an ellipsis even if they have a space between them in the text to paste": false
},
"Remove Hyphens on Paste": {
"Removes hyphens from the text to paste": false
},
"Remove Leading or Trailing Whitespace on Paste": {
"Removes any leading non-tab whitespace and all trailing whitespace for the text to paste": false
},
"Remove Leftover Footnotes from Quote on Paste": {
"Removes any leftover footnote references for the text to paste": false
},
"Remove Multiple Blank Lines on Paste": {
"Condenses multiple blank lines down into one blank line for the text to paste": true
},
"Space between Chinese Japanese or Korean and English or numbers": {
"Ensures that Chinese, Japanese, or Korean and English or numbers are separated by a single space. Follows these [guidelines](https://github.com/sparanoid/chinese-copywriting-guidelines)": false
},
"Headings Start Line": {
"Headings that do not start a line will have their preceding whitespace removed to make sure they get recognized as headers.": false
},
"Remove Trailing Punctuation in Heading": {
"Removes the specified punctuation from the end of headings making sure to ignore the semicolon at the end of [HTML entity references](https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references).": false,
"Trailing Punctuation": ".,;:!。，；：！"
},
"Auto-correct Common Misspellings": {
"Uses a dictionary of common misspellings to automatically convert them to their proper spellings. See [auto-correct map](https://github.com/platers/obsidian-linter/tree/master/src/utils/auto-correct-misspellings.ts) for the full list of auto-corrected words.": false,
"Ignore Words": ""
},
"Remove Space around Characters": {
"Ensures that certain characters are not surrounded by whitespace (either single spaces or a tab). Note that this may causes issues with markdown format in some cases.": false,
"Include Fullwidth Forms": true,
"Include CJK Symbols and Punctuation": true,
"Include Dashes": true,
"Other symbols": ""
}
},
"lintOnSave": true,
"recordLintOnSaveLogs": false,
"displayChanged": true,
"foldersToIgnore": [],
"linterLocale": "system-default",
"logLevel": 4,
"lintCommands": [],
"customRegexes": [],
"commonStyles": {
"aliasArrayStyle": "single-line",
"tagArrayStyle": "multi-line",
"minimumNumberOfDollarSignsToBeAMathBlock": 2,
"escapeCharacter": "\"",
"removeUnnecessaryEscapeCharsForMultiLineArrays": false
}
}I am taking a closer look at the logic in question, but I am not sure we will be able to accommodate comments in the YAML. I am taking a look at whether or not it is feasible to preserve them, but it looks like sort is not able to preserve the comments. I am still looking, so it may still be possible in a way I have not yet considered, but so far it is looking like the removal comments is not something that is easily preventable in this case.
karmeye
commented
1 year ago So it's only an issue when key sort is enabled?
It's quite serious though, isn't it? Users whi don't know about the issue might bulk lint a folder and unknowingly lose a lot of potentially important comments.
I have no idea why it's complicated though... I appreciate your work 👍
pjkaufman
commented
1 year ago So it's only an issue when key sort is enabled?
It's quite serious though, isn't it? Users whi don't know about the issue might bulk lint a folder and unknowingly lose a lot of potentially important comments.
I have no idea why it's complicated though... I appreciate your work +1
This does seem to be limited to the key sort, but other operations can remove them as well.
The reason it is complicated to keep a comment in YAML is it is hard to tell what is a comment accurately and what it belongs to. I am not sure I have seen a good parser that would allow for this which means using an external library like is used in the Linter will not preserve the comments by default. Thus custom logic has to be added for accounting for comments. In this case, the reason it is so difficult is that sort is designed to grab the key value and ignore other values to help move them around. Exactly how this is getting rid of the comment, I am not sure yet, but it seems to be related to copying the value of a key from one place to another in the YAML.
pjkaufman
commented
1 year ago Hopefully I will have time to take a closer look at this this weekend.
breakid
commented
5 months ago There is a Visual Studio Code extension called YAML Sort that can maintain comments. Would any of the logic from that help address this issue?
Thanks for all of the work on the Linter extension!
pjkaufman
commented
5 months ago The logic that is used to sort the keys is not feasible to use unless we cut support for a lot of customization in the YAML based on what that extension does here. Several settings are passed in to the js-yaml object, but then it formats a bunch of different places. Since the Linter allows some level of customization that is not feasible if we let js-yaml do the sorting, we could use it as a reference for how things would be sorted, but we would not be able to use it as the actual thing that outputs the sorted results. This would likely still have most if not all of the same problems of the current sort logic not respecting the line continuation indicators and the comments.
In summary, it does give an idea of one way to proceed, it may not actually fix anything with this problem.
Describe the Bug
A YAML comment on its own line is removed when linting. I can't find any rule that does this.
How to Reproduce
Expected Behavior
YAML comnents should not be removed.
Device