ThioJoe / YT-Spammer-Purge

Allows you easily scan for and delete scam comments using several methods.
GNU General Public License v3.0
4.57k stars 389 forks source link

JSON schema and yaml support #1068

Open EmilyGraceSeville7cf opened 1 year ago

EmilyGraceSeville7cf commented 1 year ago

Related Issue/Addition to code

Type of change

Proposed Changes

JSON schema generator and converter to .ini file.

Why is this change needed?

It's needed because there is no intellisence available for .ini files. ### Additional Info This script allows to: - generate JSON schema for YAML config (from remote default INI config in this repo) - rewrite INI config from YAML config (not to write INI directly) It means that users can configure tool via YAML having intellisence and validation. Script is written in [Nushell](https://www.nushell.sh/book/) and is already ported to POSIX sh. Currently this schema is automatically generated: ```json { "$schema": "http://json-schema.org/draft-07/schema", "title": "config", "description": "A config", "type": "object", "properties": { "config-meta": { "title": "config-meta", "description": "config-meta\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "use_this_config": { "title": "use this config", "description": "use this config\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "boolean" ], "default": "Ask" }, "this_config_description": { "title": "this config description", "description": "this config description\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "My Custom Config" }, "configs_path": { "title": "configs path", "description": "configs path\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "configs" } }, "additionalProperties": false }, "general": { "title": "general", "description": "general\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "your_channel_id": { "title": "your channel id", "description": "your channel id\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "auto_check_update": { "title": "auto check update", "description": "auto check update\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": true }, "release_channel": { "title": "release channel", "description": "release channel\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "All" }, "skip_confirm_video": { "title": "skip confirm video", "description": "skip confirm video\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false }, "moderator_mode": { "title": "moderator mode", "description": "moderator mode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false }, "auto_close": { "title": "auto close", "description": "auto close\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false }, "colors_enabled": { "title": "colors enabled", "description": "colors enabled\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": true } }, "additionalProperties": false }, "scanning": { "title": "scanning", "description": "scanning\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "scan_mode": { "title": "scan mode", "description": "scan mode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "max_comments": { "title": "max comments", "description": "max comments\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "integer" ], "default": "Ask" }, "videos_to_scan": { "title": "videos to scan", "description": "videos to scan\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "channel_to_scan": { "title": "channel to scan", "description": "channel to scan\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "recent_videos_amount": { "title": "recent videos amount", "description": "recent videos amount\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "integer" ], "minimum": 1, "maximum": 5000, "default": "Ask" } }, "additionalProperties": false }, "filtering": { "title": "filtering", "description": "filtering\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "filter_mode": { "title": "filter mode", "description": "filter mode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "filter_subMode": { "title": "filter submode", "description": "filter submode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "channel_ids_to_filter": { "title": "channel ids to filter", "description": "channel ids to filter\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "autoASCII_sensitivity": { "title": "autoascii sensitivity", "description": "autoascii sensitivity\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "integer" ], "minimum": 1, "maximum": 3, "default": "Ask" }, "characters_to_filter": { "title": "characters to filter", "description": "characters to filter\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "strings_to_filter": { "title": "strings to filter", "description": "strings to filter\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" }, "regex_to_filter": { "title": "regex to filter", "description": "regex to filter\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "Ask" } }, "additionalProperties": false }, "autosmart-customization": { "title": "autosmart-customization", "description": "autosmart-customization\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "detect_link_spam": { "title": "detect link spam", "description": "detect link spam\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": true }, "detect_sub_challenge_spam": { "title": "detect sub challenge spam", "description": "detect sub challenge spam\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false }, "detect_spam_threads": { "title": "detect spam threads", "description": "detect spam threads\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": true } }, "additionalProperties": false }, "advanced-filtering": { "title": "advanced-filtering", "description": "advanced-filtering\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "duplicate_check_modes": { "title": "duplicate check modes", "description": "duplicate check modes\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "AutoSmart, SensitiveSmart" }, "minimum_duplicates": { "title": "minimum duplicates", "description": "minimum duplicates\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "integer", "minimum": 3, "default": 8 }, "minimum_duplicate_length": { "title": "minimum duplicate length", "description": "minimum duplicate length\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "integer", "minimum": 1, "default": 14 }, "levenshtein_distance": { "title": "levenshtein distance", "description": "levenshtein distance\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "number", "minimum": 0, "maximum": 1, "default": 0.9 }, "stolen_comments_check_modes": { "title": "stolen comments check modes", "description": "stolen comments check modes\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "SensitiveSmart" }, "stolen_minimum_text_length": { "title": "stolen minimum text length", "description": "stolen minimum text length\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "integer", "minimum": 1, "default": 25 }, "fuzzy_stolen_comment_detection": { "title": "fuzzy stolen comment detection", "description": "fuzzy stolen comment detection\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false } }, "additionalProperties": false }, "removal": { "title": "removal", "description": "removal\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "skip_deletion": { "title": "skip deletion", "description": "skip deletion\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false }, "delete_without_reviewing": { "title": "delete without reviewing", "description": "delete without reviewing\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false }, "enable_ban": { "title": "enable ban", "description": "enable ban\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "boolean" ], "default": "Ask" }, "remove_all_author_comments": { "title": "remove all author comments", "description": "remove all author comments\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "boolean" ], "default": "Ask" }, "removal_type": { "title": "removal type", "description": "removal type\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "rejected" }, "whitelist_excluded": { "title": "whitelist excluded", "description": "whitelist excluded\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "boolean" ], "default": "Ask" }, "check_deletion_success": { "title": "check deletion success", "description": "check deletion success\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": true } }, "additionalProperties": false }, "logging": { "title": "logging", "description": "logging\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "enable_logging": { "title": "enable logging", "description": "enable logging\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "boolean" ], "default": "Ask" }, "log_path": { "title": "log path", "description": "log path\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "logs" }, "log_mode": { "title": "log mode", "description": "log mode\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "plaintext" }, "json_log": { "title": "json log", "description": "json log\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false }, "json_encoding": { "title": "json encoding", "description": "json encoding\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "string", "default": "utf-8" }, "json_extra_data": { "title": "json extra data", "description": "json extra data\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": true }, "json_log_all_comments": { "title": "json log all comments", "description": "json log all comments\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "boolean", "default": false }, "json_profile_picture": { "title": "json profile picture", "description": "json profile picture\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": [ "string", "boolean" ], "default": false } }, "additionalProperties": false }, "api": { "title": "api", "description": "api\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "quota_limit": { "title": "quota limit", "description": "quota limit\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "integer", "default": 10000 } }, "additionalProperties": false }, "info": { "title": "info", "description": "info\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "object", "properties": { "config_version": { "title": "config version", "description": "config version\nhttps://raw.githubusercontent.com/ThioJoe/YT-Spammer-Purge/main/assets/default_config.ini", "type": "integer", "default": 32 } }, "additionalProperties": false } } } ``` ### Checklist: - [ ] My code follows the style guidelines of this project and I have read [CONTRIBUTING.md](/CONTRIBUTING.md) - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] Any dependent changes have been merged and published in downstream modules - [x] I have checked my code and corrected any misspellings --- ### Screenshots ![image](https://github.com/ThioJoe/YT-Spammer-Purge/assets/42812113/f95024d7-1020-42f2-a6eb-63c6653617d4)
EmilyGraceSeville7cf commented 1 year ago

Ideally, YAML should be supported by this tool itself. But as I am not Python developer, I can't integrate YAML support directly to Python code now. (Maybe later.)

EmilyGraceSeville7cf commented 1 year ago

Now it's possible to tinker ./.vscode/settings.json (just this path is checked for existence) automatically when --setup-vscode option is passed with --yaml-config <path> --schema <path> to enable intellisence for generated JSON schema in one step.