Update file system to support list of included and excluded files

[pschellenbach]

@brentblair proposed an update to FTFSI in PR#1, which would supply a pre-defined list of files in response to the ITEM LIST function. While PR#1 was rejected, and a workaround using RCS hooks was provided, this is a good idea and one of the most frequent requests when using AccuTerm as the file system provider for MVBasic/VSCode. This feature is more important when using VSCode since the VSCode file system provider is always requesting a list of files from the root of the file system, requiring a scan of the account MD. The wED Editor would only request the file list when using the file browser.

It turns out that the ITEM LIST function in FTFSI already has a mechanism to exclude files/items using a user-defined list of patterns, and the excluded file list can be updated from the wED Editor file browser. Any items that match one of the patterns in this list is tagged with the "Hidden" file attribute. Normally, hidden files are not returned in the item list.

There are several drawbacks to the existing Hidden Files function:

• only applies to files in the MD or VOC, not to normal items at any level.
• only simple patterns are supported: exact match, left, right or middle substring match.
• no way to customize the hidden files for different accounts.
• no way to customize the behavior for different applications (wED, GED, vsCode).
• uses a very simple data structure (MV delimited list of patterns)

TODO:

• add an "Include Files" features
• extend data structure to allow for more specific behavior depending on file/item type, account, application
• enhance the pattern syntax

[pschellenbach]

The "include files" feature has been added to the existing MVCONNECTION.OPTIONS item in the ACCUTERMCTRL file. This item now has 3 attributes:

001 X
002 <list of "hidden files> separated by VM
003 <list of "included files" separated by VM

This may not be the ideal way to handle this but it will work and is simple. The file list in attribute 2 or 3 is a value-mark delimited list, and may contain exact or partial file names:

pattern	meaning
NAME	exactly matches a file named "NAME"
[NAME	filename must end with "NAME"
NAME]	filename must begin with "NAME"
[NAME]	filename must contain "NAME"

The file name patterns in this item only apply to files at the MD/VOC level, and are not case-sensitive.

When the content of MD/VOC is requested, and the selection attributes include "datafiles only", if the "included list" only contains exact names (no substrings), the normal file scan of the MD/VOC is bypassed and only the listed files are returned. This may result in better performance, especially if the MD/VOC is large.

Note: the default selection attributes do not include "datafiles only", as used by the wED Editor and GUI designer. For VSCode, the selection attributes are specified in the workspace template, and MVBasic.RestFS.SelAttr specifies the selection attributes. The template default is "folder", "item", "alias" and "datafiles only", so using the "included files" list may increase performance when used with VSCode. Also note that VSCode always requests a file list for MD/VOC; other programs like wED only request the MD/VOC file list when using the file browser.

[itsxallwater]

Looking good Pete. To clarify, VS Code is always requesting that list because part of the UI is the "solution explorer" section that lists all directories and files in the workspace or directory you have open, correct?

[pschellenbach]

Actually VSCode solution explorer always requests the root directory list when starting, which maps to MD/VOC when using the AccuTerm VSCode connector.