Docs: New limitations and verifyio docs plus edits

[CamStan]

Description

Note: For easier reviewing, how the rendered version will look can be viewed here.

Adjust the table of contents layout by moving documentation containing additional information to the middle Reference section in order to keep the User Guide less cluttered. Moved the Contributing section to the bottom.

New documentation:

• Limitation and Workarounds of UnifyFS
• How to Use VerifyIO to Determine Compatibility with UnifyFS

Additional Edits:

• Add a caveat about skipping the header and mount/unmount APIs when using the auto-mount feature
• Link to VerifyIO under Semantics section
• Add mochi-margo warning for Thallium/Mochi Suite/SDS Repo users
  • Addresses documentation request from #718
• Place Developer Documentation under new top-level header so it appears on the table of contents
• Add reference section on UnifyFS Error Codes
• Minor wording and formatting edits

Motivation and Context

Documentation for recent milestone as well as some needed edits for upcoming release

How Has This Been Tested?

Rendered in test documentation repo.

• https://camstan-test-docs-unifycr.readthedocs.io/en/latest/

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Performance enhancement (non-breaking change which improves efficiency)
• [ ] Code cleanup (non-breaking change which makes code smaller or more readable)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] Testing (addition of new tests or update to current tests)
• [x] Documentation (a change to man pages or other documentation)

Checklist:

• [x] My code follows the UnifyFS code style requirements.
• [x] I have updated the documentation accordingly.
• [x] I have read the CONTRIBUTING document.
• [ ] I have added tests to cover my changes.
• [x] All new and existing tests passed.
• [x] All commit messages are properly formatted.

[CamStan]

Once available, we may also want to mention the romio_visibility_immediate hint as well (https://github.com/pmodels/mpich/issues/5902), but I believe it's still in the works.

[CamStan]

@adammoody ping. This is ready to merge.