Remove invalid/outdated datasets

[DavidGrant-NIWC]

Only datasets which are intended to be used for testing should be provided in the repo. Retaining datasets with known errors which can't be loaded makes it cumbersome to install the valid datasets; it's necessary to manually traverse the folder structure and install each dataset one at a time.

For instance, DS0013 contains five folders and an additional dataset which isn't in a folder. The standalone dataset appears to be identical to the dataset in the "5" folder. Three of the datasets are unusable (folders "2", "3", and "4"). One of the datasets is on an older FC ("1"), which might be useful for testing except that it has other known issues.

Other dataset folders have similar issues.

[DavidGrant-NIWC]

Also remove extraneous files from the docs folder:

[kusala9]

or just clearly mark folders as historical and not for general consumption. I certainly find access to older versions can be useful and also gives us audit/trail of previous versions. Potentially, we just need a dev folder and a production folder with production only containing the active, most up to date version of the dataset. Of course, if we did this by compiling text files and using branches we could use github to do the change management - but that may be a step too far right now.... I would rather the historical versions of the dataset are archived and available. Agree with the sentiment here though, keep only

• versioned requirements documents
• datasets
• supporting resources (alongside only the datasets they apply to)

prune files like .vld which I don't believe are required.

[DavidGrant-NIWC]

Bad datasets should be updated in place or removed. There is a historical record of changes to the branch even if datasets are deleted.

https://github.com/iho-ohi/S-101-Test-Datasets/commits/main

[TomRichardson6]

@DavidGrant-NIWC and @kusala9 I have now tidied up these folders and retained a sub directory indicating the edition number.

This doesn't totally address David's point about it being cumbersome to install the cells so I would welcome your view on me removing these edition directories?

[DavidGrant-NIWC]

Much better - I can now drop the dev/cells folder on the ShoreECDIS and have it install all cells without errors (I did remove the zip file though).

Recommend updating the Readme.md on the repo to indicate what all the folders are for: dev vs. S-164 (recommend remove the old S-164 folder) cells / docs / exports exports 1.0 vs 2.0 (recommend remove the old folder)

[plebihan29n]

I have tried to organize the new version S-164_1.1/V2/ to clearly separate TestDataSets and other files (logs and dumps)

[DavidGrant-NIWC]

I have tried to organize the new version S-164_1.1/V2/ to clearly separate TestDataSets and other files (logs and dumps)

That is helpful. Some additional recommendations:

• The V2 files are in a zip,
  • Recommend remove the zip and provide individual files so that they can be individually updated.
  • This also supports examining the contents without having to download and extract the zip.
• There are now two versions of the S-164_1.1 datasets.
  • The V1 files should be removed.
• Having a version folder underneath the S-164_1.1 folder is not necessary.
  • Move contents of S-164_1.1/V2 to S-164_1.1
• Retaining the original S-164 folder is not necessary.
  • Recommend that the current S-164 folder is removed and S-164_1.1 is renamed to S-164.
  • If you want to provide historical releases, do so by publishing a release, or by creating a branch (or both).
• Remove the empty Readme.txt file in S-164_1.1
  • Or change it to a Readme.md and update the contents.
• Update the Readme.md on the main page: Readme.md to reflect the current contents of the repo

[plebihan29n]

These recommendations have been followed the delivered version 3 of S-164 . Next step is to deliver a S-158 package

[DavidGrant-NIWC]

The S-164_1.1 folder should be removed. The rest looks good.

[TomRichardson6]

I think this has now been addressed following various changes. Please raise new issues if anything further is identified.