If an image fails to download, the fathomnet-generate script will crash. Add error handling to log an error/warning and proceed with other images. These images should be excluded from the dataset.
In the case that the image should be re-fetched and added to the dataset once the link is working, re-running the same fathomnet-generate command will not re-fetch already-downloaded images, and instead only download the working images.
If an image fails to download, the
fathomnet-generatescript will crash. Add error handling to log an error/warning and proceed with other images. These images should be excluded from the dataset.In the case that the image should be re-fetched and added to the dataset once the link is working, re-running the same
fathomnet-generatecommand will not re-fetch already-downloaded images, and instead only download the working images.