Closed dbuscombe-usgs closed 1 year ago
This PR includes fixes for https://github.com/Doodleverse/doodleverse_utils/issues/27 and https://github.com/Doodleverse/segmentation_gym/issues/117
Added options to the wiki https://github.com/Doodleverse/segmentation_gym/wiki/04_Creation-of-%60config%60-files
I made a few more minor edits I have made on the checkpoints branch. These deal with
save_weights
does not make the "fullmodel" version of the weights, which is now dealt with by a try/except loop because it happens apparently randomly (probably a bug in the tensorflow build), and it is not crucialseg_images_in_folder
script to modify outputsseg_images_in_folder
script - it works sometimes, but randomly errors out almost every time. It is an unreproducible bug. It is left in as a hidden (experimental) method while we figure out a better alternativePlease review this soon. I'd like to get it incorporated so it is up to date with the rest of the doodleverse, which is now using doodleverse version 29 https://github.com/Doodleverse/doodleverse_utils/releases/tag/v0.0.29
This PR ...
CLEAR_MEMORY
[default=False] will clear the keras session and carry out garbage collection at the end of each training epoch. This is for memory leaks that have occurred on large datasetsLOAD_DATA_WITH_CPU
[default=False] will use the CPU to do the tf.data pipeline, which may also help memory leaks or other memory issuesrun_eagerly=True
is removed (it is not needed)adds parallel processing option
do_parallel
to theseg_images_in_folder
script, which applies models to imagery in parallel using joblibadded a new/updated conda yml and recipe for windows and linux