Doodleverse / segmentation_gym

A neural gym for training deep learning models to carry out geoscientific image segmentation. Works best with labels generated using https://github.com/Doodleverse/dash_doodler
MIT License
45 stars 11 forks source link

Clear memory options #125

Closed dbuscombe-usgs closed 1 year ago

dbuscombe-usgs commented 1 year ago

This PR ...

  1. adds two new config options:
  1. run_eagerly=True is removed (it is not needed)

  2. adds parallel processing option do_parallel to the seg_images_in_folder script, which applies models to imagery in parallel using joblib

  3. added a new/updated conda yml and recipe for windows and linux

dbuscombe-usgs commented 1 year ago

This PR includes fixes for https://github.com/Doodleverse/doodleverse_utils/issues/27 and https://github.com/Doodleverse/segmentation_gym/issues/117

Added options to the wiki https://github.com/Doodleverse/segmentation_gym/wiki/04_Creation-of-%60config%60-files

dbuscombe-usgs commented 1 year ago

I made a few more minor edits I have made on the checkpoints branch. These deal with

  1. situations when the save_weights does not make the "fullmodel" version of the weights, which is now dealt with by a try/except loop because it happens apparently randomly (probably a bug in the tensorflow build), and it is not crucial
  2. added the 'profile' to the seg_images_in_folder script to modify outputs
  3. played with joblib's Parallel method in seg_images_in_folder script - it works sometimes, but randomly errors out almost every time. It is an unreproducible bug. It is left in as a hidden (experimental) method while we figure out a better alternative

Please review this soon. I'd like to get it incorporated so it is up to date with the rest of the doodleverse, which is now using doodleverse version 29 https://github.com/Doodleverse/doodleverse_utils/releases/tag/v0.0.29