Closed delhomer closed 4 years ago
No real improvement with generator, all the CPUs are active as well, and the measured time is not really better than with np.array
inputs.
There are two additional drawbacks regarding the generators (one uses the keras
API, and flow_from_directory
, see https://github.com/Oslandia/deeposlandia/blob/cd56791f4d38d0857313649082d176b979c27902/deeposlandia/generator.py#L144) :
flow_from_directory
means that all the images within the targetted directory will be evaluated, this being incompatible with the current folder management in Deeposlandia.I close this issue for now, we might open it again if the question comes back later.
The batch size parameter does not seem to have any impact on the system resource usage. One could expect that providing a smaller batch size would kind of "save" memory at the price of longer processing times.
However it does not fit this goal:
batch_size = 1
real 2m24,930s user 6m45,306s sys 0m12,090s
11:20 $ time deepo postprocess -D tanzania -b 20 -i grid_034 -s 384 Using TensorFlow backend. 2020-05-06 11:23:46,304 :: INFO :: postprocess :: main : Raw image size: 3850, 3860 2020-05-06 11:23:46,306 :: INFO :: postprocess :: main : The image will be splitted into 121 tiles 2020-05-06 11:23:50,833 :: INFO :: postprocess :: get_trained_model : Model weights have been recovered from ./data/tanzania/output/semseg/checkpoints/best-model-384.h5 2020-05-06 11:23:50,833 :: INFO :: postprocess :: main : Predict labels for input images... 121/121 [==============================] - 158s 1s/step 2020-05-06 11:26:29,071 :: INFO :: postprocess :: main : Labelled image dimension: 3860, 3850
real 2m55,432s user 6m47,825s sys 0m47,111s