Loading the model weights for each individual image appears to slow down inference. I took the model load from within inference, and required it as an argument to be passed at inference time. This allows the model weights to be loaded once, and then multiple images to be processed on the same model. There still appears to be a bit of slowdown from sequential processing, which multiprocessing may resolve for many files.
Some refactoring was necessary to separate model loading and inference. Arguments were also given to the handler.py required by the Options class in inference.py, which were model specific.
The parameter args was refactored to options for some functions, due to some confusion that already occurred to myself. arg alone passed as the lone parameter made sense, but adding a model and config to the parameter list made args seem redundant. The argument was already being called opts in other cases.
A minor bugfix was also applied to inference.py, where the Options class was being passed, and not the instantiated object. This prevented inference.py from being run as main.
Let me know if something needs to be changed or removed.
Loading the model weights for each individual image appears to slow down inference. I took the model load from within inference, and required it as an argument to be passed at inference time. This allows the model weights to be loaded once, and then multiple images to be processed on the same model. There still appears to be a bit of slowdown from sequential processing, which multiprocessing may resolve for many files.
Some refactoring was necessary to separate model loading and inference. Arguments were also given to the handler.py required by the Options class in inference.py, which were model specific.
The parameter
args
was refactored tooptions
for some functions, due to some confusion that already occurred to myself.arg
alone passed as the lone parameter made sense, but adding amodel
andconfig
to the parameter list madeargs
seem redundant. The argument was already being calledopts
in other cases.A minor bugfix was also applied to inference.py, where the Options class was being passed, and not the instantiated object. This prevented inference.py from being run as main.
Let me know if something needs to be changed or removed.