More routes/methods to implement pre-post processing

[falibabaei]

Hi, As you know, in most Deep Learning developments, preprocessing plays a crucial role in preparing the data for training. For example, in one of my developments where I want to add an API to an object detection model, image conversion in tfrecord is required. I would like to include this preprocessing step in my API. Users will have the option to perform this step themselves during model training. Those unfamiliar with this process can use this built-in preprocessing step to easily convert their data into tfrecord. I would like to see this method included in the API.

Another valuable feature would be an evaluation method. After users train a model on the new dataset, they may want to compare its performance to the older version of the model on a new test dataset using certain metrics they define. By integrating this method into the API, users can efficiently evaluate and compare the performance of the model on the test dataset.

Thank you for taking the time and effort to include these additions

[vykozlov]

In a very general case, data-processing might be an asset in itself, like we may have modules which do only 'training' (e.g. benchmarks), we may have modules with only predict (well-trained model, only inference is published), why not to complement with modules which focus on data-processing. Not sure, but this could be, e.g.:

GET  /v3/dataops/usermodel/          # returns list of dataops methods available, e.g. [acquire, parse, augment, clean, convert]
POST /v3/dataops/usermodel/acquire      # configure and execute data acquisition
POST /v3/dataops/usermodel/parse        # configure and execute parsing of data
POST /v3/dataops/usermodel/augment     # data augmentation
POST /v3/dataops/usermodel/clean         # data cleaning
POST /v3/dataops/usermodel/convert       # data format conversion
POST /v3/dataops/usermodel/process       # execute combination of dataops, input is RequestBody JSON with corresponding dataop and parameters (?)

OR it is still under "models":

GET  /v3/models/usermodel/dataops/       # returns list of dataops methods available
POST /v3/models/usermodel/dataops/acquire
POST /v3/models/usermodel/dataops/parse
POST /v3/models/usermodel/dataops/augment
POST /v3/models/usermodel/dataops/clean
POST /v3/models/usermodel/dataops/convert
POST /v3/models/usermodel/dataops/process 

For evaluation it should be easy:

POST /v3/models/usermodel/evaluate/       # to evaluate model on the testing dataset (e.g. model changed or test data changed)