This concept might be a bit confused: "loading" is work that is done, where data is moved into the database. But this sort of loader might request the performance of additional work to generate a more complex transformation of the dataset than can be accomplished in a regular loader.
For example, an ordinary loader might take an XML file with some flow data in it and dump it into the database. However, we might instead want to take the raw data, do a complex reanalysis of it, and load THAT data into the database.
For that we need a "work loader", which takes a bunch of data and creates a job request and submits it to Polyphemus. Someone else (a pipeline somewhere) will perform the work and return the job request, at which point another loader can clean up and load the results of the work done.
This concept might be a bit confused: "loading" is work that is done, where data is moved into the database. But this sort of loader might request the performance of additional work to generate a more complex transformation of the dataset than can be accomplished in a regular loader.
For example, an ordinary loader might take an XML file with some flow data in it and dump it into the database. However, we might instead want to take the raw data, do a complex reanalysis of it, and load THAT data into the database.
For that we need a "work loader", which takes a bunch of data and creates a job request and submits it to Polyphemus. Someone else (a pipeline somewhere) will perform the work and return the job request, at which point another loader can clean up and load the results of the work done.