This work is in it's alpha stage and under heavy development so the interface may change frequently.
This repository contains the interface that the Golem compatible application should implement as well as constants used by the protocol. The interface and the constants are defined under the golem_task_api/proto
directory in the Protocol Buffers files.
This repository also contains programming language specific packages of the gRPC
protocol which may be used for concrete implementation. This is for the ease of development of the application but it's not required to use them in the application.
If you don't see a programming language you're interested in, feel free to create an issue or even a pull request and we will add it.
The API is divided into two independent parts - requestor and provider.
For requestor the app should implement a long running RPC service which implements the RequestorApp
interface from the proto files. The app should assume it will have access to a single directory (let's call it work_dir
). Each task will have its own separate working directory under the main work_dir
. You can assume that for a given task_id
the first call will always be CreateTask
and the following directories will exist under work_dir
and they will be empty:
{task_id}
{task_id}/{constants.TASK_INPUTS_DIR}
{task_id}/{constants.SUBTASK_INPUTS_DIR}
{task_id}/{constants.TASK_OUTPUTS_DIR}
{task_id}/{constants.SUBTASK_OUTPUTS_DIR}
CreateTask
task_id
, max_subtasks_count
, and task_params_json
.{work_dir}/{task_id}
as the working directory for the given task.task_params_json
is a JSON string containing app-specific task parameters. Format of these parameters is entirely up to the application developer.task_id
.task_id
is unique per node.{task_id}/{constants.TASK_INPUTS_DIR}
contains all the resources provided by task creator.env_id
and prerequisites_json
specifying the environment and prerequisites required for providers to compute the task. See environments section for details.NextSubtask
task_id
, subtask_id
and opaque_node_id
.opaque_node_id
is an identifier of the node which is going to compute the requested subtask. 'Opaque' means that the identifier doesn't allow to obtain any further information about the node (e.g. public key, IP address).subtask_id
is unique per node.CreateTask
was called earlier with the same task_id
.subtask_params_json
which is the JSON string containing subtask specific parameters.resources
which is a list of names of files required for computing the subtask. Files with these names are required to be present in {task_id}/{constants.SUBTASK_INPUTS_DIR}
directory.HasPendingSubtasks
task_id
.true
, the next NextSubtask
call should return successfully (although it can still return an empty message).Verify
task_id
and subtask_id
which specify which subtask results should be verified.task_id
and subtask_id
values.result
which is one of the defined verification result statuses:SUCCESS
- the subtask was computed correctly,FAILURE
- the subtask was computed incorrectly,INCONCLUSIVE
- cannot determine whether the subtask was computed correctly,AWAITING_DATA
- cannot perform verificaton until results of other subtasks are available.reason
which is a string providing more detail about the result.DiscardSubtasks
task_id
and subtask_ids
.Benchmark
AbortTask
task_id
.AbortSubtask
task_id
and subtask_id
.Shutdown
When the last subtask is successfully verified on the requestor's side, the work_dir/task_id/constants.TASK_OUTPUTS_DIR
directory should contain all result files and nothing else.
Both provider and requestor apps run on top of Golem's execution environments. Environment for requestor is specified in the application definition and cannot vary. Provider environment is specified by the return value of CreateTask
call. A single application could use different environments for different types of tasks, it could also use different environment for requestor and provider. Environments have their unique IDs and prerequisites formats. Prerequisites are additional requirements for the environment to run the app (e.g. Docker environment prerequisites specify image). Environment IDs and prerequisites formats are listed in envs.proto file.
Currently the following environments are supported:
docker_cpu
- standard Docker environment{
"image": "...",
"tag": "..."
}
docker_gpu
- GPU-enabled Docker environment, Linux onlydocker_cpu
Provider app should implement a short-lived RPC service which implements the ProviderApp
interface from the proto files. Short-lived means that there will be only one request issued per service instance, i.e. the service should shutdown automatically after handling the first and only request.
Compute
task_work_dir
to operate on.task_work_dir
.task_id
, subtask_id
, subtask_params_json
as arguments.{task_work_dir}/{constants.SUBTASK_INPUTS_DIR}
directory exists.{task_work_dir}/{constants.SUBTASK_INPUTS_DIR}
are the resources specified in the corresponding NextSubtask
call.task_work_dir
) of the result file which will be sent back to the requestor with unchanged name.Benchmark
Shutdown