Recources across nodes

[KarlKeiser]

Resources are being cleaned up when the corresponding Erlang term is garbage collected. However, this does not work across nodes because of reference counting: the refcount reaches 0 for the slave node and it deallocates the resource. A mechanism is required that corrects the issue of resources being freed too early. It should also be able to free them as soon as they are not required anymore, not just when the whole context dies (for instance millions of statements being made and destroyed back to back) and perhaps it could also handle resources left behind in the wake of unexpected termination.

[c-bik]

Parse transform refactoring

DPI has two types of APIs: resource manipulating (create, change, destroy) and resource non-manipulating.

Resource manipulating APIs

APIs which accepts a double pointer to a opaque C structure (e.g. dpiContext **context) also documented as [OUT] parameter in ODPI API specificcation. For example:

context [OUT] – a pointer to a context handle which will be populated upon successful completion of this function.

Alternatively, resource destructures are also manipulating APIs (for example dpiContext_destroy). _All These API calls need to be passed through slave gen_server to increase the reference count for persistance._

Resource NON-manipulating APIs

The APIs wich either do not accept any resurce argument, or accepts readonly resource (in general single pointers, e.g. dpiContext *context, exceptions are the destructors e.g. dpiContext_destroy). These APIs can be directly mapped through RPC calls.

Resource manipulating APIs

// Create dpiContext
int dpiContext_create(
    unsigned int majorVersion,
    unsigned int minorVersion,
    dpiContext **context,
    dpiErrorInfo *errorInfo
);
// Delete dpiContext
int dpiContext_destroy(dpiContext *context);

Are to be accessed through gen_server in slave node as (generated through dpi_transform):

-module(dpi).

-export([..., context_create/2, context_destroy/1, ...]).
-export([..., context_create_nif/2, context_destroy_nif/1, ...]).

context_create(Arg1, Arg2) when is_integer(Arg1), is_integer(Arg2) ->
    gen_server:call(
        {dpi, get(dpi_slave)},
        {context_create_nif, [Arg1, Arg2], save}
   ).

context_destroy(Arg1) when is_reference(Arg1) ->
    gen_server:call(
        {dpi, get(dpi_slave)},
        {context_destroy, [Arg1], delete}
   ).

handle_call({Fun, Args, Op}, _From, State) ->
    Result = fa(Fun, Args),
    NewState = process_res(Op, State, Args, Result),
    {reply, Result, NewState}.

process_res(save, State, _Args, Result) ->
    % TODO recursively process Result and add all resources to State (list) return new state
process_res(delete, State, Args, _Result) ->
    % TODO: recursively process Args and delete all resources to State (list) return new state 

fa(Fun, Args) ->
    % TODO generate body through parse transform

Resource NON-manipulating APIs

All other NIF APIs are to be mapped through direct as RPC: For example, the following APIs

void dpiContext_getClientVersion(const dpiContext *context, dpiVersionInfo *versionInfo)
void dpiContext_getError(const dpiContext *context, dpiErrorInfo *errorInfo)

...are to be mapped as follows:

-module(dpi).

-export([..., context_getClientVersion/1, context_getError/1, ...]).
-export([..., context_getClientVersion_nif/1, context_getError_nif/1, ...]).

context_getClientVersion(Arg1) when is_reference(Arg1) ->
    rpc_call(get(dpi_slave), dpi, fa, [context_getClientVersion, [Arg1]]).

context_getError(Arg1) when is_reference(Arg1) ->
    rpc_call(get(dpi_slave), dpi, fa, [context_getClientVersion, [Arg1]]).

fa(Fun, Args) ->
    % TODO generate body through parse transform

-nif attribute definition change

The ?*F(...) macros: https://github.com/K2InformaticsGmbH/oranif/blob/de8b840ff47dd8c003ed6675ea72e4ecf06a7c3d/src/dpi.hrl#L16-L19 ...needs to be extended to support optional save/delete parameters as follows:

-nifs({dpiContext, [
    ?CF(ontext_create, [integer, integer], save),
    ?CF(ontext_destroy, [reference], delete),
    ?CF(ontext_getClientVersion, [reference]),
    ?CF(ontext_getError, [reference]),
...
]}).