irods / contrib

A pooled collection of community-contributed code that works alongside iRODS
BSD 3-Clause "New" or "Revised" License
13 stars 19 forks source link

new microservice to rebalance a single data object #20

Open trel opened 8 years ago

trel commented 8 years ago

From @rwmoore:

Current resources:

demoResc
LTLResc:passthru
└── LTLRepl:replication
    ├── LTLRenci:unixfilesystem
    └── LTLSils:unixfilesystem

I put a file on demoResc, and tried to use msiDataObjRepl() to replicate the file. Using

  • "destRescName=LTLRenci" gives an error "DIRECT_CHILD_ACCESS"
  • "destRescName=LTLRepl" gives an error "DIRECT_CHILD_ACCESS"
  • "destRescName=LTLResc" works. Two files are created, one on LTLRenci and one on LTLSils.

The problem occurs when a file is created on LTLSils through execution of a script for encryption. I want to replicate the encrypted file to LTLRenci, but cannot specify the destination resource.

What is the correct way to replicate a file that already exists on one of the resources specified by a replication node? Previously I invoked a reBalance operation on the replication node. But I only needed to reBalance a single file.

misterbass commented 7 years ago

Hi @trel does this new microservice is available in 4.1.9 or do you have a workaround to handle the situation, I am facing same issue !

trel commented 7 years ago

We have not added this microservice yet.

The workaround is to have the iadmin modresc RESCNAME rebalance command run periodically on the server and it will keep the policy defined by the replication resource 'up to date' by making sure the specified children resources keep their replicas are current.

trel commented 4 years ago

candidate msiRunRebalance from 2016 from @stillwell ...

#define RODS_SERVER 1

#include "reGlobalsExtern.hpp"
#include "reFuncDefs.hpp"
#include "icatHighLevelRoutines.hpp"
#include "rcMisc.h"
#include "generalAdmin.h"
#include "irods_server_properties.hpp"
#include "irods_ms_plugin.hpp"

// =-=-=-=-=-=-=-
// STL Includes
#include <iostream>

extern "C" {
/**
 * \fn msiRunRebalance(msParam_t *resource, ruleExecInfo_t *rei)
 *
 * \brief runs modresc rebalance for given resource
 *
 * \module dfc
 *
 * \since 4.1.x
 *
 *
 *
 * \usage See clients/icommands/test/rules/
 *
 * \param[in] resource - a STR_MS_T with the name of the resource where
 *      the reblalnce will apply.
 * \param[in,out] rei - The RuleExecInfo structure that is automatically
 *    handled by the rule engine. The user does not include rei as a
 *    parameter in the rule invocation.
 *
 * \iCatAttrDependence None
 * \iCatAttrModified Updates synchronizes child resource if provided resource is type: replication
 * \sideeffect None
 *
 * \return integer
 * \retval 0 on success
 * \pre None
 * \post None
 * \sa None
 *
 **/
// =-=-=-=-=-=-=-

int
msiRunRebalance( msParam_t* resource, ruleExecInfo_t* rei ) {
    generalAdminInp_t generalAdminInp;
    int status;

    /* For testing mode when used with irule --test */
    RE_TEST_MACRO( "    Calling msiRunRebalance" )

    /* Sanity checks */
    if ( rei == NULL || rei->rsComm == NULL ) {
        rodsLog( LOG_ERROR, "msiRunRebalance: input rei or rsComm is NULL." );
        return SYS_INTERNAL_NULL_INPUT_ERR;
    }

    /* Must be called from an admin account */
    if ( rei->uoic->authInfo.authFlag < LOCAL_PRIV_USER_AUTH ) {
        status = CAT_INSUFFICIENT_PRIVILEGE_LEVEL;
        rodsLog( LOG_ERROR, "msiRunRebalance: User %s is not local admin. Status = %d",
                 rei->uoic->userName, status );
        return status;
    }

    /* Prepare generalAdminInp. It needs to be set up as follows:
     *    generalAdminInp.arg0: "modresc"
     *    generalAdminInp.arg1: resource name
     *    generalAdminInp.arg2: "rebalance"
     *    generalAdminInp.arg3: ""
     *    generalAdminInp.arg4: ""
     *    generalAdminInp.arg5: ""
     *    generalAdminInp.arg6: ""
     *    generalAdminInp.arg7: ""
     *    generalAdminInp.arg8: ""
     *    generalAdminInp.arg9: ""
     */
    memset( &generalAdminInp, 0, sizeof( generalAdminInp_t ) );
    generalAdminInp.arg0 = "modify";
    generalAdminInp.arg1 = "resource";

    /* Parse resource name */
    if ( ( generalAdminInp.arg2 = parseMspForStr( resource ) ) == NULL ) {
        rodsLog( LOG_ERROR, "msiRunRebalance: Null resource provided." );
        return SYS_INTERNAL_NULL_INPUT_ERR;
    }

    //memset( &generalAdminInp, 0, sizeof( generalAdminInp_t ) );
    generalAdminInp.arg3 = "rebalance";

    /* Fill up the rest of generalAdminInp */
    generalAdminInp.arg4 = "";
    generalAdminInp.arg5 = "";
    generalAdminInp.arg6 = "";
    generalAdminInp.arg7 = "";
    generalAdminInp.arg8 = "";
    generalAdminInp.arg9 = "";

    /* Call rsGeneralAdmin */
    status = rsGeneralAdmin( rei->rsComm, &generalAdminInp );

    /* Done */
    return status;
}

irods::ms_table_entry*  plugin_factory() {
    irods::ms_table_entry* msvc = new irods::ms_table_entry( 1 );
    msvc->add_operation( "msiRunRebalance", "msiRunRebalance" );
    return msvc;
}

};