add infrastructure and ports allocation policy description to MultiProtocolDeployment class and improve deployment configuration

[activeeon-bot]

Original issue created by Lorenzo Cantelmi on 01, Feb 2013 at 12:35 PM - PROACTIVE-1238

---

General improvements to MultiProtocolDeployment are need. This class takes care about the proper deployment, by knowing the main details about MP virtual infrastructure. In particular:

• get such details be explicit, by describing the virtual topology;
• classify protocols in two main categories, according with their communication topology: client/server or peer-to-peer
• define ports allocation policy for each protocol, by distinguish from the local and remote tests and the protocols topology
• define hostname where GCMDeployment is possible, by distinguishing from local and remote test cases
• change methods in order to accomplish the above mentioned policy
• as it happens for all FunctionalTest, rely on automatic AgentID allocation also for remote Functional Test (where PAMR router is now deployed on a dedicated machine)

[activeeon-bot]

Original comment posted by Lorenzo Cantelmi on 11, Mar 2013 at 11:25 AM

---

package functionalTests.remoteobject.multiprotocol.gcmdeployment;

import java.io.IOException; import java.net.URI; import java.net.URL; import java.util.ArrayList; import java.util.Arrays; import java.util.Enumeration; import java.util.Iterator; import java.util.List;

import org.junit.After; import org.objectweb.proactive.ActiveObjectCreationException; import org.objectweb.proactive.api.PAActiveObject; import org.objectweb.proactive.api.PARemoteObject; import org.objectweb.proactive.core.ProActiveException; import org.objectweb.proactive.core.config.ProActiveConfiguration; import org.objectweb.proactive.core.node.Node; import org.objectweb.proactive.core.node.NodeException; import org.objectweb.proactive.core.remoteobject.RemoteObjectHelper; import org.objectweb.proactive.core.remoteobject.RemoteObjectProtocolFactoryRegistry; import org.objectweb.proactive.core.remoteobject.RemoteObjectSet.NotYetExposedException; import org.objectweb.proactive.core.remoteobject.exception.UnknownProtocolException;

import functionalTests.GCMFunctionalTest; import functionalTests.remoteobject.multiprotocol.ServerObject; import functionalTests.remoteobject.multiprotocol.gcmdeployment.local.MultiProtocolLocalDeploymentTestCase;

public class MultiProtocolDeployment extends GCMFunctionalTest {

/            *      *  Description of MultiProtocol (MP) virtual test infrastructure:
 **/  
//  ________                             ______________  
// | client |         _________         |              |
// |___VM___|eth0----| virtual |----eth0|-|Firewall| |

| :-- | :-- | // |SW_| | ! | // ! | node | // ! |__VM__| // eth0 // | scheduler | // | rm | // |pamr router| // |__VM___|

// TO DO: furnish a description of VMs configurations and how/where the infrastructure is accessible

/*  Classify protocols according with their actual topology (peer-to-peer vs client/server)
 According with the following classification, different rules (firewall, and so on) should be applied for tests  
If you add new protocols, classify them avoiding to insert other protocols that sharing the same listening server!
**/ 
// PEER-2-PEER TOPOLOGY is typical of those protocols that allow each software entity, i.e. node,
// to communicate with the others, i.e. peer nodes, in a decentralized way, i.e. without passing through
// any centralized software router. 
// N.B. By definition, in a p2p topology each entity is both client and server at the same time for a given service!
protected List<String> p2pTopologyProtocols = new ArrayList<String>(Arrays.asList( new String(link: ) {"rmi", "pnp", "http"}));

// CLIENT-SERVER TOPOLOGY is typical of those protocols that rely on a centralized software router (server),
// in order to allow the communication between software entities (clients), i.e. nodes.
protected List<String> csTopologyProtocols = new ArrayList<String>(Arrays.asList( new String(link: ) {"pamr", "amqp"}));

/            *      *  PORTS ALLOCATION POLICY:
 **/
// For p2p protocols, if we have p1 and p2 
// - local tests: 
//   *  client_p1-server:x1, node_p1-server:x1+1
//   *  x1 = 1024, x1 = port number where client p1 server is listening to 
//   
//            *  client_p2-server:x2, node_p2-server:x2+1
//   *  x2 = 1026, x2 = port number where client p2 server is listening to
//
// - remote tests:
//   *  client_p1-server:y1, node_p1-server:y1+1
//   *  y1 = 1024, y1 = port number where client p1 server is listening to 
//   
//   *  client_p2-server:y2, node_p2-server:y2+1
//   *  y2 = 1025, y2 = port number where client p2 server is listening to
//
// N.B. 
// a. RMI has different ports for the registry and for services registered on it, all belonging to USER ports range (link: 1024,65535).
//    That is because RMI architecture handles by self ports services allocation.
//    The client gets info about those ports while it performs a lookup in order to receive the stub of the remote object.
//    So the actual communication involves ports different from registry one.
//    For the other p2p protocols, registry port == service port!
// b. The policy above is due to limit the RMI ports random choice, we mentioned before, to a well-known range where we could focus on.
//    Otherwise, it is hard to apply, for instance, a specific firewall rule for RMI tests.
// c. No policy is needed by c-s protocols, since neither client neither node
//    run software router server, instead deployed on a third dedicated host

// Configuration for GCMDeployment in case of local tests
static final protected URL LOCAL_APP_DESC = MultiProtocolDeployment.class
        .getResource("/functionalTests/remoteobject/multiprotocol/gcmdeployment/local/LocalNodeDeploymentApp.xml");

// Configuration for GCMDeployment in case of remote tests
static final protected URL REMOTE_APP_DESC = MultiProtocolDeployment.class
        .getResource("/functionalTests/remoteobject/multiprotocol/gcmdeployment/remote/SSHBridge_RemoteNodeDeployment.xml");

// Hostnames definition where it is possible to have GCMDeployment
// local tests
static final protected String LOCAL_NODE_HOSTNAME = "localhost";
// remote tests
static final protected String REMOTE_NODE_HOSTNAME = "node";

// actual Deployment host
protected String deploymentHostname;

// Test Components
protected ServerObject soStub;
protected String name = "myServerActiveObject";
protected String defaultProtocolUriString;
protected String defaultProtocol = ProActiveConfiguration.getInstance().getProperty("proactive.communication.protocol");

// Add here protocols do not need to be tested, since not supported anymore
protected List<String> unsupportedProtocols = new ArrayList<String>(Arrays.asList( new String(link: ) {"pamrd", "ibis"}));

// Auxiliary Test data structures
public ArrayList<String> allPossibleLocalFactoriesProtocols = null;
public ArrayList<String> actualEnabledLocalJVMProtocols = null;
public ArrayList<String> enabledServerObjectProtocols = null;
public ArrayList<String> workableServerObjectProtocols = null;
public ArrayList<String> orderedProtocols = null;

public MultiProtocolDeployment(URL applicationDescriptor)
        throws ProActiveException, IOException {
    super(applicationDescriptor);
    super.setHostCapacity(1);
    super.setVmCapacity(1);
    super.startDeployment();

    inizializeDeploymentComponents(applicationDescriptor);
    inizializeTestComponents();

    // collected via active factories exploration
    allPossibleLocalFactoriesProtocols = getAllPossibleLocalFactoriesProtocols(); 

    // collected  via client-side ProActiveConfiguration settings
    // actualEnabledLocalJVMProtocols is a subset of allPossibleLocalFactoriesProtocols
    actualEnabledLocalJVMProtocols = getActualEnabledLocalJvmProtocols(); 

    // protocols on server (i.e. node) side set up, by own choice, as local JVM procotols
    enabledServerObjectProtocols = actualEnabledLocalJVMProtocols; 

    // collected via  pinging Server Object (should be a subset of )
    // actualEnabledLocalJVMProtocols is a subset of enabledServerObjectProtocols
    workableServerObjectProtocols = getWorkableServerObjectprotocols(); 

    // collected via client-side ProActiveConfiguration settings
    orderedProtocols = getOrderedProtocols();

}

public void executeAlgorithm() throws Exception {
    /                *      *  implemented by sub-classes
     **/
}

public ArrayList<String> getAllPossibleLocalFactoriesProtocols() {
    ArrayList<String> allPossibleLocalFactoriesProtocols = new ArrayList<String>();
    // discover all possible protocols
    Enumeration<String> eventualProtocols = RemoteObjectProtocolFactoryRegistry
            .keys();
    String currentProtocol = defaultProtocol;
    while (eventualProtocols.hasMoreElements()) {
        currentProtocol = eventualProtocols.nextElement();
        Class<?> rofClazz = RemoteObjectProtocolFactoryRegistry
                .get(currentProtocol);
        // Take into account just protocols with RemoteObjectFactory created
        if (rofClazz != null) {
            logger.info("All Possible Local Factories Protocols: "
                    + currentProtocol);
            allPossibleLocalFactoriesProtocols.add(currentProtocol);
        }
    }
    return allPossibleLocalFactoriesProtocols;
}

public ArrayList<String> getActualEnabledLocalJvmProtocols() {

    if (allPossibleLocalFactoriesProtocols == null) {
        allPossibleLocalFactoriesProtocols = getAllPossibleLocalFactoriesProtocols();
    }

    ArrayList<String> actualEnabledLocalJVMProtocols = new ArrayList<String>();

    String(link: ) additionalProtocols = ProActiveConfiguration.getInstance()
            .getProperty("proactive.communication.additional_protocols")
            .split(",");
    actualEnabledLocalJVMProtocols.add(defaultProtocol);
    logger.info("LocalJVMActualProtocol: " + defaultProtocol);
    for (int i = 0; i < additionalProtocols.length
            && !actualEnabledLocalJVMProtocols
                    .contains(additionalProtocols(link: i))
            && allPossibleLocalFactoriesProtocols
                    .contains(additionalProtocols(link: i)); i++) {
        actualEnabledLocalJVMProtocols.add(additionalProtocols(link: i));
        logger.info("LocalJVMActualProtocol: " + additionalProtocols(link: i));
    }

    return actualEnabledLocalJVMProtocols;
}

public ArrayList<String> getEnabledserverobjectprotocols() {
    return enabledServerObjectProtocols;
}

// dynamic discovery of the protocols working on remote server object
public ArrayList<String> getWorkableServerObjectprotocols() {
    if (allPossibleLocalFactoriesProtocols == null) {
        allPossibleLocalFactoriesProtocols = getAllPossibleLocalFactoriesProtocols();
    }

    if (soStub == null) {
        inizializeTestComponents();
    }

    ArrayList<String> workableServerObjectProtocols = new ArrayList<String>();
    Iterator<String> iterator = allPossibleLocalFactoriesProtocols
            .iterator();
    String currentProtocol = defaultProtocol;
    boolean isPingable = false;
    while (iterator.hasNext()) {
        currentProtocol = iterator.next();
        // test if remote object is pingable by current protocol forcing
        try {
            // Ping the active object by forcing the communication protocol
            isPingable = PAActiveObject.pingActiveObject(soStub, currentProtocol);
        } catch (NotYetExposedException e) {
            logger.info("Pinging with " + currentProtocol + ": " + e.toString());
            isPingable = false;
        } catch (UnknownProtocolException e) {
            logger.info("Pinging with " + currentProtocol + ": " + e.toString());
            isPingable = false;
        }
        // Take into account just protocols with RemoteObjectFactory created
        // and Remote Object pingable
        if (isPingable) {
            logger.info("Workable protocols on server object: "
                    + currentProtocol);
            workableServerObjectProtocols.add(currentProtocol);
        } else {
            logger.info("NOT Workable protocols on server object: "
                        + currentProtocol);
        }
    }

    return workableServerObjectProtocols;
}

private void inizializeDeploymentComponents(URL appDescUrl){
    if (applicationDescriptor.equals(LOCAL_APP_DESC)) {
        deploymentHostname = LOCAL_NODE_HOSTNAME;
    } else {
        deploymentHostname = REMOTE_NODE_HOSTNAME;
    }
}

private void inizializeTestComponents() {

    try {
        defaultProtocolUriString = getProperServerObjectURL(defaultProtocol)
                .toString();

        Node remoteNode = super.getANode();

        soStub = (ServerObject) PAActiveObject.newActive(ServerObject.class,
                new Object(link: ) { name }, remoteNode);
            soStub.addProtocol(defaultProtocol);

        //log after having added defaultProtocol
        logAllRemoteObject(defaultProtocol);

        // look up for the (remote) active object and get its stub
        soStub = PAActiveObject.lookupActive(ServerObject.class,
                defaultProtocolUriString);
    } catch (ActiveObjectCreationException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (NodeException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (ProActiveException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

public ArrayList<String> getOrderedProtocols() {

    if (allPossibleLocalFactoriesProtocols == null) {
        allPossibleLocalFactoriesProtocols = getAllPossibleLocalFactoriesProtocols();
    }

    ArrayList<String> orderedProtocols = new ArrayList<String>();

    if (ProActiveConfiguration.getInstance().getProperty(
            "proactive.communication.protocols.order") != null) {

        String(link: ) orderString = ProActiveConfiguration.getInstance()
                .getProperty("proactive.communication.protocols.order")
                .split(",");

        for (int i = 0; i < orderString.length
                && allPossibleLocalFactoriesProtocols
                        .contains(orderString(link: i)); i++) {
            orderedProtocols.add(orderString(link: i));
            logger.info("Protocols Order: " + i + " " + orderString(link: i));
        }
    }

    return orderedProtocols;
}

// method to resolve the bootstrap from client side
// Implementation takes into account PORTS ALLOCATION POLICY (see above)
public URI getProperServerObjectURL(String protocol) {

    URI serverObjectUrl = null;
    try {
        serverObjectUrl = RemoteObjectHelper.generateUrl(protocol, name);
    } catch (ProActiveException e1) {
        // TODO Auto-generated catch block
        e1.printStackTrace();
    }

// logger.info("Replacing ... Server Object url: " + serverObjectUrl.toString()); // set remote node ports configuration, as client ones plus 1.

    // do not care about protocol not supported anymore
    if (! unsupportedProtocols.contains(protocol) ){
        String hostname = serverObjectUrl.getHost();
        String port = String.valueOf(serverObjectUrl.getPort());
        String serverObjectUrlString = serverObjectUrl.toString();
        if (protocol.equals("pamr")) {
            //Remote Node will get the Client consecutive PAMR agent ID
            serverObjectUrlString = serverObjectUrlString.replace(hostname,
                        String.valueOf(Integer.parseInt(hostname) + 1) );
        } else {
            serverObjectUrlString = serverObjectUrlString.replace(hostname,
                    deploymentHostname);
            serverObjectUrlString = serverObjectUrlString.replaceAll(
                        port, getServerRegistryPort(protocol));
        }

// logger.info("Current Server Object url: " + serverObjectUrlString); serverObjectUrl = URI.create(serverObjectUrlString); }

    return serverObjectUrl;
}

public void logAllRemoteObject(String protocol){        
    URI protocolRegisterUrl = getProperServerObjectURL(protocol);   
    logger.info("Registered Remote Objects @ " + protocolRegisterUrl.getScheme() + "://" + protocolRegisterUrl.getHost() + ":" + protocolRegisterUrl.getPort() );
    try {
        for(int i = 0; i < PARemoteObject.list(protocolRegisterUrl).length; i++){
            logger.info("\t " + PARemoteObject.list(protocolRegisterUrl)(link: i).toString());
        }
    } catch (Exception e) {
        logger.warn(e.getMessage());
    }
}

public void logAllProtocolsRemoteObjects(){ 
    ArrayList<String> allProtocols = getAllPossibleLocalFactoriesProtocols();   
    logger.info("Logging all protocols remote objects: ");
    for(Iterator<String> i = allProtocols.iterator(); i.hasNext();){
        String protocol = i.next();
        if (! unsupportedProtocols.contains(protocol) ) { 
            logAllRemoteObject(protocol);
        }
    }
}

// method to know from client side which port to lock on node, via ssh + iptables
// Implementation takes into account PORTS ALLOCATION POLICY (see above)
protected String get2BeFirewalledServerPort(String protocol) {
    URI serverObjectUrl = null;
    String destinationPort = "";
    try {
        serverObjectUrl = RemoteObjectHelper.generateUrl(protocol, name);
    } catch (ProActiveException e1) {
        // TODO Auto-generated catch block
        e1.printStackTrace();
    }

    String pnpPort = ""; 
    String pnpsPort = ""; 
    String httpPort = "";

    // See PORTS ALLOCATION POLICY (see above)
    if ( this instanceof MultiProtocolLocalDeploymentTestCase){

// logger.info("Replacing ... Server Object url: " + serverObjectUrl.toString()); // Since it is not possible to share a port between two different services on the same localhost, // set remote node ports configuration as client ones plus 1 (by own choice). pnpPort = String.valueOf(Integer.parseInt(ProActiveConfiguration .getInstance().getProperty("proactive.pnp.port")) + 1); pnpsPort = String.valueOf(Integer.parseInt(ProActiveConfiguration .getInstance().getProperty("proactive.pnps.port")) + 1); httpPort = String.valueOf(Integer .parseInt(ProActiveConfiguration.getInstance().getProperty( "proactive.http.port")) + 1); } else { // In case of remote tests, client and node ports cloud be the same pnpPort = String.valueOf(Integer.parseInt(ProActiveConfiguration .getInstance().getProperty("proactive.pnp.port"))); pnpsPort = String.valueOf(Integer.parseInt(ProActiveConfiguration .getInstance().getProperty("proactive.pnps.port")) ); httpPort = String.valueOf(Integer .parseInt(ProActiveConfiguration.getInstance().getProperty( "proactive.http.port"))); } // Rmi Server port is not known, since is decided at runtime by transport layer protocols, // not possible to configure it. So, a ports range is required in iptables format, that is: // x:y, where x < y, x,y in (link: 1024, 65535) String rmiPortsRange = getFirstFreeUserPort() + ":65535"; if (! unsupportedProtocols.contains(protocol)) { if (protocol.equals("pamr")) { // pamr required just an Agent ID destinationPort = ""; } else { if (protocol.equals("http")) { destinationPort = httpPort; } else if (protocol.startsWith("rmi")) { destinationPort = rmiPortsRange; } else if (protocol.equals("pnp")) { destinationPort = pnpPort; } else if (protocol.equals("pnps")) { destinationPort = pnpsPort; } } } return destinationPort; }

protected String getServerRegistryPort(String protocol) {
    // RMI architecture distinguishes the port access to registry from the actual service,
    // i.e. the remote active object, decide instead dynamically by its own stack.
    // Such a thing does not happen to the other p2p protocol, where registry and active objects
    // listen to the same port since running in a jetty server.
    // So ad-hoc differentiation should be considered, as follows.
    if( protocol.startsWith("rmi")){
        if( this instanceof MultiProtocolLocalDeploymentTestCase){
            return String.valueOf(Integer.parseInt(ProActiveConfiguration
                    .getInstance().getProperty("proactive.rmi.port")) + 1);
        } else {
            return String.valueOf(Integer.parseInt(ProActiveConfiguration
                    .getInstance().getProperty("proactive.rmi.port")));
        }
    } else {
        return get2BeFirewalledServerPort(protocol);
    }
}

@After
public  void killNodeDeployment() {
    try {
        killDeployment();
    } catch (Throwable e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

private String getFirstFreeUserPort(){
    // See above PORTS ALLOCATION POLICY 
    // Ports allocation for client and node hosts is done in configs files.
    // In particular, each server is listening at a different consecutive port after 1023,
    // that is the last system port available for system ports range (link: 0,1023)
    Integer lastSystemPort = 1023;

    // In order to calculate the first free USER port, we have declared a priori a data structure,
    // where are listed all P2P Protocols having its own dedicated server, called "actual p2p protocols"
    // By its size we will be able to establish the first free port. 
    Integer numberOfActualP2PProtocols = p2pTopologyProtocols.size();

    if (this instanceof MultiProtocolLocalDeploymentTestCase){
        numberOfActualP2PProtocols = numberOfActualP2PProtocols    *  2;
    }
    // ports till lastSystemPort + numberOfActualP2PProtocols number 
    // are a priori allocated, so return just the next one, that is free.
    return String.valueOf(lastSystemPort + numberOfActualP2PProtocols + 1);
}

}