BioLockJ-Dev-Team / sheepdog_testing_suite

Test suite for BioLockJ development team.
3 stars 8 forks source link

Need to check for R packages during check dependencies #257

Closed IvoryC closed 3 years ago

IvoryC commented 4 years ago

By far the most common reason for failure in R_module modules is just not having one or more of the required packages. We really need a way to specify these...

Rmarkdown module might have: r.packages=ggpubr, coin, vegan

GenMod might reference r.packages

R_module would have some static list of packages, and individual modules can expand the list, and the R_module class will have the code to check packages.

and in checkDependencies, biolockj will launch an R session just to see if those packages are present, (maybe attempt to install them if they are not), and throw and exception if they are not available.

IvoryC commented 3 years ago

This was part of minor release v1.3.10. That introduced the RequiredRPackageException.

This should be revisited after the exe check is put in place. And there is some refinement that could happen in the future (see TODOs), but the functionality if fundamentally in place.

Essential methods in R_module:

    protected Map<String, String> requiredRPackages(){
        Map<String, String> packages = new HashMap<>();
        //These packages are required by the current BioLockJ_Lib.R functions
        packages.put("properties","https://CRAN.R-project.org");
        packages.put("stringr","https://CRAN.R-project.org");
        packages.put("ggpubr","https://CRAN.R-project.org");
        return packages;
    }

    private void checkRPackages() throws SpecialPropertiesException, IOException, InterruptedException,
        RequiredRPackageException, ConfigNotFoundException {
        Set<String> failedPackages = new HashSet<String>();
        for( String pack: requiredRPackages().keySet() ) {
            if( !checkPackage( pack ) ) failedPackages.add( pack );
        }
        if( failedPackages.size() > 0 ) { throw new RequiredRPackageException( failedPackages, requiredRPackages() ); }
    }

    private boolean checkPackage(String packageName) throws SpecialPropertiesException, IOException, InterruptedException, ConfigNotFoundException {
        boolean isGood;
        String cmd = Config.getExe( this, Constants.EXE_RSCRIPT ) + " -e 'library(" + packageName + ");packageVersion(\"" + packageName + "\")'";
        //TODO: revisit this after the check exe feature is finished.
//      if (Config.getString( this, Constants.PIPELINE_ENV ).equals(Constants.PIPELINE_ENV_CLUSTER)) {
//          for (String addLib : BashScriptBuilder.loadModules(this)) {
//              cmd = addLib + "; " + cmd;
//          }
//      }
        if (DockerUtil.inDockerEnv()) {
            //TODO: revisit this after the check exe feature is finished.
            cmd = Config.getExe( this, Constants.EXE_DOCKER ) + " run --rm " + DockerUtil.getDockerImage( this ) + " " + cmd;
        }
        Log.info(R_Module.class, "Checking for R package [" + packageName + "] using command: " + cmd);
        Process p = Runtime.getRuntime().exec( cmd );
        final BufferedReader brOut = new BufferedReader( new InputStreamReader( p.getInputStream() ) );
        final BufferedReader brErr = new BufferedReader( new InputStreamReader( p.getErrorStream() ) );
        String out = null;
        String err = null;
        do{
            if (out != null) Log.info(getClass(), out);
            if (err != null) Log.info(getClass(), err);
            out = brOut.readLine();
            err = brErr.readLine();
        }while( out != null ||  err != null ) ;
        p.waitFor();
        p.destroy();
        isGood = p.exitValue() == 0;
        return isGood;
    }