binaryFolders feature in sasjs compile

When compiling SAS services / jobs / tests, there are currently two main sources of dependency:

SAS Macros -> these are simply inserted at the beginning of the compiled file (after initVars)
SAS Programs -> these are embedded in the compiled file using put statements and a fileref

The above works great for text files, however it does not support binary content. We DO support binary content, however, when compiling web content for SAS 9 (images, mp3 etc) under 'streaming apps'.

We need to provide a similar capability for SAS Jobs & Services (and Tests). There are times when it is necessary to embed zip files, or excel, or images. The approach will be identical to <h4> SAS Programs </h4> ( @li filename.sas fileref) except that the binary files will first be base64 encoded.

Example:

sasjsconfig

{
  "$schema": "https://cli.sasjs.io/sasjsconfig-schema.json",
  "binaryFolders": [
    "sasjs/binary"
  ],
  "jobConfig": {
    "jobFolders": [
      "sasjs/jobs/admin"
    ]
  },
  "targets": [
    {
      "name": "viya",
      "serverUrl": "https://sas.analytium.co.uk",
      "serverType": "SASVIYA",
      "appLoc": "/Public/app/dev",
    }
  ]
}

Job Code

/**
  @file 
  @brief Add excel to a job
  @details This job embeds a zip file

  <h4> SAS Macros </h4>
  @li mp_init.sas
  @li mp_unzip.sas

  <h4> Binary Files </h4>
  @li somefile.zip  myzip

**/

%mp_init()
%mp_unzip(ziploc=myzip,outdir=&sasjswork/somefolder)

The generated SAS code will first need to write out the Base64 encoded text, THEN convert it back to a binary file, eg as follows:

filename _sjstmp "%sysfunc(pathname(work))/base64_myzip" ;
data _null_;
  file _sjstmp;
  put 'UEsDBBQACAAIAHxqKlQAAAAAAAAAABoAAAAPACAAdGVzdF9iaW5hcn'@;
  put 'kudHh0VVQNAAdMCO+/vWFNCO+/vWFMCO+/vWF1eAsAAQTvv70BAAA'@;
  put 'EFAAAAO+/vVTvv71VKEktLu+/vU/vv73vv71LLO+/vVRI77+977+9K0nvv70'@;
  put 'rUVRUBABQSwcIV++/vVTvv70cAAAAGgAAAFBLAQIUAxQACAAIAHxqKlR'@;
  put 'X77+9VO+/vRwAAAAaAAAADwAgAAAAAAAAAAAA77+977+9AAAAAHRl'@;
  put 'c3RfYmluYXJ5LnR4dFVUDQAHTAjvv71hTQjvv71hTAjvv71hdXgLAAEE77+'@;
  put '9AQAABBQAAABQSwUGAAAAAAEAAQBdAAAAeQAAAAAA';
run;
filename myzip "%sysfunc(pathname(work))/sasjs_myzip";
data _null_;
  length filein 8 fileout 8;
  filein = fopen("_sjstmp",'I',4,'B');
  fileout = fopen("myzip",'O',3,'B');
  char= '20'x;
  do while(fread(filein)=0);
    length raw $4;
    do i=1 to 4;
      rc=fget(filein,char,1);
      substr(raw,i,1)=char;
    end;
    rc = fput(fileout,input(raw,$base64X4.));
    rc = fwrite(fileout);
  end;
  rc = fclose(filein);
  rc = fclose(fileout);
run;
filename _sjstmp clear;

In the above, the _sjstmp is a reserved fileref, used to dump the base64 content to a file in the SAS WORK directory (first data step) prefixed with sasjstmp_ + $(fileref). The actual file is then decoded into the SAS WORK directory (second data step), prefixed sasjs_ + $(fileref).

Overall Compilation Process

We now have the following types of file:

Jobs
Services (like Jobs but with service specific precode)
Tests (compiled as Services)
SAS Macros
SAS Includes
Binary Files

SAS Includes and Binary Files should only be compileable from Jobs, Services & Tests. There is no recursiveness in either of these, and it makes no sense to compile them into Macros either.

SAS Macros can contain other SAS Macros (recursive). They should not contain SAS Includes or Binary Files.

Jobs / Services / Tests may contain SAS Macros, SAS Includes, and/or Binary Files.

All 6 file types may be referenced from the root of the sasjsconfig, or the target. The sasjsconfig itself may be global (to the user), or local (to the project).

sasjs / cli