Building a makefile dependency generator for RPGLE

worksofliam commented 4 years ago

This is the first of two entries. This will be about how to get the information. The second about actually building the dep list. I will keep expanding this post as I find more information.

Starting list

DSPPGMREF PGM(GITCM/*ALL) OUTPUT(*OUTFILE) OBJTYPE(*PGM *SRVPGM) OUTFILE(QTEMP/INFO)
SELECT WHLIB, WHPNAM, WHFNAM, WHLNAM, WHOBJT, WHSPKG FROM qtemp/info WHERE WHFNAM not like 'Q%' (maybe a group by should be used here)

Should give you a base list to work from:

A list of programs and service programs
A list of files, programs and services that those programs depend on (no way to determine what type the file is yet, like table/PF, logical/view/index, display file, etc) - might require another call to DSPOBJD with an outfile?

You should end up with a list of:

programs
service programs
files

Program / Service program info

We need to find:

Each module which makes up a program and service program
Each service program that is used in our programs and service programs

DSPPGM and DSPSRVPGM don't have simple outfiles and nobody in 2020 should be parsing spool files. There are two APIs to get the info we want:

List ILE Program Information (QBNLPGMI) API
List Service Program Information (QBNLSPGM) API

They both write information to a user space, which could then be written to an outfile. A bit of extra work but worth it in the long run to get this working.

So now, in total, we should have:

A list of programs, their modules, what service programs they reference and files they reference
A list of service programs, their modules, what service programs they reference and files they reference
A list of files, and a list of what programs/service programs use them

Still to determine

What data areas are used by what programs
What attribute each file has to determine what type of object it really is

Update: using SQL

The OS now ships information about program/service program references. Here's a little SQL I wrote to use this to generate the start of a makefile.

--CL: DSPPGMREF PGM(LIBHTTP/*ALL) OUTPUT(*OUTFILE) OBJTYPE(*ALL) OUTFILE(QTEMP/FILEREFS);
--CL: DSPDBR FILE(LIBHTTP/*ALL) OUTPUT(*OUTFILE) OUTFILE(QTEMP/DBREFS); 

select * from qtemp.dbrefs where WHREFI != '';

with makelist as (
  SELECT 
    PROGRAM_NAME as parent_name, 
    SUBSTR(OBJECT_TYPE, 2) as parent_type, 
    BOUND_MODULE as child_name, 
    MODULE_ATTRIBUTE as child_type
  FROM QSYS2.BOUND_MODULE_INFO 
  WHERE PROGRAM_LIBRARY = 'LIBHTTP' 
UNION ALL
  select
    WHRFI as parent_name,
    WHRTYP as parent_type,
    WHREFI as child_name,
    'lf' as child_type
  from qtemp.dbrefs 
  where WHREFI != ''
UNION ALL
  SELECT 
    PROGRAM_NAME as parent_name, 
    SUBSTR(OBJECT_TYPE, 2) as parent_type, 
    BOUND_SERVICE_PROGRAM as child_name, 
    'SRVPGM' as child_type
  FROM QSYS2.BOUND_SRVPGM_INFO
  WHERE 
    PROGRAM_LIBRARY = 'LIBHTTP' and
    bound_service_program_library != 'QSYS'
UNION ALL
  SELECT
    WHPNAM as parent_name,
    case
      when WHSPKG = 'P' then 'PGM'
      when WHSPKG = 'V' then 'SRVPGM'
      when WHSPKG = 'M' then 'MODULE'
    end as parent_type,
    WHSNAM as child_name,
    'FILE' as child_type
  FROM QTEMP.FILEREFS
  where 
    whotyp = '*FILE' and
    (WHSNAM not like 'Q%' and WHSNAM not like '*%') and 
    WHSNAM != ''
)
select * from makelist;

jkdavew commented 4 years ago

Hi @worksofliam - you can also try using these SQL functions as they might be easier than working with the APIs and then dumping to temp files/outfile.

qsys2.bound_srvpgm_info
qsys2.bound_module_info
qsys2.program_info
qsys2.program_export_import_info

I'm not 100% sure it gets you all the same information as the APIs, but worth looking into.

worksofliam commented 4 years ago

Hey @jkdavew

I updated the post with an SQL script I started to write. Thanks for the suggestion!

worksofliam / blog