Provide a simple tool to generate Linux-style man pages from z/OS man pages

[MikeFultonDev]

Initial thoughts:

There is a utility called txt2man that does most of what we need: https://github.com/mvertes/txt2man/blob/master/txt2man I copied it as is to z/OS and just changed the first line to #/bin/env bash

If you do the following for a command called cmd (e.g. tsocmd)

(export MANPATH=/usr/man/%L; /bin/man _cmd_ >_cmd_.txt)

Convert mixed case titles like Format, Description, Options, Localization, Usage notes, Exit values, Portability, Relative information to corresponding upper-case versions with no leading blanks and prepend a first line called NAME

txt2man -p -t _cmd_ -v "z/OS User Manual" _cmd_.txt >~/zopen/boot/zosman/share/man/man1/_cmd_.1

then the output will look ok. Here's an the output for tsocmd

I don't know of a good way to get the list of man pages in /usr/man/%L because they are in a proprietary file (looks like many if not all are in bpxa5mst.stfpt) so an alternate technique is to just generate man pages for each command found in /bin/, e.g. /bin/cc or /bin/xlC

@lbdyck @IgorTodorovskiIBM fyi...

[lbdyck]

you could do a man -k . on a 'vanilla' system and thern do a man for each man page found into a file to convert.

great job - thanks for the hard work

so there would need to be a process on each zot system where the sysadmin/sysprog would need to:

1. before activating the zot tools run a conversion tool
2. update /etc/ with the path to the converted man pages
3. activate the zot tools
4. run mandb
5. ???

@MikeFultonDev @IgorTodorovskiIBM

[lbdyck]

you can use this to get all the current man pages into text:

/* rexx */
x = bpxwunix('pwd',,s.,e.)
home = word(s.1,1) 

/* requires a tmp directory under users home directory */

x = bpxwunix('man -k .',,s.,e.)

do i = 1 to s.0
 if left(s.i,1) = ' ' then iterate
 page = word(s.i,1)
 say 'Processing' page
 cmd = 'man' page '>' home'/tmp/'page'.txt'
 x = bpxwunix(cmd,,sc.,ec.)
end

[MikeFultonDev]

Thanks! Didn’t know of man -k .

for your step 2 - is /etc where you have man pages on your system?

[lbdyck]

for (2) I meant to type /etc/profile :)

[MikeFultonDev]

almost @lbdyck ... Unfortunately some of the commands are multi-line, e.g. dnsdomainname and so your REXX code doesn't handle that right... The output from man -k for that entry is:

dnsdomain-   - Display the DNS domain name (dnsdomainname)
name

[lbdyck]

I can fix that in the AM if you like.

[MikeFultonDev]

you bet...

[lbdyck]

Try this - sorry it took so long

/* rexx */                                                
x = bpxwunix('pwd',,s.,e.)                                
home = word(s.1,1)                                        

/* requires a tmp directory under users home directory */ 

x = bpxwunix('man -k .',,s.,e.)                           

do i = 1 to s.0                                           
 if left(s.i,1) = ' ' then iterate                        
 page = word(s.i,1)                                       
 if right(page,1) = '-' then do                           
  i = i + 1                                               
  page = page''word(s.i,1)                                
 end                                                      
 say 'Processing' page                                    
 cmd = 'man' page '>' home'/tmp/'page'.txt'               
 x = bpxwunix(cmd,,sc.,ec.)                               
end                                                       

[MikeFultonDev]

looks right!

[MikeFultonDev]

next question - any idea how to get the 'man' page for a builtin? I see that

(export MANPATH=/usr/man/C; /bin/man -k . | grep which)

prints out:

which        - Display next executed command (tcsh shell)

I (eventually) guessed I could find the man page using tcshwhich:

(export MANPATH=/usr/man/C; /bin/man tcshwhich)

Using that trick, I could find the following:

(export MANPATH=/usr/man/C; /bin/man -k . | grep ' shell)')
alloc        - Show the amount of dynamic memory required (tcsh shell)
bindkey      - List all bound keys (tcsh shell)
breaksw      - Cause a break from a switch (tcsh shell)
builtins     - Prints the names of all builtin commands (tcsh shell)
bye          - Terminate a login shell (tcsh shell)
chdir        - Change the working directory (tcsh shell)
complete     - List all completions (tcsh shell)
default      - Label the default case in a switch statement (tcsh shell)
dirs         - Print, save or clear the directory stack (tcsh shell)
echotc       - Exercise the terminal capabilities in args (tcsh shell)
filetest     - Apply file inquiry operator to files (tcsh shell)
getspath     - Print the system execution path (tcsh shell)
getxvers     - Print the experimental version prefix (tcsh shell)
glob         - Write each word to standard output (tcsh shell)
hashstat     - Print statistic on hash table effectiveness (tcsh shell)
hup          - Run command so it exits on a hangup signal (tcsh shell)
limit        - Limit consumption of processes (tcsh shell)
log          - Print the watch tcsh shell variable (tcsh shell)
login        - Terminate a login shell (tcsh shell)
logout       - Terminate a login shell (tcsh shell)
migrate      - Migrate a process or job (tcsh shell)
notify       - Notify user of job status changes (tcsh shell)
popd         - Pop the directory stack (tcsh shell)
pushd        - Make exchanges within directory stack (tcsh shell)
rehash       - Recompute internal hash table (tcsh shell)
repeat       - Execute command count times (tcsh shell)
sched        - Print scheduled-event list (tcsh shell)
setenv       - Set environment variable name to value (tcsh shell)
setspath     - Set the system execution path (tcsh shell)
setty        - Control tty mode changes (tcsh shell)
setxvers     - Set the experimental version prefix (tcsh shell)
source       - Read and execute commands from name (tcsh shell)
telltc       - List terminal capability values (tcsh shell)
uncomplete   - Remove completions whose names match pattern (tcsh shell)
unhash       - Disable use of internal hash table (tcsh shell)
universe     - Set universe to universe (Masscomp/RTU only) (tcsh shell)
unlimit      - Remove resource limitations (tcsh shell)
watchlog     - Print the watch shell variable (tcsh shell)
where        - Report all known instances of command (tcsh shell)
which        - Display next executed command (tcsh shell)

I checked a couple, like tcshpopd and they seemed to work.

[lbdyck]

correction:

/* rexx */                                                       
x = bpxwunix('pwd',,s.,e.)                                       
home = word(s.1,1)                                               

/* requires a tmp directory under users home directory */        

x = bpxwunix('man -k .',,s.,e.)                                  

do i = 1 to s.0                                                  
 if left(s.i,1) = ' ' then iterate                               
 page = word(s.i,1)                                              
 if right(page,1) = '-' then do                                  
  i = i + 1                                                      
  page = left(page,length(page)-1)''word(s.i,1)                  
 end                                                             
 say 'Processing' page                                           
 cmd = 'man' page '>' home'/tmp/'page'.txt'                      
 x = bpxwunix(cmd,,sc.,ec.)                                      
end                                                               

[lbdyck]

needed to remove the - (sigh)

[lbdyck]

On the built-in question - no clue

[MikeFultonDev]

@lbdyck i made some progress. See: https://github.com/ZOSOpenTools/man-dbport/blob/zosmangen/bin/zosmangen which just generates the text files for man pages so far. It has warts in it still - part of the problem is that, at least for me, some of the entries from man -k . seem wrong - for example sshdconfig is listed instead of sshd_config. Is this true for you as well? Perhaps I am missing maintenance on my system?

The function ZOSCommands does the messy work...

[lbdyck]

I was thinking and remembered that the man -k . only works for the man pages in the ibm database. It has no 'knowledge' of other man pages such as those from Rocket or Co:Z. There must be a better way to find all available man pages.

[lbdyck]

This finds ALL man pages anywhere and everywhere providing the filesystem is mounted - not this will also include the zot pages which can be filtered by testing for their path

/* rexx */                                                
x = bpxwunix('pwd',,s.,e.)                                
home = word(s.1,1)                                        
pages = ''                                                

/* requires a tmp directory under users home directory */ 

x = bpxwunix('find / | grep /man/',,s.,e.)                

do i = 1 to s.0                                           
 if right(s.i,2) /= '.1' then iterate                     
 page = translate(s.i,' ','/')                            
 page = word(page,words(page))                            
 page = left(page,length(page)-2)                         
 if wordpos(page,pages) = 0 then do                       
    pages = pages page                                    
    txt = page'.txt'                                      
    dup.page = 0                                          
    end                                                   
 else do                                                  
      dup.page = dup.page + 1                             
      txt = page'.txt'dup.page                            
      end                                             
 say 'Processing' page 'into' txt 'from' s.i          
 cmd = 'man' page '>' home'/tmp/'txt                  
 x = bpxwunix(cmd,,sc.,ec.)                           
end                                                   

[lbdyck]

perhaps it would be worthwhile if the zot could distribute the ibm provided man pages, after conversion, as part of zot so each site does not have to do that conversion.

[MikeFultonDev]

that catches some more @lbdyck but doesn't it miss the man pages that are only in files like:

/usr/man/C/man1/arcd000.stfpt
/usr/man/C/man1/bpxa5mst.stfpt
/usr/man/C/man1/fnuz200.stfpt
/usr/man/C/man1/fotz200.stfpt

?

[lbdyck]

it does because I wasn't aware of those - the code can be easily fixed for those if you like. But not sure how to do find what's in them.

[lbdyck]

perhaps a combination of the 2 techniques is required. What is found in the stfpt/book files would be in the ibm mandb so would be found by the man -k . - thoughts?

[lbdyck]

Given these challenges - getting permission to distribute the ibm man pages after they have been converted for use with the zot man/mandb would make sense. Making the conversion process/tool available for ISV's would also be worthwhile (thinking of the Co:Z folks and Rocket)

[lbdyck]

try this (duplicates guaranteed)

/* rexx */
x = bpxwunix('pwd',,s.,e.)
home = word(s.1,1)
pages = ''

/* requires a tmp directory under users home directory */

  /* ----------------------- *
   | Process all mandb pages |
   * ----------------------- */
x = bpxwunix('man -k .',,s.,e.)

do i = 1 to s.0
 if left(s.i,1) = ' ' then iterate
 page = word(s.i,1)
 if right(page,1) = '-' then do
  i = i + 1
  page = left(page,length(page)-1)''word(s.i,1)
 end
 pages = pages page
 say 'Processing' page
 cmd = 'man' page '>' home'/tmp/'page'.txt'
 x = bpxwunix(cmd,,sc.,ec.)
end

  /* ---------------------- *
   | Now find all man pages |
   * ---------------------- */
x = bpxwunix('find / | grep /man/',,s.,e.)

do i = 1 to s.0
 if right(s.i,2) /= '.1' then iterate
 page = translate(s.i,' ','/')
 page = word(page,words(page))
 page = left(page,length(page)-2)
 if wordpos(page,pages) = 0 then do
    pages = pages page
    txt = page'.txt'
    dup.page = 0
    end
 else do
      dup.page = dup.page + 1
      txt = page'.txt'dup.page
      end
 say 'Processing' page 'into' txt 'from' s.i
 cmd = 'man' page '>' home'/tmp/'txt
 x = bpxwunix(cmd,,sc.,ec.)
end

[lbdyck]

Any chance of providing a tool that:

1. does the man -k . to collect the list of all ibm native man pages ** must be run prior to doing the zopen-config
2. uses the txt2man or ?? and then feeds the results somewhere that mandb will be able to find
3. run mandb

Then a site can do this when they 1st install zopen and after a z/OS upgrade.

[MikeFultonDev]

interesting idea @lbdyck . So the gist of it is:

• provide a tool that will scan the IBM and vendor-provided man pages that are not Linux man pages
• generate simple Linux man pages in a new directory (or set of directories)
• instruct the person how to update their MANPATH and PATH to use the z/OS Open Tools 'man' tool

Did I get that right?

What should we do for the handful of tools that we have 2 versions of (e.g. 'cp') ?

[IgorTodorovskiIBM]

interesting idea @lbdyck . So the gist of it is:

• provide a tool that will scan the IBM and vendor-provided man pages that are not Linux man pages
• generate simple Linux man pages in a new directory (or set of directories)
• instruct the person how to update their MANPATH and PATH to use the z/OS Open Tools 'man' tool

Did I get that right?

What should we do for the handful of tools that we have 2 versions of (e.g. 'cp') ?

Cool idea, one thought would be that we can prefix the legacy manpages with "zos-". So you would need to invoke zopen zos-cp to view it

We have also have similar issue with conflicting binaries (e.g. /bin/man and z/OS Open Tools man). One option is to extend zopen alt so that you can select z/OS binaries as alternatives. It would need to be more granular though, so not at the package level but at the executables level. We could also maintain a list of known z/OS tools to make it simpler...

[lbdyck]

I would say that the legacy man pages should be created in the right format and then added to the end of the MANPATH so that the newer man pages are found first and used. No need to prefix them with anything.

Same for the legacy binaries as those should be after the newer zot binaries in the PATH/`LIBPATH right?

[MikeFultonDev]

@lbdyck so you would be ok that you have to change your MANPATH or PATH if you want to get info on the z/OS cp or run the z/OS cp command?

[lbdyck]

If I wanted to see, or use, the original cp then I would bypass the zopen-config