[Q] How to ensure that user-specific to autostart after IPL?

KerimG commented 1 year ago

I created a Node.js app that I'm trying to autostart after IPL. But it seems like it's not exactly working. The configuration of the service is as follows and was created using scinit without any modifications afterwards. I thought it might be related to the issue of running multiple STRTCPSVR commands simultaneously and did set ADDENVVAR ENVVAR(SC_TCPSVR_SUBMIT) VALUE('Y') LEVEL(*SYS) REPLACE(*YES)

My config is as follows

gueney@TFSVR3:~ $ sc info autostart_example

---------------------------------------------------------------------
autostart_example (Autostart Example 1)

Defined in: /home/GUENEY/.sc/services/autostart_example.yml

Working Directory: /home/GUENEY/code/autostart_example

Startup Command: node src
Startup Wait Time (s): 60

Shutdown Wait Time (s): 45

Check-alive conditions: PORT:9998
Batch Mode: <submitted to batch>
    Batch Job Name: SC_autosta

Inherits environment variables?: true
Custom environment variables:
    PATH=/QOpenSys/pkgs/bin:/QOpenSys/pkgs/lib/nodejs14/bin/:/home/GUENEY/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:.:/usr/bin
    PORT=9998
---------------------------------------------------------------------

gueney@TFSVR3:~ $ sc loginfo autostart_example
autostart_example: ¯\_😀_/¯ (try checking in log directory /home/GUENEY/.sc/logs)

ThePrez commented 1 year ago

Greetings. Authority/permissions issues are the most common reason for autostart failure. You may need to manually edit the YAML and use sbmjob_opts

See this snippet from the PostgreSQL sample: https://github.com/ThePrez/ServiceCommander-IBMi/blob/main/samples/oss_common/postgres.yaml#L17

ThePrez commented 1 year ago

Ope! Also, because the strtcpcvr code runs the QTCP user profile, the service definition needs to be available to that user. The easy way is to make it global (you can move your .yml file to `/QOpenSys/etc/sc/services)

KerimG commented 1 year ago

Hi Jesse,

thanks for the response. I tried following configuration:

The service configuration is placed in the global services folder and the Node.js project is also in a publicy accessible project folder.

gueney@TFSVR3:~ $ cat /QOpenSys/etc/sc/services/auto5.yml
name: Autostart 5
dir: /nodejs/autostart_example
start_cmd: node src
check_alive: '10002'
batch_mode: 'true'
sbmjob_opts: user(gueney)
environment_vars:
- PATH=[Pathy Stuff]
- PORT=10002
- NODE_ENV=development
groups:
- autostart

However, that didn't work either. Checking the spoolfiles for QTCP reveals:

Performing operation 'START' on service 'auto5'                                                                                   
ERROR: Service 'auto5' is configured to run as user 'gueney' but that profile doesn't exist or you do not have sufficient authorit

the user 'gueney' exists, since I'm logged into it. What authorities does QTCP need to start a job with a specific user during IPL?

chrjorgensen commented 1 year ago

@KerimG

First, thanks for your great and enthusiastic presentation about IBM i and Open Source tools on CEC2023!

Second, to submit a job for another user on IBM i, the user submitting the job must have *ALLOBJ or *USE authority to the other user. This is also a requirement for SC, which simply builds and executes a SBMJOB command when starting a service in batch mode.

For your setup to work, QTCP (running SC for AUTOSTART(*YES)) does not have *ALLOBJ (by default) and thus must have *USE authority to your userprofile gueney!

This is maybe not what you want - I personally don't like other users having *USE authority on my userprofile, since I'm a system administrator and have additional capabilities.

In our setup, we abandoned AUTOSTART(*YES) on the STRTCPSVR setup for Service Commander and have created our own scheduled job to run STRTCPSVR SERVER(*SC) INSTANCE(*AUTOSTART) at IPL with a service account having *ALLOBJ authority. This way we avoid the security issues you're experiencing, and all our autostart services start as expected.

KerimG commented 1 year ago

Hello @chrjorgensen ,

I am glad you enjoyed it! It was initially supposed to be a workshop so that people could leave with a nice PASE setup and do the examples I showed on their own but scheduling issues made that impossible. Maybe next time 😁

Thank you for that information. I am relatively new to the IBM i's authority/permission management and I think I was really needing that info.

The scheduled job idea is great. I'll try that out asap!

chrjorgensen commented 1 year ago

@KerimG

We use the built-in job scheduler for all jobs needed to be started at IPL after the system is back to normal. The last command in our QSTRUP program calls a CL command, which runs the following SQL:

/* Start IPL Scheduled Jobs.                                                                      */

chgvar     &Sql (                                                                                +
  'begin                                                                                 ' *tcat +
  '  declare cmd varchar(256);                                                           ' *tcat +
  '  for                                                                                 ' *tcat +
  '    select                                                                            ' *tcat +
  '           right(digits(SCHEDULED_JOB_ENTRY_NUMBER), 6) as SCHEDULED_JOB_ENTRY_NUMBER ' *tcat +
  '         , SCHEDULED_JOB_NAME                                                         ' *tcat +
  '      from qsys2.scheduled_job_info                                                   ' *tcat +
  '     where SCHEDULED_DATE_VALUE = ''SCHEDULED_DATE'' and                              ' *tcat +
  '           upper(left(DESCRIPTION, 7)) = ''AT IPL:''                                  ' *tcat +
  '    do                                                                                ' *tcat +
  '      set cmd = ''CHGJOBSCDE JOB('' concat SCHEDULED_JOB_NAME concat '') '' concat    ' *tcat +
  '                           ''ENTRYNBR('' concat SCHEDULED_JOB_ENTRY_NUMBER concat     ' *tcat +
  '                               '') '' concat                                          ' *tcat +
  '                           ''SCDDATE(*CURRENT) SCDTIME(*CURRENT)'';                   ' *tcat +
  '      call qcmdexc( cmd );                                                            ' *tcat +
  '  end for;                                                                            ' *tcat +
  'end                                                                                   '       +
             )

runsql     ( &Sql ) commit(*none)

The SQL runs through all scheduled jobs and changes those having the text "At IPL:" to SCDDATE(*CURRENT) SCDTIME(*CURRENT), thus making the scheduler start those jobs.

By having this, we can easily add new IPL jobs by adding a job schedule entry with the text starting with "At IPL:" and a future date. And it is easy to restart the job, simply change the job with SCDDATE(*CURRENT) SCDTIME(*CURRENT).

I ought to make a blog entry about this - it has been on my todo list for too long time now! 😆

KerimG commented 1 year ago

That is a really cool way! Would love to read the blog article about it.

Is this CCSID-aware? We have a German system running with CCSID 273 (or actually 1141, I think) and have had problems in the past with text-comparison based operations because of CCSID issues.

Alternatively, I guess one could write the SCHEDULED_JOB_ENTRY_NUMBERs (or names) into a DTARAA or a table and check that table with QRTRUP instead of the "AT IPL:" comparison.

chrjorgensen commented 1 year ago

There is no CCSID issue - all characters in AT IPL: are invariant, i.e. have the same hexcode in all codepages. You can put the text everywhere, I just chose to make it a constant in the SQL statement. Haven't had the need for changing this or use other value(s).

chrjorgensen commented 1 year ago

You can use any criteria for selecting the scheduled job, and store this criteria somewhere. But you add an additional step besides adding the scheduled job by having to add the scheduled job to your criteria. Our method is the simplest and has worked fine for us for years now. 😃

KerimG commented 1 year ago

There is no CCSID issue - all characters in AT IPL: are invariant, i.e. have the same hexcode in all codepages.

Duh, you're absolutely right. So cool! I'll show this to our sysadmin. Thanks so much @chrjorgensen

ThePrez commented 1 year ago

ork, QTCP (running SC for AUTOSTART(*YES)) does not have *ALLOBJ (by default) and thus must have *USE authority to your userprofile gueney!

One thing to add, though: at latest versions, the STRTCPSVR handler program is built with USRPRF(*OWNER), and the owner of the program is set to QSYS. So generally, the handler program should now have access to the necessary profile https://github.com/ThePrez/ServiceCommander-IBMi/blob/62c1c3ef8e4f61ec996cb2884470fbffdabe1af7/strtcpsvr/install_sc_tcpsvr#L32 https://www.ibm.com/docs/en/i/7.3?topic=keywords-usrprfuser-owner

It's still complicated:

running with sbmjob_opts: user(gueney) should solve the problem
submitting to batch as QTCP won't properly emulate the IPL permissions unless using the handler program (the easy way would be to submit STRTCPSVR)
When the handler program spawns the service commander job, it loses the authority adopted through QSYS (the sbmjob_opts gets around this issue by delegating to the proper owner before said user privileges are needed)

Are you willing to verify that the sbmjob_opts solves your problem?

Certainly this is a common issue in user experience with autostarting jobs and:

Docs should be updated with some explanation/pointers about the authority issue
scinit should default to using sbmjob_opts so things "just work" a large percentage of time.

ThePrez commented 1 year ago

Are you willing to verify that the sbmjob_opts solves your problem?

Oops.... Sorry, I see that you already did. I wonder if that failure is a separate bug. We should take another look at that check

ThePrez commented 1 year ago

However, that didn't work either. Checking the spoolfiles for QTCP reveals:

Performing operation 'START' on service 'auto5'                                                                                   
ERROR: Service 'auto5' is configured to run as user 'gueney' but that profile doesn't exist or you do not have sufficient authorit

That check is coming from this section of code:

    private void verifyBatchUser() throws SCException {
        final String batchUser = getBatchUser();
        if (StringUtils.isEmpty(batchUser)) {
            return;
        }
        final File usrprfChecker = new File("/qsys.lib/" + batchUser + ".usrprf");
        if (!usrprfChecker.exists()) {
            throw new SCException(m_logger, FailureType.INVALID_SERVICE_CONFIG, "ERROR: Service '%s' is configured to run as user '%s' but that profile doesn't exist or you do not have sufficient authorities!!", m_mainService.getName(), batchUser);
        }
    }

I don't know why that'd be failing, but it seems to me like we're throwing a false/inappropriate error here.

@chrjorgensen , anything obvious that I'm missing?

chrjorgensen commented 1 year ago

@ThePrez No, nothing obvious - I'm as puzzled as you are... Are we sure the user running SC at the time of the userprofile check has the right authority? I don't know how to debug this, I'm not a Java programmer... :wink:

ThePrez / ServiceCommander-IBMi

[Q] How to ensure that user-specific to autostart after IPL? #220