Project Technical Roadmap

[chrisdaaz]

Deadline: March, 2025. This project is related to the stateless RHEL 8 upgrade and the new access model for GPUs.

Deliverables

• [ ] Managers and PIs can add users Users to Projects and Allocations
• [ ] Managers and PIs can remove Users from Projects and Allocations
• [ ] Managers and PIs can request compute resources
• [ ] Managers and PIs can request storage resources
• [ ] Managers and PIs can renew compute resources
• [ ] Managers and PIs can renew storage resources
• [ ] RCS can approve or deny requests for new or renewed resources
• [ ] Managers and PIs can view compute/ storage resource expiration dates

Environments

How do we develop, test, and deploy ColdFront to production?

• [x] DEV: Install and configure ColdFront application
  • [ ] (Optional) Build an image for use in containers
• [x] DEV: Install and configure PostgreSQL database to connect to ColdFront
• [ ] TEST: Install and configure ColdFront application
• [ ] TEST: Install and configure PostgreSQL database to connect to ColdFront
• [ ] PROD: Install and configure ColdFront application
• [ ] PROD: Install and configure PostgreSQL database to connect to ColdFront

Data Migration

• [x] Write and test data migration scripts to convert /etc/passwd, /etc/group, and allocation_log.txt entries into JSON objects for ColdFront import
• [ ] Quest allocations are represented as ColdFront Projects and Allocations
• [ ] Quest users are represented as ColdFront Users
• [ ] Quest resources are represented as ColdFront resources

Identity Management

How do we prevent overwriting the /etc/passwd, /etc/group, and /etc/shadow files with synchronous requests to add change user attributes?

There is an effort to overhaul user authentication and management in Quest as part of the move to RHEL 8, which will involve the creation of a centralized identity management solution.

• [ ] Set up and configure a centralized identity management system (Free IPA or Red Hat IDM) to manage UIDs and GIDs for Quest
• [ ] Determine how to grant ColdFront access to new Quest user management system, such as writing a plugin that is similar to the FreeIPA plugin ColdFront provides
• [ ] Connect ColdFront application to Northwestern LDAP service
• [ ] Connect ColdFront application to Northwestern Single sign-on service
• [ ] Enable Multi-factor authentication for access to ColdFront
• [ ] Enable ColdFront to read and write to the centralized IDM system

Slurm

How are ColdFront User and access permissions synced with Slurm accounts?

• [x] DEV: Install, configure, and connect a Slurm instance with ColdFront
• [ ] TEST: Install, configure, and connect a Slurm instance with ColdFront
• [ ] PROD: Install, configure, and connect a Slurm instance with ColdFront

[chrisdaaz]

We can attempt to sync the login and analytics nodes to be near-instantaneous, whereas the compute nodes could be every 15 or 20 minutes.

[chrisdaaz]

The UIDs and GIDs that Quest currently uses are different than what the ISO Authentication System uses. Moving to the ISO Authentication System would require changing permissions on every file.

We will stick to our current local authentication system (IDM) in the meantime. We should use a centralized identity management solution, which will serve as the source of truth for authentication management. This would be baked into the RHEL 8 image. This project is happening with or without ColdFront. Red Hat IDM is the same as FreeIPA.

The FreeIPA app would check with LDAP for a match. FreeIPA manages UIDs and GIDs, which is not provided by LDAP.

[chrisdaaz]

ColdFront-to-Quest Onboarding Process

• User logs into ColdFront
• ColdFront authenticates user via Northwestern SSO and LDAP
• ColdFront retrieves the User's NetID, Name, and Email Address and stores it in a database
• The User exists in ColdFront
• The User creates a Project or is added to a Project
• The User creates an Allocation within a Project or is added to an Allocation within a Project
  • A plugin will retrieve Users info from ColdFront, create a Linux user on Quest (if needed), and add them to the Linux group associated with the Allocation
  • The Slurm plugin will retrieve the User's info from ColdFront (netid, email) and create a Slurm account (if needed) with any relevant associations to the Allocation
• The User may login to Quest and submit jobs