Open ju1es opened 9 months ago
Create the endpoints needed by the frontend application to create and save the guest tasks. The guest onboarding process requires the guest to complete multiple tasks. Each task will have some associated data that needs to be stored. The onboarding is considered complete once all tasks are complete.
The backend should store the overall progress of the guest application, along with the detailed information associated with each task.
In this initial PR, we will only store completed tasks. We can introduce draft task persistance in the future, this PR will be complex enough without this additional feature.
application_and_onboarding
host_matching
host_match_finalization
Great research! If possible, we should try to avoid representing these very detailed questions and answers as concrete data models. An "easy" approach would be to define a separate model for each application and add an entry for each user. Doing this, however, would be cumbersome and brittle. Adding/Removing questions would require altering our data model, and adding new applications would require creating new models. Updates to the underlying model are expensive because each time you update a database model you need to migrate previous versions. See the research sections I added below for an idea on how to achieve this.
As a starting point, we will take questions from this application https://docs.google.com/forms/d/e/1FAIpQLSc2Zm709r_7avFQYcaL9pZRwAnUknCZengn8rXP6jxx3sm9vQ/viewform?gxids=7757.
No. This is a problem because we need to know if the user is a guest, a host, an admin, etc. to be able to dictate what actions they can take on the webapp, i.e. what endpoints they are allowed to access. This could also be a privacy concern because we need to make sure guests and hosts can only access their own data.
# HUU API
# /api/openapi_server/models/database.py
from sqlalchemy import create_engine
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, DateTime, LargeBinary, Boolean
from os import environ as env
DATABASE_URL = env.get('DATABASE_URL')
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True, index=True)
first_name = Column(String(80))
last_name = Column(String(80))
email = Column(String, unique=True, index=True)
email_confirmed = Column(Boolean, default=False)
password_hash = Column(String)
date_created = Column(DateTime)
is_admin = Column(Boolean, default=False)
is_host = Column(Boolean, default=False)
is_guest = Column(Boolean, default=False)
# optional decriptive string representation
def __repr__(self):
return f"<User(id={self.id}, first_name={self.first_name}, last_name={self.last_name}, email={self.email}, date_created={self.date_created}, is_admin={self.is_admin}, is_host={self.is_host}, is_guest={self.is_guest})>"
# creating tables from models in db
from sqlalchemy import create_engine
engine = create_engine("sqlite://", echo=True, future=True)
Base.metadata.create_all(engine)
{
"first_name": "Alejandro",
"last_name": "Gomez",
"email": "ale.gomez@hackforla.org",
"email_confirmed": false,
"password_hash": "lfOcifi3DoKdjfvhwlrbugvywe3495!#$%",
"date_created": "2023-09-19 12:00:00",
"is_admin": false,
"is_host": false,
"is_guest": true
}
# api/openapi_server/repository/guest_repository.py
from typing import Optional, List
from sqlalchemy.orm import Session
from openapi_server.models.database import User, DataAccessLayer
"""
inserting data into the database
"""
class GuestRepository:
def create_guest(self, data: dict) -> Optional[User]:
"""
create a new guest - adds data entry to the database
"""
with DataAccessLayer.session() as session:
new_guest = User(
# db auto generates and auto increments an id
first_name=data["first_name"],
last_name=data["last_name"],
email=data["email"],
email_confirmed=data["email_confirmed"],
password_hash=data["password_hash"],
date_created=data["date_created"],
is_admin=data["is_admin"],
is_host=data["is_host"],
is_guest=data["is_guest"]
)
session.add(new_guest)# places instance into the session
session.commit() # writes changes to db
session.refresh(new_guest) # erases all attributes of the instance and refreshes them with the current state of the db by emitting a SQL query. this is important for autoincrementing id
return new_guest # returns the info from the db to the business logic
# api/openapi_server/repository/guest_dashboard_repository.py
from typing import Optional, List
from sqlalchemy.orm import Session
from openapi_server.models.database import User, DataAccessLayer
class GuestRepository:
def get_guest_by_id(self, id: int) -> Optional[User]:
"""
gets a guest by id
"""
with DataAccessLayer.session() as session:
return session.query(User).get(id)
Good example showing a reference, but the terminology 'task' and 'subtask' is a bit misleading. A task has a specific meaning in asynchronous programming. It is typically a wrapper around a function that you want to run asynchronously.
I'm thinking instead of Task and Subtask, we can have a Dashboard, Application, & Questions. A dashboard has several ordered applications, with a progress indicator for each item.
Addressed further in a comment below. We are using task_group -> tasks
# HUU API
# /api/openapi_server/models/database.py
from sqlalchemy import create_engine
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session, relationship
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, DateTime, LargeBinary, Boolean
from os import environ as env
DATABASE_URL = env.get('DATABASE_URL')
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True, index=True)
first_name = Column(String(80))
last_name = Column(String(80))
email = Column(String, unique=True, index=True)
email_confirmed = Column(Boolean, default=False)
password_hash = Column(String)
date_created = Column(DateTime)
is_admin = Column(Boolean, default=False)
is_host = Column(Boolean, default=False)
is_guest = Column(Boolean, default=False)
tasks = relationship("Task", backref="user")
class Task(Base):
__tablename__ = 'task'
id = Column(Integer, primary_key=True, index=True)
guest_id = Column(Integer, ForeignKey("user.id"))
title = Column(String(80))
status = Column(String(80))
subtasks = relationship("Subtask", backref="task")
class Subtask(Base):
__tablename__ = 'subtask'
id = Column(Integer, primary_key=True, index=True)
guest_id = Column(Integer, ForeignKey("user.id"))
task_id = Column(Integer, ForeignKey("task.id"))
status = Column(String(80))
openapi.yaml
file
# HomeUniteUs/api/openapi_server/openapi/openapi.yaml
post: # POST Request
description: Sign in a user
operationId: signin
requestBody: # email and password as a string enters endpoint
content:
application/json:
schema:
type: object
properties:
email:
type: string
password:
type: string
required:
- email
- password
responses:
'200':
content:
application/json:
schema: # response comes from the schema... shown below
$ref: '../../openapi.yaml#/components/schemas/ApiResponse'
description: successful operation
tags:
- auth
x-openapi-router-controller: openapi_server.controllers.auth_controller
#######################################################################
# New file below #
#######################################################################
# HomeUniteUs/api/openapi_server/openapi/openapi.yaml
# **endpoints and rest of yaml doc goes here...**
components:
securitySchemes:
jwt:
type: http
scheme: bearer
bearerFormat: JWT
x-bearerInfoFunc: openapi_server.controllers.security_controller.requires_auth
responses:
$ref: "./responses/_index.yaml"
parameters:
$ref: "./parameters/_index.yaml"
schemas:
$ref: "./schemas/_index.yaml"
ApiResponse: # according to this, the response should be a code, type, and message
example:
code: 0
type: type
message: message
properties:
code:
format: int32
title: code
type: integer
type:
title: type
type: string
message:
title: message
type: string
title: ApiResponse
type: object
{ "token" : access_toke, "user" : user}
which doesn't fit the schema above# HomeUniteUs/api/openapi_server/openapi/openapi.yaml
openapi: 3.0.0
info:
license:
name: GPL 2.0
title: Home Unite Us
version: 1.0.0
servers:
- url: http://homeunite.us/api
paths:
/users/{user_email}: # added path
$ref: "./paths/usersEmail.yaml"
# rest of paths below...
# HomeUniteUs/api/openapi_server/openapi/parameters/_index.yaml
Email:
name: user_email
description: The email of the user to retrieve
in: path
required: true
schema:
type: string
style: simple
# HomeUniteUs/api/openapi_server/openapi/schemas/_index.yaml
ApiResponse:
example:
code: 0
type: type
message: message
properties:
code:
format: int32
title: code
type: integer
type:
title: type
type: string
message:
title: message
type: string
title: ApiResponse
type: object
User:
type: object
properties:
id:
format: int64
type: integer
first_name:
title: first_name
type: string
last_name:
title: last_name
type: string
email:
title: email
type: string
email_confirmed:
title: email_confirmed
type: boolean
password_hash:
title: password
type: string
date_created:
title: date_created
type: string
format: date-time
is_admin:
title: is_admin
type: boolean
is_host:
title: is_host
type: boolean
is_guest:
title: is_guest
type: boolean
required:
- id
- first_name
- last_name
- email
- email_confirmed
- date_created
- is_admin
- is_host
- is_guest
# HomeUniteUs/api/openapi_server/openapi/paths/usersEmail.yaml
get:
description: Get user by email
operationId: get_user_by_email # this is the controller function
tags:
- users
parameters:
- $ref: "../parameters/_index.yaml#/Email"
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: "../schemas/_index.yaml#/User"
default:
description: Unexpected error
$ref: "../responses/_index.yaml#/UnexpectedError"
x-openapi-router-controller: openapi_server.controllers.users_controller
pytest
from the 'api' directory and all tests in the 'test' directory will run. It will display =================== 27 passed, 209 warnings in 8.30s ===================
Fixtures are functions that typically provide resources for the tests.
import pytest
def create_user(first_name, last_name):
return {"first_name": first_name, "last_name": last_name}
@pytest.fixture
def new_user():
return create_user("Alejandro", "Gomez")
@pytest.fixture
def users(new_user):
return [create_user("Jose", "Garcia"), create_user("Juan", "Lopez"), new_user]
def test_new_user_in_users(new_user, users):
assert new_user in users
Might not be important in this issue, but it is important to note that pytest fixtures also allow you to perform setup and teardown. You can do this using the
yield
keyword.
Thanks for showing this! This will be super useful
@pytest.fixture
def demo_setup_teardown():
# Code before yield is executed before test starts
doSetup()
yield testData
# Code after yield is executed after test finishes
# Even if an exception is encountered
doTeardown()
# HomeUniteUs/api/openapi_server/test/test_service_provider_repository.py
# Third Party
import pytest
# Local
from openapi_server.models.database import DataAccessLayer
from openapi_server.repositories.service_provider_repository import HousingProviderRepository
@pytest.fixture
def empty_housing_repo() -> HousingProviderRepository:
'''
SetUp and TearDown an empty housing repository for
testing purposes.
'''
DataAccessLayer._engine = None
DataAccessLayer._conn_string = "sqlite:///:memory:"
DataAccessLayer.db_init()
yield HousingProviderRepository()
test_engine, DataAccessLayer._engine = DataAccessLayer._engine, None
test_engine.dispose()
@pytest.fixture
def housing_repo_5_entries(empty_housing_repo: HousingProviderRepository) -> HousingProviderRepository:
'''
SetUp and TearDown a housing repository with five service providers.
The providers will have ids [1-5] and names Provider 1...Provider5
'''
for i in range(1, 6):
new = empty_housing_repo.create_service_provider(f"Provider {i}")
assert new is not None, f"Test Setup Failure! Failed to create provider {i}"
assert new.id == i, "The test ids are expected to go from 1-5"
yield empty_housing_repo
# this function adds a value to the db and checks it
def test_create_provider(empty_housing_repo: HousingProviderRepository):
'''
Test creating a new provider within an empty database.
'''
EXPECTED_NAME = "MyFancyProvider"
newProvider = empty_housing_repo.create_service_provider(EXPECTED_NAME) # adds value to the db via a pytest.fixture of a db and HousingProviderRepository
# checks the value returned to assert a test status
assert newProvider is not None, "Repo create method failed"
assert newProvider.id == 1, "Expected id 1 since this is the first created provider"
assert newProvider.provider_name == EXPECTED_NAME, "Created provider name did not match request"
# HomeUniteUs/api/openapi_server/test/test_service_provider_controller.py
from __future__ import absolute_import
from openapi_server.test import BaseTestCase
class TestServiceProviderController(BaseTestCase):
"""ServiceProviderController integration test stubs"""
def test_create_service_provider(self):
"""
Test creating a new service provider using a
simulated post request. Verify that the
response is correct, and that the app
database was properly updated.
"""
REQUESTED_PROVIDER = {
"provider_name" : "-123ASCII&" # creates a fake test object
}
response = self.client.post(
'/api/serviceProviders',
json=REQUESTED_PROVIDER) # sends POST request with payload to API endpoint
self.assertStatus(response, 201, # asserts the response status
f'Response body is: {response.json}')
assert 'provider_name' in response.json # asserts data in response object
assert 'id' in response.json
assert response.json['provider_name'] == REQUESTED_PROVIDER['provider_name']
# verifies there was a 'write' on the db, i.e. that the db was updated based on the API endpoint response
db_entry = self.provider_repo.get_service_provider_by_id(response.json['id'])
assert db_entry is not None, "Request succeeeded but the database was not updated!"
assert db_entry.provider_name == REQUESTED_PROVIDER['provider_name']
We hope to support multiple providers. Each provider will have their own set of requirements. Ideally providers would be able to add new questions, and custom tailor their guest and host applications.
Our design should support unique applications for each provider. If we create explicit database models for each application, then we will need to modify our database model each time an application is edited or each time a new application is added.
We can avoid restructuring our entire database model each time a routine application modification is made by abstracting the application requirements and constructing the provider applications at runtime.
To achieve this, we could store each application as table. Applications would contain a list of ordered questions. Adding a question would require adding a row to an existing application table. Adding a new application would require adding a new application table. In both cases the underlying schema would remain the same.
The ER diagram was generated using the ERAlchemy package. What's interesting is that most of these models are not used by our application at all.
This design supports provider-specific applications, and would allow us to add/remove/edit applications without editing the data model.
The provider_id & role define the dashboard that the frontend will use. The dashboard defines the collection of applications that need to be completed, along with each application's progress. The application defines the set of questions that need to be asked. Each question defines the question's text and the expected response type.
Responses to the User's question are stored in the Response table. We can use this table to easily look up user responses for a given application, using the (user_id, question_id) composite key. If no response is found then we know that application has an unanswered question.
---
title: HUU Application Dashboard Data Model
---
erDiagram
User {
int user_id PK
int provider_id FK
string name
string email UK
enum role "Guest,Host,Coord"
}
Provider {
int provider_id PK
string name
}
Dashboard {
int dashboard_id PK
int provider_id FK
string title
enum role "Provider has guest & host dashboard"
}
Application {
int app_id PK
int dashboard_id FK
int dashboard_order
string title
enum progress
}
App_Question_Junction {
int app_id FK
int question_id FK
int question_order
}
Question {
int question_id PK
int text
int response_type_id FK
}
ResponseType {
int response_type_id PK
string type_name "str, bool, int, etc"
}
Response {
int user_id FK
int question_id FK
string response_value
date timestamp
}
Provider }|--|| User : has
Provider ||--|{ Dashboard : defines
Dashboard ||--|{ Application :"consists of"
Application }|--o{ Question : "utilizes"
ResponseType ||--o{ Question : "defines"
User }o--o{ Response : "user answers"
Do we do anything to identify Guest user accounts as 'Guest', as opposed to 'Host', 'Coordinator', 'etc'? The Applicant model provides a mechanism for associating a user with a role. It doesn't look like we use this model yet.
I think we should consider restarting our models as you suggested. Or at minimum get together and clean up (remove) unused models.
We have a model that handles the identification of user, i.e. applicant type. Here's the models used for signing up a user
# HomeUniteUs/api/openapi_server/models/database.py
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True, index=True)
email = Column(String, nullable=False, unique=True)
class ApplicantType(Base):
__tablename__ = "applicant_type"
id = Column(Integer, primary_key=True, index=True)
applicant_type_description = Column(String, nullable=False)
class ApplicantStatus(Base):
__tablename__ = "applicant_status"
id = Column(Integer, primary_key=True, index=True)
applicant_type = Column(Integer, ForeignKey('applicant_type.id'), nullable=False)
status_description = Column(String, nullable=False)
class Applicant(Base):
__tablename__ = "applicant"
id = Column(Integer, primary_key=True, index=True)
applicant_type = Column(Integer, ForeignKey('applicant_type.id'), nullable=False)
applicant_status = Column(Integer, ForeignKey('applicant_status.id'), nullable=False)
user = Column(Integer, ForeignKey('user.id'), nullable=False)
Here is what the code could like if we wanted to implement a new guest user, indicating their role
# HomeUniteUs/api/openapi_server/controllers/auth_controller.py
from openapi_server.models.database import DataAccessLayer, User, ApplicantType, ApplicantStatus, Applicant
def signUpGuest(): # noqa: E501
"""Signup a new Guest
"""
if connexion.request.is_json:
body = connexion.request.get_json()
secret_hash = get_secret_hash(body['email'])
# Signup user as guest
with DataAccessLayer.session() as session:
user = User(email=body['email'])
applicant_type_id = session.query(ApplicantType.id).filter_by(applicant_type_description="guest").first()
applicant_status = ApplicantStatus(
applicant_type=applicant_type_id,
status_description="unconfirmed_email"
)
session.add_all([user, applicant_status])
# commit and refresh to db to be able to get applicant_status.id and user.id as they will be autogenerated
session.commit()
session.refresh(user, applicant_status)
applicant = Applicant(
applicant_type=applicant_type_id,
applicant_status=applicant_status.id,
user = user.id
)
session.add(applicant)
try:
session.commit()
except IntegrityError:
session.rollback()
raise AuthError({
"message": "A user with this email already exists."
}, 422)
try:
response = userClient.sign_up(
ClientId=COGNITO_CLIENT_ID,
SecretHash=secret_hash,
Username=body['email'],
Password=body['password'],
ClientMetadata={
'url': ROOT_URL
}
)
return response
except botocore.exceptions.ClientError as error:
match error.response['Error']['Code']:
case 'UsernameExistsException':
msg = "A user with this email already exists."
raise AuthError({ "message": msg }, 400)
case 'NotAuthorizedException':
msg = "User is already confirmed."
raise AuthError({ "message": msg }, 400)
case 'InvalidPasswordException':
msg = "Password did not conform with policy"
raise AuthError({ "message": msg }, 400)
case 'TooManyRequestsException':
msg = "Too many requests made. Please wait before trying again."
raise AuthError({ "message": msg }, 400)
case _:
msg = "An unexpected error occurred."
raise AuthError({ "message": msg }, 400)
except botocore.excepts.ParameterValidationError as error:
msg = f"The parameters you provided are incorrect: {error}"
raise AuthError({"message": msg}, 500)
The plan is to implement endpoints that the frontend can use to query a user-specific dashboard populated with applications & their progress. The dashboard will contain app_ids
that can be used to load and save the application in the backend. Each application will contain an ordered list of questions and responses
---
title: Application Logic Flow
---
flowchart TB;
subgraph frontend;
f1["Client\nSign in"]
f2["Load page\nusing role"]
f3["Show Dashboard\n & Wait"]
f4["Dashboard\nApp OnClick"]
f5["Display\nApp"]
f6["App\nOnSave"]
f1 ~~~ f2 ~~~ f3 ~~~ f4 ~~~ f5 ~~~ f6
end
subgraph backend;
b1["sign-in()"]
b2["app_dashboard()"]
b3["get_app"]
b4["update_app()"]
b1 ~~~ b2 ~~~ b3 ~~~ b4
end
f1 --"POST\n{\nusername,\n password\n}"--> b1
b1 --"{\n role\n jwt\n}"--> f2
f2 --"GET\n/api/dashboard"--> b2
b2 --"{\napp1 {\nprogress,\ntext,\napp_id\m},\napp...\n}"--> f3
f4 --"GET\n/api/application/app_id"--> b3
b3 --"{\nprogress\nquestion1{\n progress,\n text,\n response_type,\n response\n},\nquestion2...\n}"--> f5
f6 --"PUT\n{\nprogress\nquestion1{\n progress,\n text,\n response_type,\n response\n},\nquestion2...\n}"--> b4
linkStyle 8,9,10,11,12,13,14 text-align:left
Implement a flexible application model backend system, that will allow the front-end app to query Guest and Host applications using a user_id. See the model erdiagram and application flow diagram above.
Each user will be associated with a single provider. Each provider will have a unique guest and host application. The front-end application will receive all the information it needs to dynamically generate guest and host applications.
Our backend has a 'demo' database model, however many of the models are currently unused by the frontend application. We need to decide if we want to start fresh, or if want to modify the existing model to meet our current needs. Modifying an unused model can be very challenging since we do not know if the current unused model works. I propose removing the unused models and enforcing a requirement that all new models need to be used by either the frontend app or our test project.
Model
We will implement the model by starting at the user and working our way towards responses. We will create test cases to exercise the new models at each step.
Role
enumerated type with Guest
, Host
, Coordinator
, and Admin
rolesUser
model to include a role and provider_idsignup
methods to specify a role when the user signs upResponseType
model that defines the supported response types. The frontend will rely on this to interpret and edit the question responsesQuestion
model to store the bank of questions. The question bank will be shared across applications.Dashboard
model to store a provider's role-based dashboards.Application
model to store the dashboard's ordered list of applicationsAppQuestionJunction
table to store the many-to-many relationship between each application and the ordered list of questionsDatabase Migration
With these changes we will need to store the dashboard and application structure within the database, since this structure can be defined and edited by each provider.
Endpoint
app_dashboard()
endpoint
app_id
that can be used to query the applicationget_app()
endpoint
Response
table using the user_id
and question_id
. If no response is found then the user has not answered that question. If a response is found, then return it within the application.update_app()
endpoint
PUT
the same basic json received from the get_app()
endpoint, except the response_value
fields will contain user responsesFrontend
This will be apart of a separate issue.
Hey @agosmou,
An initial project plan is ready for this issue. Please submit your answers and reassign me once it is ready for review.
My comments are included as
quoted text
Please don't remove those section. I'll remove them as I review your responses.
Working through this on markdown doc in VSCode. Ill paste it in here when I finish over the next day or so.
@agosmou @Joshua-Douglas I simplified the dashboard data a bit and wanted to leave an example here for reference incase it's helpful.
Sample response for tasks and sub-tasks:
{
id: 1,
title: 'Application and Onboarding',
status: 'in-progress',
subTasks: [
{
id: 1,
title: 'Application',
status: 'complete',
description:
'Start your guest application to move on to the next step.',
buttonText: 'Start application',
url: '/guest-application',
},
{
id: 2,
title: 'Coordinator Interview',
status: 'in-progress',
description: 'Meet with your Coordinator to share more about yourself.',
buttonText: 'Schedule interview',
url: '/schedule',
},
{
id: 3,
title: 'Training Session',
status: 'locked',
description:
'Complete a training session to prepare you for the host home experience.',
buttonText: 'Schedule training',
url: '/schedule',
},
],
},
Potential entity relation diagram:
erDiagram
USER ||--|{ TASK : has
TASK {
int id PK
int userId FK
string title
int statusId FK
}
TASK ||--|{ SUB_TASK: contains
TASK ||--|| STATUS: has
SUB_TASK ||--|| STATUS: has
SUB_TASK{
int id PK
int taskId FK
string title
string description
int statusId FK
string buttonText
url string
}
STATUS {
int id PK
string status "locked, inProgress, complete"
}
@erikguntner
Awesome info. Thanks Eric! I'm wrapping up the design doc. I'll post this afternoon.
@agosmou this looks awesome! Just FYI Cognito tracks whether the user is confirmed in case that changes whether we want to keep that info in the database as well or not
Hey @agosmou, Thanks for the designs! I'll make sure to review them by Wednesday night!
Figma Designs for backend-design consideration
@Joshua-Douglas - I can review these tomorrow night to see if there are any effects on the above
Hey @erikguntner,
Thanks a lot for the Task/SubTask idea. After reading through @agosmou's research I think a generic approach like this is going to be easier to implement than the more 'naive' approach of defining each application as a separate model.
It would be great if you could look through the proposed data model & application flowchart I posted above. If the frontend is already relying on the backend to query the dashboard and applications then that gives us the flexibility to define provider-specific and role-specific dashboards. This would mean that each individual provider could define a custom Guest and Host dashboards, complete with custom applications.
Hey @agosmou,
The design document is ready for review! Your research looked great, and it inspired me to think up a generic approach that could accommodate the Task/Subtask approach outlined by @erikguntner.
I left
comments
on some of your responses, and added several new sections (everything after the Data Model Questions
section). I left one question un-answered, so I could really use your help getting to the bottom of it.
Can you answer that question, and review the design? I'd like to get your feedback. I'm sure something is missing or could be improved. If you agree with the general approach then I'll create a set of implementation questions that we can use to outline the trickiest parts of the implementation before opening a branch.
@agosmou The Guest Dashboard story #500 uses Dashboard -> Steps -> Tasks language to describe the Guest Dashboard (see the "Key Decisions" section). In the diagram provided by @erikguntner's, Task should be "Step" and Sub_Task should be "Task" to align with the story's description of the Guest Dashboard feature. If anyone has found it makes more sense to name these entities differently, loop @sanya301 in to iterate on the language we'll use to describe and implement the feature.
The Dashboard Application Model diagram provided by @Joshua-Douglas is a little bit out of scope for this feature but still eventually necessary. It would need to be modified to associate the Application model with a Task: A Dashboard contains Steps, Steps are a group of Tasks, Task has-a/is-a:
Is the above a correct representation of the Guest Dashboard model?
Hey @paulespinosa, How is the guest dashboard data model diagram out of scope for the guest dashboard endpoint issue? Do you think we should split the two issues and backlog this one, or do you think that changes to the data model are not strictly necessary?
@Joshua-Douglas specifically, the "HUU Application Dashboard Data Model" diagram. It contains models (Application
, Question
, ResponseType
, Response
, App_Question_Junction
), that are circled in the image below, that are more representative of an application form and are great candidate models for the issues: Guest Application Form #533 and Guest application implementation #610 and beyond.
Caveat: I've been using the terminology "Steps" and "Tasks" as defined in the main issue Guest Dashboard #500. However, it appears terminology has evolved from there, so I've pinged Sanya https://github.com/hackforla/HomeUniteUs/issues/500#issuecomment-1751054169 to motivate alignment on how the team describes the Dashboard features.
The Guest Dashboard #500 issue, the parent issue for this issue, Implement Guest Dashboard Endpoints #572, specifies displaying "high-level" information such as status, description, and progress about Steps. It's not asking to display the details of an application (form). When #500 talks about being able to "click into any additional page(s)", it means as a generic action where the details of those landing pages are unspecified and left for implementation in separate issues.
So the question becomes: how do you get the status and progress of a Step? Perhaps, by asking each of its Tasks for their status and progress. How do you get the status and progress of a Task representing, say, an Application Form? Consider, as an implementation option, defining Task as an interface or abstract base class. Have concrete Dashboard context representations of Tasks, such as an Application Form, contain the logic to calculate or get the status and progress. For this issue, the concrete implementations (or simply the Task for now) can return canned responses until specific issues are defined to implement their details (e.g. regarding Application Form, it can be an issue on its own or an action item for implementing the backend for Guest Application Form #533).
Unfortunately, the Action Items in this issue appears to be misaligned with its parent. Based on the context of #500, the action items circled above should probably read:
Regarding #500's Consideration: "In the future, the plan is for the guest onboarding experience for multiple organizations (SPY and beyond), where different organizations might want to modify the steps needed for a guest.". This word "modify" is under-specified and needs to be discussed with Product Management https://github.com/hackforla/HomeUniteUs/issues/500#issuecomment-1751054169.
According to this comment, we are doing the following naming convention:
"Task Groups has Tasks"
I think we can redefine the scope above to limit this issue to guest dashboard task groups and tasks, so we can remove the extra action items. We can use all the models above on issue #500 and develop them more further on their respective issues.
@Joshua-Douglas Take a look at the edits above on the design doc. Let me know if you want to meet to discuss our models further - the ERDiagram you posted puts into perspective the reorganizing that has to be done
cc: @tylerthome
Hi @paulespinosa - Are you able to look over my progress setting up models? I want to make sure Im understanding this and going in the right direction.
I tried my hand at what the models would look like so I could then work on the tests, domain objects, and domain logic.
As far as the statemachine (state chart pattern), we will have models.py
and statechart.py
that has the logic for this. At this time, I am planning on handrolling unless we agree to using a library.
models to be reviewed
##########################
# api/models/database.py #
##########################
import enum # adds python enum import
from sqlalchemy import create_engine
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session, declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey, DateTime, LargeBinary, Enum # adds enum
from sqlalchemy.orm import relationship # adds imports
Base = declarative_base()
# proposed models...
class TaskStatus(enum.Enum):
LOCKED = 'Locked'
START = 'Start'
IN_PROGRESS = 'In Progress'
MORE_INFORMATION_NEEDED = 'More Information Needed'
COMPLETE = 'Complete'
class TaskGroupStatus(enum.Enum):
LOCKED = 'Locked'
START = 'Start'
IN_PROGRESS = 'In Progress'
MORE_INFORMATION_NEEDED = 'More Information Needed'
COMPLETE = 'Complete'
class TaskGroupSubProcess(enum.Enum):
LOCKED = 'Locked'
APPLICATION_AND_ONBOARDING_PROCESS = 'Application & Onboarding Process'
HOST_MATCHING_PROCESS = 'Host Matching Process'
MATCH_FINALIZATION_PROCESS = 'Match Finalization Process'
class Guest(Base):
__tablename__ = 'guest'
id = Column(Integer, primary_key=True, index=True)
email = Column(String, ForeignKey('user.email'), nullable=False)
tasks = relationship("Task", back_populates="guest", cascade="all, delete")
task_groups = relationship("TaskGroup", back_populates="guest", cascade="all, delete")
guest_workflows = relationship("GuestWorkflow", back_populates="guest", cascade="all, delete")
class Task(Base):
__tablename__ = 'task'
id = Column(Integer, primary_key=True, index=True)
title = Column(String, nullable=False)
description = Column(String, nullable=False)
task_group_id = Column(Integer, ForeignKey('task_group.id'))
status = Column(Enum(TaskStatus), default=TaskStatus.LOCKED, nullable=False) # from finite state machine (statechart pattern)
guest_id = Column(Integer, ForeignKey('guest.id', ondelete='CASCADE'))
# methods for state machine (statechart pattern)
class TaskGroup(Base):
__tablename__ = 'task_group'
id = Column(Integer, primary_key=True, index=True)
name = Column(Enum(TaskGroupSubProcess), default=TaskGroupSubProcess.LOCKED, nullable=False) # from finite state machine (statechart pattern)
guest_workflow_id = Column(Integer, ForeignKey('guest_workflow.id'))
status = Column(Enum(TaskGroupStatus), default=TaskGroupStatus.LOCKED,nullable=False) # from finite state machine (statechart pattern)
guest_id = Column(Integer, ForeignKey('guest.id', ondelete='CASCADE'))
guest = relationship("Guest", back_populates="task_groups")
tasks = relationship("Task", back_populates="task_groups")
# methods for state machine (statechart pattern)
class GuestWorkflow(Base):
__tablename__ = 'guest_workflow'
id = Column(Integer, primary_key=True, index=True)
guest_id = Column(Integer, ForeignKey('guest.id', ondelete='CASCADE'))
task_groups = relationship('TaskGroup', back_populates='guest_workflow')
guest = relationship("Guest", back_populates="guest_workflows")
# existing code..
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True, index=True)
email = Column(String, nullable=False, unique=True)
class ApplicantType(Base):
__tablename__ = "applicant_type"
id = Column(Integer, primary_key=True, index=True)
applicant_type_description = Column(String, nullable=False)
class Applicant(Base):
__tablename__ = "applicant"
id = Column(Integer, primary_key=True, index=True)
applicant_type = Column(Integer, ForeignKey('applicant_type.id'), nullable=False)
applicant_status = Column(Integer, ForeignKey('applicant_status.id'), nullable=False)
user = Column(Integer, ForeignKey('user.id'), nullable=False)
# rest of models here...
class DataAccessLayer:
_engine: Engine = None
@classmethod
def db_init(cls, conn_string):
cls._engine = create_engine(conn_string, echo=True, future=True)
Base.metadata.create_all(bind=cls._engine)
@classmethod
def session(cls) -> Session:
return Session(cls._engine)
Hi @agosmou PM has been working on statuses in a Google Doc at https://docs.google.com/document/d/1mksBNqE9hc-bAW49mdHQDImkk8f92YewtwENNpqvFHc/edit. We (devs) will need to work together with PM on defining the statuses. As of this writing, it looks like the statuses in the Google Doc are "user statuses". It's not yet clear how the relation between "User status" and "Task status" will work.
I think the Guest should not contain references to its Tasks or TaskGroups; they should be obtained via the GuestWorkflow or similar. With respect to Guest holding a reference to the GuestWorkflow, that can be done or the GuestWorkflow can be obtained via another mechanism; we won't really know which suits us until we start working with the code and gain more clarity on the domain.
TaskStatus
, TaskGroupStatus
, TaskGroupSubProcess
are ok to hard code for starters but will need to be stored in the DB.
Try hand rolling a basic state machine to get a feel for it and gain better clarity about our needs. Thank you.
cc: @tylerthome
Thanks for this info, @paulespinosa ! Given the 'Host' items also take status, it'd be good to keep this in mind to make the code reusable for other endpoints.
Below Iadjusted the models and made a quick run at a state machine
#models/database.py
# proposed models...
class TaskStatus(enum.Enum):
LOCKED = "Locked"
START = "Start"
IN_PROGRESS = "In Progress"
MORE_INFORMATION_NEEDED = "More Information Needed"
COMPLETE = "Complete"
class TaskGroupStatus(enum.Enum):
LOCKED = "Locked"
START = "Start"
IN_PROGRESS = "In Progress"
MORE_INFORMATION_NEEDED = "More Information Needed"
COMPLETE = "Complete"
class TaskGroupSubProcess(enum.Enum):
LOCKED = "Locked"
APPLICATION_AND_ONBOARDING_PROCESS = "Application & Onboarding Process"
HOST_MATCHING_PROCESS = "Host Matching Process"
MATCH_FINALIZATION_PROCESS = "Match Finalization Process"
class Guest(Base):
__tablename__ = "guest"
id = Column(Integer, primary_key=True, index=True)
email = Column(String, ForeignKey("user.email"), nullable=False)
guest_workflows = relationship(
"GuestWorkflow", back_populates="guest", cascade="all, delete"
)
class Task(Base):
__tablename__ = "task"
id = Column(Integer, primary_key=True, index=True)
title = Column(String, nullable=False)
description = Column(String, nullable=False)
task_group_id = Column(Integer, ForeignKey("task_group.id"))
status = Column(
Enum(TaskStatus), default=TaskStatus.LOCKED, nullable=False
) # from finite state machine (statechart pattern)
# methods for state machine (statechart pattern)
pseudo-implementation of hand rolled state machine using state chart pattern
transitions
# models/state_machine.py
class TaskStateMachine:
def __init__(self, task_id, session: Session):
self.task = session.query(Task).get(task_id)
self.session = session
self.state = self.task.status
def transition(self, trigger):
match trigger:
case "start":
self.state = TaskStatus.START
case "progress":
self.state = TaskStatus.IN_PROGRESS
case "need_info":
self.state = TaskStatus.MORE_INFORMATION_NEEDED
case "complete":
self.state = TaskStatus.COMPLETE
print(f"Transitioning task to {self.state}")
self.task.status = self.state
self.session.commit()
class TaskGroupStateMachine: def init(self, task_group_id, session: Session): self.task_group = session.query(TaskGroup).get(task_group_id)
self.session = session
self.state = self.task_group.status
def transition(self, trigger):
match trigger:
case "start":
self.state = TaskGroupStatus.START
case "progress":
self.state = TaskGroupStatus.IN_PROGRESS
case "need_info":
self.state = TaskGroupStatus.MORE_INFORMATION_NEEDED
case "complete":
self.state = TaskGroupStatus.COMPLETE
print(f"Transitioning task group to {self.state}")
self.task_group.status = self.state
self.session.commit()
## Design
[Figma file](https://www.figma.com/file/BNWqZk8SHKbtN1nw8BB7VM/HUU-Everything-Figma-Nov-2022?type=design&node-id=1483-15519&mode=design&t=zmiPWI8j6AnuwXJW-0)
## UI Design
`~Ready for hand off - Section 2 guest dashboard`
![image](https://github.com/hackforla/HomeUniteUs/assets/73868258/6f12e775-122d-4c5b-b137-3ab6badd8237)
## Entities
`~Brainstorm - Sandbox`
![image](https://github.com/hackforla/HomeUniteUs/assets/73868258/bffbd24d-0d2b-44e3-9eba-fa42a3c9da0a)
This should be ice-boxed at least until existing MVP scope is nominally complete -- this is a requirement for compliance and traceability, but not strictly required for stakeholder demo
Overview
The API shall manage the HUU housing Workflow. API clients (e.g. the front-end application) should be able to view Tasks (name, description, status) assigned to Guests by the Workflow. The API clients should also allow Guests to navigate to Tasks to view and work on.
Tasks are an abstract concept and their concrete implementations represent such things as Forms, Training/Lessons, Scheduling, Matching, etc. The Tasks in the Workflow are logically grouped together into Task Groups. The Task Groups represent major sub-processes of the Workflow such as "Application & Onboarding process", "Host Matching process", "Match Finalization process", etc.
For this issue, all Task Groups and Tasks up until the matching process should be returned by the API.
Task Groups and Tasks go through phases represented by the following statuses:
The API shall maintain the following invariants imposed by the Workflow:
Action Items
Domain concerns:
The following data must be available in the endpoint(s):
Database concerns:
Resources/Instructions