Update models to reflect table definition doc

[rshewitt]

User Story

In order to use the most recent table definitions, datagov wants to update the harvest database tables to reflect what's in the table definition document

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

• [ ] GIVEN the existing db models in harvesting logic \ WHEN an update to the model happens \ THEN the new table(s) should exist as they are in the table definition file \ AND columns with foreign keys should not be nullable

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

[Notes or a checklist reflecting our understanding of the selected approach]

[rshewitt]

part of this ticket involves separating the harvest_error table into the harvest_job_error and harvest_record_error tables. the reason is to maintain referential integrity in the db between errors and the jobs or records which cause them. we have 3 routes involving harvest_errors.

• /harvest_error/
  • reads all harvest errors ( will be removed )
• /harvest_error/<error_id>
  • read a harvest error or read a harvest error by job
• /harvest_error/add
  • add a harvest error

How are new routes gonna look? ( error_type being "record" or "job" )

• /harvest_error/<error_type>/<error_id>
• /harvest_error/<error_type>/add

[rshewitt]

did some general fixes with the models like properly setting the declarative base class with our custom one

class Base(DeclarativeBase):
    __abstract__ = True  # Indicates that this class should not be created as a table
    id = db.Column(db.String(36), primary_key=True, default=lambda: str(uuid.uuid4()))

db = SQLAlchemy(model_class=Base)

and creating a base error class for jobs and record

class Error(db.Model):
    __abstract__ = True
    date_created = db.Column(db.DateTime, default=func.now())
    type = db.Column(db.String)
    message = db.Column(db.String)

class HarvestJobError(db.Model, Error):
    __tablename__ = "harvest_job_error"

    harvest_job_id = db.Column(
        db.String(36), db.ForeignKey("harvest_job.id"), nullable=False
    )

class HarvestRecordError(db.Model, Error):
    __tablename__ = "harvest_record_error"

    harvest_record_id = db.Column(
        db.String, db.ForeignKey("harvest_record.id"), nullable=False
    )

[rshewitt]

here's the new error routes

/harvest_job/<job_id>/errors /harvest_job/<job_id>/error/<job_error_id> /harvest_record/<record_id>/error/<record_error_id> # paginate this ( use 100 for now )

we're also removing the/add route