Update database ingestion workflow for existing v1 api

[manasaV3]

Motivation

As we are migrating the from v1 to v2 api, we should update the db ingestion such that the v1 is stable while v2 is available for users to migrate to.

Definition of Done

1. The new tables and relationships are created in v1 system.
2. The db ingestion can be done for the new file structures

Tasks

Update DB ingestion to support the new tables and metadata to the existing tables.

Update all the file paths that have been updated

Ensure the ingestion is reading from the new file strucuture

Add new tables

New tables to be added for Alignment, PerSectionAlignmentParameters

Alignment {
    arrray[array[float]] affine_transformation_matrix
    string alignment_type
    integer deposition_id
    integer id
    bool is_canonical
    string local_alignment_file
    integer run_id
    float tilt_offset
    float volume_x_dimension
    float volume_y_dimension
    float volume_z_dimension
    float volume_x_offset
    float volume_y_offset
    float volume_z_offset
    float volume_x_rotation
}

PerSectionAlignmentParameters {
    integer z_index
    float x_offset
    float y_offset
    float in_plane_rotation
    float tilt_angle
    integer id
}

Add fields for existing tables

Tomograms {
    ...
    bool is_visualization_default
    bool is_portal_standard
    bool is_author_submitted
    date deposition_date
    date last_modified_date
    date release_date
    integer alignment_id
    Alignment alignment
    string related_database_entries
    string publications
    ...
}

AnnotationFiles {
...
    integer alignment_id
    Alignment alignment
    bool is_standardized
...
}

Deprecate fields for existing tables

Tomograms {
    ...
    arrray[array[float]] affine_transformation_matrix
    ...
}

Relevant links

https://docs.google.com/document/d/1InjJKrx78Sv44QguMXR0p1o9Y34oPnMOY9J3aj375ks/edit#heading=h.eoh3rqmh58k2

[junxini]

@manasaV3 and @jgadling to decide who will take this ticket

[manasaV3]

We need to update, the tomograms table to have pairity with the new fields in v2 api. This includes

• Support for having multiple tomograms for a run
• Adding is_visualization_default
• Adding is_author_submitted
• Adding is_portal_standard