search

informatics-isi-edu / deriva-webapps

Deriva-based web applications
Apache License 2.0
2 stars 1 forks source link

Plot app: Configurable User Controls functionality in plot-config #156

Open jrchudy opened 1 year ago

jrchudy commented 1 year ago

Currently only the violin plot has selectors that show above the plot that are hardcoded. If violin plot wants to be used for a different use case, some of these user controls might not make sense. We also have a use case to add a user control specifically for the heatmap plot type that allows for the column that is used for aggregation to be changed so the plot can show different "z values" for the same set of x/y data (issue #144 ).

The following properties are how we will add support for configurable user controls in plot-config:

layout: [
  { layout },
],
user_controls: [
  { control }
],
grid_layout_config: {},
plots: [{
  uid: 'plot-id',
  plotly: {
    layout: { ... },
    config: { ... }
  },
  config: {
    xaxis: {
      type_pattern: '...'
    },
    ... 
  },
  user_controls: [
    { control },
    { control },
    ...
  ],
  layout*: [
      { layout },
      { layout },
      ...
  ]
  grid_layout_config: {
    width?:              number - width in pixels, not needed if using responsive grid layout component,
    auto_size?:          boolean - height grows/shrinks to fit content,
    position?:           string - where the grid of selectors will show ('top' | 'bottom')
    breakpoints?:        object - map to identify when a different layout configuration should be used based on grid size,
    cols?:               string | object - number of columns to show, object used for breakpoints,
    margin?:             array of numbers | object - margin between grid components in pixels,
    container_padding?:  array of numbers | object - padding inside of each grid component in pixels,
    row_height?:         number - height of each row in pixels,
    is_draggable?:       boolean - controls ability to move all components,
    is_resizable?:       boolean - controls ability to resize all components
  },
  traces: [{
    url_pattern: '...',
    x_col_pattern: [ '...' ],
    y_col_pattern: [ '...' ],
    z_col_pattern: [ '...' ]
  }, ... ]
}]

Each individual control will be configured with the following:

* = required
? = optional

user_controls: [{
  uid*:                   string - key for referencing this user control; in other configuration properties,
  label*:                 string - name to display next to this user control,
  type*:                  string - the type of control to display, more info below about control types,
  url_param_key?:         string | object - parameter name or object of parameter names from url params that ire associated with this user control,
  request_info?: {
    url_pattern?:            string - string that allows for handlebars templating for fetching data for this user control,
    data?:                   array of objects - values to use in the user control if no query_pattern is provided, 
    default_values?:         string | array of strings  - the initial value(s) to use for this user control on page load,
    value_key*:              string - column to use for templating in queries,
    selected_value_pattern*: string - a pattern string to show in the user control for each selected option
    tick_markdown_pattern*:  string - a markdown pattern to be used for the axis labels. `control.axis`
    wait_for?:               string | array of strings - other user control request(s) that this control relies on
  }
}, ...]

Each individual layout will be configured with the following:

layout: [{
    source_uid*:    string - corresponds to the component key (`uid`) from user_controls (or plots if global layouts),
    x*:             number - grid index in grid units for the x position, 
    y*:             number - grid index in grid units for the y position,
    w*:             number - number of grid units the width of the component spans, 
    h*:             number - number of grid units the height of the component spans,
    is_draggable?:  boolean - can the component be moved around, overrides `static`,
    is_resizable?:  boolean - can the component be resized, overrides `static`,
    static?:        boolean - if true, implies `isDraggable: false` and `isResizeable: false`,
    min_w?:         number - min width in grid units,
    max_w?:         number - max width in grid units,
    min_h?:         number - min height in grid units,
    max_h?:         number - max height in grid units
}, ... ]

Notes about configuration language:

General Notes:

jrchudy commented 1 year ago

For example, violin plot will have the following configuration for the current set of hardcoded user controls:

layout: [{
    component: 'gene',
    x: 0, y: 0, w: 1, h: 1,
    static: true
  }, {
    component: 'groupby',
    x: 1, y: 0, w: 1, h: 1,
    static: true
  }, {
    component: 'scale',
    x: 2, y: 0, w: 1, h: 1,
    static: true
  }, {
    component: 'study',
    x: 0, y: 1, w: 3, h: 3,
    static: true
}],
grid_layout_config: {
  auto_size: true,
  cols: 3,
  row_height: 30
},
user_controls: [{
      uid: 'gene',
      label: 'Gene',
      type: 'facet-search-popup',
      url_param_key: 'Gene',
      compact: true,
      request_info: {
        url_pattern: '/ermrest/catalog/2/entity/RNASeq:Replicate_Expression/{{#if (gt $control_values.study.length 0)}}{{#each $control_values.study}}Study={{{this.values.RID}}}{{#unless @last}};{{/unless}}{{/each}}/{{/if}}(NCBI_GeneID)=(Common:Gene:NCBI_GeneID)',
        value_key: 'NCBI_GeneID',
        selected_value_pattern: '{{{$self.values.NCBI_Symbol}}}'
      }
    }, {
      uid: 'groupby',
      label: 'Group By',
      type: 'dropdown',
      request_info: {
        data: [
          {
            Name: "Experiment"
            Display: "Experiment"
          }, {
            Name: 'Experiment_Internal_ID',
            Display: 'Experiment Internal ID'
          }, ... 
        ],
        default_values: 'Experiment',
        value_key: 'Name',
        selected_value_pattern: '{{{$self.values.Display}}}',
        tick_markdown_pattern: '{{{$self.values.Experiment}}}',
      }
    }, {
      uid: 'scale',
      label: 'Scale',
      type: 'dropdown',
      request_info: {
        data: [
          {
            Name: 'Linear'
          }, {
            Name: 'Log'
          }
        ],
        default_values: 'Linear',
        value_key: 'Name',
        selected_value_pattern: '{{{$self.values.Name}}}'
      }
    }, {
      uid: 'study',
      label: 'Study',
      type: 'button-facet-search-popup',
      url_param_key: 'Study',
      request_info: {
        url_pattern: '/ermrest/catalog/2/entity/RNASeq:Replicate_Expression/NCBI_GeneID={{{$control_values.gene.values.NCBI_GeneID}}}/(Study)=(RNASeq:Study:RID)',
        value_key: 'RID',
        selected_value_pattern: {{{$self.values.RID}}},
        wait_for: 'gene'
      }
    }
  }
},
config: {
  xaxis: {
    title_markdown_pattern: '{{{$control_values.groupby.values.Display}}}'
  },
  yaxis: {
    type_pattern: '{{{$control_values.scale.values.Name}}}'
  } 
},
traces: [{
  url_pattern: '/ermrest/catalog/2/attributegroup/M:=RNASeq:Replicate_Expression/{{#if (gt $control_values.study.length 0)}}({{#each $control_values.study}}Study={{{this.values.RID}}}{{#unless @last}};{{/unless}}{{/each}})&{{/if}}NCBI_GeneID={{{$control_values.gene.values.NCBI_GeneID}}}/$M/Anatomical_Source,Experiment,Experiment_Internal_ID,NCBI_GeneID,Replicate,Sex,Species,Specimen,Specimen_Type,Stage,Age,Starts_At,Ends_At,TPM'
  x_col_pattern: ['{{{$control_values.groupby.values.Name']
}]
jrchudy commented 1 year ago

Overview of selector types (one subset of user controls)

The types of selectors we need to support are the following (with more details about each one below):

facet-search-popup

The only property that needs to be defined a specific way is type, the rest are for configuring how the control behaves.

Notes:

button-facet-search-popup

The only property that needs to be defined a specific way is type, the rest are for configuring how the control behaves.

Notes:

dropdown

2 properties defined along with type include action and axis. These tell the app what action will be applied and to what axis when a selection is made in the dropdown

Notes:

simple-search-dropdown

This is the type of input that is used on the top left of the mouse matrix app. One way this input could work is to load all of the data from request_info.data or request_info.url_pattern into the list that the search box searches through. Upon selection, this could affect the way an axis is scaled, what data is used for grouping one of the axes (x, y, or z), or sending a request that fetches new data for the plot.

Notes:

checkbox-list

This input is meant to mirror what was designed for CFDE charts on the personal dashboard page. It can be used for data from a related table (foreign key) or for simpler sets of data that allow for multiple choices

{
  type: 'checkbox',
  uid,
  label,
  url_param_key,
  request_info: {
    url_pattern,
    data: [
      { value, label, default },
      ...
    ],
    default_values,
    value_key,
    selected_value_pattern,
    tick_markdown_pattern,
    wait_for
  }
}
image
jrchudy commented 1 year ago

RBK/gudmap host change selector

The following control is an example for changing the host for bar plot on Gudmap homepage to show gudmap or RBK data

plots: [{
plotly: {...},
layout: [{
    component: 'host',
    x: 2, y: 0, w: 1, h: 1,
    static: true
}],
grid_layout_config: {
  auto_size: true,
  cols: 3,
  row_height: 30
},
user_controls: [{
    uid: 'host',
    label: 'Host',
    type: 'dropdown',
    request_info: {
      url_pattern: "ermrest/catalog/2/entity/Common:Consortium",
      data: [{
        Name: "RBK",
        Description: "The host RBK",
        ...
      }, {
        Name: "Gudmap",
        Description: "..."
      }],
      default_values: "RBK",
      value_key: "Name",
      selected_value_pattern: "{{{$self.values.Name}}}: {{{$self.values.Description}}}",
    }
}],
traces: [{
  url_pattern: '/ermrest/catalog/2/entity/M:=Dashboard:Release_Status/Consortium={{{$control_values.host.values.Name}}}/!(%23_Released=0)/!(Data_Type=Antibody)/!(Data_Type::regexp::Study%7CExperiment%7CFile)'
}]
}, ... ]

date range selector

The following control is an example for changing the range of data for bar plot on Gudmap homepage to show all data that was created or released, or only data from the last year that was created or released

plots: [{
plotly: {...},
layout: [{
    component: 'range',
    x: 2, y: 0, w: 1, h: 1,
    static: true
}],
grid_layout_config: {
  auto_size: true,
  cols: 3,
  row_height: 30
},
user_controls: [{
    uid: 'range',
    label: 'Resource Range',
    type: 'dropdown',
    request_info: {
      data: [{
        Name: '#_Released',
        Display: '# Released' 
      }, {
        Name: '#_Created',
        Display: '# Created' 
      }, {
        Name: '#_Released_in_Latest_Year',
        Display: '# Released in Last Year' 
      }, {
        Name: '#_Created_in_Latest_Year',
        Display: '# Created in Last Year' 
      }],
      default_values: '#_Released',
      value_key: 'Name',
      selected_value_pattern: '{{{$self.values.Display}}}',
    }
}],
traces: [{
  url_pattern: '/ermrest/catalog/2/entity/M:=Dashboard:Release_Status/Consortium=GUDMAP/!(%23_Released=0)/!(Data_Type=Antibody)/!(Data_Type::regexp::Study%7CExperiment%7CFile)/$M@sort(ID::desc::)?limit=26',
  legend: ["#_Released"],   // name of traces in legend
  x_col_pattern: ['{{{$control_values.range.values.Name}}}'],
  y_col_pattern: ['Data_Type'],
  ...
}]
}, ... ]

Notes:

separate scale selector for multiple plots (local controls)

'gudmap-release-all': {
  plots: [{
    plotly: { ... },
    layout: [{
        component: 'scale1',
        x: 2, y: 0, w: 1, h: 1,
        static: true
    }],
    grid_layout_config: {
      auto_size: true,
      cols: 3,
      row_height: 30
    },
    user_controls: [{
        uid: 'scale1',
        label: 'Scale',
        type: 'dropdown',
        request_info: {
          data: [
            { Name: 'Linear' }, 
            { Name: 'Log' }
          ],
          default_values: 'Linear',
          value_key: 'Name',
          selected_value_pattern: '{{{$self.values.Name}}}'
        }
    }],
    config: {
      xaxis: {
        type_pattern: '{{{$control_values.scale1.values.Name}}}'
      }
    }
    traces: [{ ... }]
  }, {
    plotly: { ... },
    layout: [{
        component: 'scale2',
        x: 2, y: 0, w: 1, h: 1,
        static: true
    }],
    grid_layout_config: {
      auto_size: true,
      cols: 3,
      row_height: 30
    },
    user_controls: [{
        uid: 'scale2',
        label: 'Scale',
        type: 'dropdown',
        request_info: {
          data: [
            { Name: 'Linear' }, 
            { Name: 'Log' }
          ],
          default_values: 'Linear',
          value_key: 'Name',
          selected_value_pattern: '{{{$self.values.Name}}}'
        }
    }],
    config: {
      xaxis: {
        type_pattern: '{{{$control_values.scale2.values.Name}}}'
      }
    }
    traces: [{ ... }]
  }]
}

Notes:

'gudmap-release-all': {
  layout: [{
      component: 'scale',
      x: 2, y: 0, w: 1, h: 1,
      static: true
  }, {
      component: 'plot1',
      x: 0, y: 1, w: 3, h: 6,
      static: true
  }, {
      component: 'plot2',
      x: 0, y: 7, w: 3, h: 6,
      static: true
  }],
  grid_layout_config: {
    auto_size: true,
    cols: 3,
    row_height: 30
  },
  user_controls: [{
      uid: 'scale',
      label: 'Scale',
      type: 'dropdown',
      request_info: {
        data: [
          { Name: 'Linear' }, 
          { Name: 'Log' }
        ],
        default_values: 'Linear',
        value_key: 'Name',
        selected_value_pattern: '{{{$self.values.Name}}}'
      }
  }],
  plots: [{
    uid: 'plot1',
    plotly: { ... },
    config: {
      xaxis: {
        type_pattern: '{{{$control_values.scale.values.Name}}}'
      }
    }
    traces: [{ ... }]
  }, {
    uid: 'plot2',
    plotly: { ... },
    config: {
      xaxis: {
        type_pattern: '{{{$control_values.scale.values.Name}}}'
      }
    }
    traces: [{ ... }]
  }]
}

checkbox list of options (studies as an example)

plots: [{
plotly: {...},
layout: [{
    component: 'checkbox',
    x: 1, y: 0, w: 2, h: 2,
    static: true
}],
grid_layout_config: {
  auto_size: true,
  cols: 3,
  row_height: 30
},
user_controls: [{
    uid: 'checkbox',
    label: 'Study',
    type: 'checkbox',
    url_param_key: 'Study',
    request_info: {
      url_pattern: '/ermrest/catalog/2/entity/RNASeq:Replicate_Expression/NCBI_GeneID={{{$control_values.gene.values.NCBI_GeneID}}}/(Study)=(RNASeq:Study:RID)',
      value_key: 'RID',
      selected_value_pattern: {{{$self.values.RID}}},
      wait_for: 'gene'
    }
}],
traces: [{
  url_pattern: '...'
}]
}, ... ]

Notes:

jrchudy commented 1 year ago

From further investigation into vitessce, we determined they are using a library called ReactGridLayout to handle the responsiveness and grid like layout. We are going to use that same library to simplify what we are doing and provide a configuration language to pass directly to that component.

The list of grid layout props can be found at this link. The ones that we want to consider allowing data modelers to configure are the following:

The following properties are the ones that are defined on the objects in the layout property above:

Notes

jrchudy commented 1 year ago

Overview of input types (another subset of user controls)

The types of inputs that we need to support are the following (with more details about each below):

range-picker

{
  type: 'range-picker',
  uid,
  label,
  table_column,
  url_param_key
}

Notes:

Example configuration:

The following control is an example for changing the range of data for bar plot on Gudmap homepage to provide a different way to filter the data. Instead of based on specific column values being returned that have a summary of counts, this will filter the data based on the range over RCT column:

plots: [{
  plotly: {...},
  layout: [{
    component: 'rct-range',
    x: 0, y: 0, w: 2, h: 2,
    static: true
  }],
  grid_layout_config: {
    auto_size: true,
    cols: 2,
    row_height: 30
  },
  user_controls: [{
    uid: 'rct-range',
    type: 'range-picker',
    label: 'Filter by date created',
    table_column: 'RCT',
    url_param_key: {
      min: 'rct-min', 
      max: 'rct-max'
    }
  }],
  traces: [{
    url_pattern: '/ermrest/catalog/2/entity/M:=Dashboard:Release_Status/Consortium=GUDMAP/!(%23_Released=0)/!(Data_Type=Antibody)/!(Data_Type::regexp::Study%7CExperiment%7CFile)/$M@sort(ID::desc::)?limit=26',
  }]
}]

search

{
  type: 'search',
  uid,
  label,
  table_column
}

Example configuration

The following control is an example for filtering the content of the specimen table by doing string matching for any column in the specimen table

plots: [{
  plotly: {...},
  layout: [{
    component: 'search',
    x: 0, y: 0, w: 1, h: 1,
    static: true
  }],
  grid_layout_config: {
    auto_size: true,
    cols: 2,
    row_height: 30
  },
  user_controls: [{
    uid: 'search',
    type: 'search',
    label: 'Filter through search',
  }],
  traces: [{
    url_pattern: '/ermrest/catalog/2/attributegroup/M:=Gene_Expression:Specimen/stage:=left(Stage_ID)=(Vocabulary:Developmental_Stage:ID)/$M/Assay_Type,stage:Name',
  }]
}]

text

This can be a static or dynamic link that uses templating based on $url_parameters or $control_values. Instead of having a "label_markdown_pattern" and the displayed "control value" (link), the label will be used as the control_value.

{
  type: 'text',
  uid,
  label_markdown_pattern,
}

Example configuration

plots: [{
  plotly: {...},
  layout: [{
    component: 'link',
    x: 3, y: 0, w: 1, h: 1,
    static: true
  }],
  grid_layout_config: {
    auto_size: true,
    cols: 4,
    row_height: 30
  },
  user_controls: [{
    uid: 'link',
    type: 'text',
    label_markdown_pattern: '/chaise/record/#1/Common:Gene/RID={{{$control_values.gene.values.RID}}}',
  }, {
    uid: 'gene',
    label: 'Gene',
    type: 'facet-search-popup',
    request_info: {
      url_pattern: '/ermrest/catalog/2/entity/RNASeq:Replicate_Expression/(NCBI_GeneID)=(Common:Gene:NCBI_GeneID)',
      default_value: '11669',
      value_key: 'NCBI_GeneID',
      selected_value_pattern: '{{{$self.values.NCBI_Symbol}}}'
    }
  }],
  traces: [{
    url_pattern: '...'
  }]
}]
jrchudy commented 1 month ago

List of what is implemented that is described in this issue:

grid layout configuration

Based on what is said in the original body of this issue

layouts

Based on what is said in the original body of this issue

user controls

For each user control, following what was defined in the original body of this issue