OpenWaterFoundation / owf-app-infomapper-ng

Open Water Foundation InfoMapper web application for menu-driven maps and visualizations, using Angular
GNU General Public License v3.0
1 stars 2 forks source link

Add time series table popup #280

Closed smalers closed 4 years ago

smalers commented 4 years ago

Now that the map layer attribute table can be viewed, it would be useful to add a tabular view for time series data. This is equivalent to the table view in TSTool. Getting something basic in place would allow us to continue refining features. This will also be an acid test to performance because time series can contain a large number of data points. Below are some considerations to keep in mind:

  1. The TSTool functionality has worked well. For example, a TSViewJFrame class is created any time a time series view is started, using an initial view, such as graph or table. If a table is then selected from the graph window using the Table button, it shares the data used by the graph. This groups a related set of windows. Pressing Graph on the table view just brings the original graph to the foreground. Overall, this design works well. Although it has not been done in TSTool, it would be possible to link data scrolling and selections in a table to the graph.
  2. Additionally, windows are tracked in an application using a WindowManager. This allows the application to, for example, provide a tool to close all windows, such as when a many graphs have been updated. This may be necessary if popups are somehow hidden because they have been dragged to the edge.
  3. The table view/window is intelligent enough to display time series grouped by interval. For example, all monthly data are shown in one table with dates aligned. Otherwise, it is necessary to insert blank cells to make the tables align for different intervals and that is complicated and confusing. The TSViewTableJFrame class handles the layout. In most cases, data have the same interval and are shown in one table. Implementing something similar in InfoMapper could start with the main intervals (month, year, day) and then add shorter interval (hour, minute) and irregular.
  4. To achieve fast performance and not duplicate memory space, the TSViewTable_TableModel class acts as the go-between the Java time series (TS class) and viewing component, providing an MVC design. There are also some optimizations to increase performance, like caching values and remembering the last request time.
  5. The data types for table columns make sense for the time series including date/time, double for value, and string for flag.
  6. The TSTool time series table allows turning on data flags. This causes the data column to be changed from floating point to string, with flag shown as ^ carret as a superscript. It may be possible to do this nicer in HTML. Adding a separate column for flag may be OK but expands the table if multiple time series are shown
  7. A right-click menu on the table allows copy/paste, which is useful to copy into Excel for additional calculations. Need to figure out if copy paste will work with InfoMapper, such as Ctrl-a to select all.
  8. Being able to save the table to a file will also be useful. The save capability is one of the final fundamental features to be implemented.
Nightsphere commented 4 years ago

The push has been made with the updates to display the time series data table. I will keep this open until more testing has been done.

smalers commented 4 years ago

Here are general comments on the initial implementation. I'm mostly concerned about obvious visual issues and basic conventions. I think csv issues won't be resolved until csv metadata are implemented and I have started a separate issue to focus on CSV. The following issues are related to DateValue files, which is the most robust implementation. I updated the Poudre application to use DateValue files for County population.

  1. In the graph window, move the Data Table button next to Close. Placing buttons on the lower right seems to be a best practice but does vary. There are style guides for order of buttons for languages and operating systems and they change over time. At a minimum, put the buttons next to each other. The buttons are now next to each other but seem to be a bit far apart. Review other UIs to see what an appropriate spacing is.
  2. Introducing an artificial comma in time series identifier is problematic.
    1. The TSID (or alias) should not be manipulated in the Graph window or elsewhere if possible because it corresponds to a unique identifier. Don't insert comma or other characters. Show as is. For DateValue there are properties that can be used.
    2. For time series table heading, use the TSAlias or TSID as is similar to TSTool. Are there reasons why this won't work? Look at TSTool tables as an example and we can discuss issues. Is it because the column width is too wide? Need to figure this out. If necessary, additional properties can be saved to the time series graph configuration file.
  3. Units for DateValue should be displayed in parenthesis in the column header: ... (Units). This is a simple standard that can be used in other places as well, to show units, such as graph axes. If no units, don't include the parentheses.
  4. Has horizontal scroll been tested for wide table?
  5. Graph mouse-over data points truncates the legend. Maybe this is OK A long label would have its own issues. This is just an observation. Keep an eye on this as we implement more complex graphs. The main issue is if the displayed information is not unique. Currently the color can be cross-referenced to color in the legend.
  6. Need to figure out how to set the precision. For population it should be formatted with no digits after the decimal (format as an integer). Again, csv is limited and hopefully DateValue handles. See the Java DataUnits class. I will add an issue to implement a data type file for defaults.
  7. The label for the first column should be DATE if day, month, year precision and DATE/TIME if hour, minute or finer precision. This is consistent with TSTool.
  8. The date/time column values need to be formatted according to the precision of the time series. Use ISO 8601 without the T and without time zone. The DateTime class used with TS ported from Java handles properly.
  9. The Larimer county population data file ends in 2035 because that is from the original data source. The graph should stop in 2935. The DateValue file ends in 2035 and this seems OK. However, the legend is in the data area and obscures data. Is it possible to place the legend outside of the graph drawing area? TSTool has options to control and ideally those properties are used to position the Plotly legend. There are cases where the legend may have many time series.
  10. The rows should be not too high. See TSTool and Excel. Remove whitespace if possible. All cells should NOT wrap since the data values should generally fit. There may be an issue if flags are added. I'll add a separate issue.

In addition to the above, next, need to figure out a Save button and functionality. See other issue related to this.

Nightsphere commented 4 years ago

The new push has been merged and has taken care of the feedback from the previous comment:

  1. The buttons on the bottom have been placed together on the right, and spaced closer together as well
  2. The InfoMapper will attempt to use the TSAlias first, since it should be shorter than then full TSID, and if the TSAlias is not given, then it will use the TSID Location.
  3. See 2
  4. See 2
  5. If units are given, they will be shown in parentheses, and left out if not given.
  6. I will test this soon.
  7. I will look into this later as suggested.
  8. This will be the next issue I look into.
  9. DATE and DATE / TIME have been implemented and shown according to the time series precision
  10. I have some questions regarding this I will ask Steve.
  11. The legend will use the same options as TSTool, and show it using the following properties: Bottom, BottomLeft, BottomRight, Left, Right, InsideLowerLeft, InsideLowerRight, InsideUpperLeft, InsideUpperRight.
  12. I will look into this after setting the precision of data values.
smalers commented 4 years ago

As to #5...The table for population is showing units of Total Population. This the LeftYAxisTitleString. The code needs to not confuse this process with time series units. Units may sometimes be used for the Y axis label as a default, but the assumption in the other direction is not valid. I don't see in the code where units are used as a default so this may just be a memory.

I did look at the StateMod dataset viewer and the column heading is good so there must be something going on with the population graph configuration.

Actually, the plot configuration file has a property LeftYAxisUnits that can be used to label the axis as follows, omitting the units if blank or not specified:

LeftYAxisTitleString (LeftYAxisUnits)

As for the time series table column, because the table column is specific to a time series, the units should be taken from the time series object (not graph configuration proper).

smalers commented 4 years ago

To format the graph legend, use code similar to the following, which is taken from Java:

ts.formatLegend(ts.getLegend());

The above allows a default of Auto. The legend shown in TSTool is maybe too verbose, but using the same logic will provide flexibility to configure the time series legend in the configuration file.

As for the table column heading, see the section "Time Series Table View" in TSTool documentation. Which states:

Column headings by default indicate alias if set (or location otherwise), sequence number (used with ensembles), data type and units. If the TableViewHeaderFormat time series property is set, it will be used to format the header. The format can contain % specifiers and ${ts:Property} properties.

See the Java code TSViewTable_TableModel getColumnName method. The main change I would make is to default to include units at the end with (units) as discussed elsewhere. This was not done in TSTool but I think would clarify interpretation of the table column.

Nightsphere commented 4 years ago

The core functionality of this issue has been completed. Closing the issue.