search

artefactual-labs / am

Development environment for Archivematica
https://archivematica.org
GNU Affero General Public License v3.0
10 stars 20 forks source link

Revised AM dashboard #88

Closed cole closed 4 years ago

cole commented 4 years ago

Cleaner layout with sections, made many graphs more accurate and clearer.

This needs to be reviewed with https://github.com/artefactual/archivematica/pull/1501.

cole commented 4 years ago

In the previous screenshot, note how some of the new charts are empty...

This is pretty odd. Note that these are all the new histogram charts. I see in the screenshot the loading indicator is active for all of them; I'm guessing they just never loaded? For any of them to work you'll need the archivematica updates in the dev/instrumentation-improvements branch.

The file format charts behave in an interesting way. I ran three demo transfers consecutively.

Is it possible your first AIP failed? Mine sometimes fail during indexing, and it looks like that. We only store file format data on AIP storage, currently.

Execution times show NaNs at the beginning Thanks, fixed.

Some chart axis seem like they've stored data before the instance was flushed (around 12:35). I understand it may be the nature of the charts, but compare for instance the timeline of the Bytes processedchart vs the orange (meaning SIP rejected) timeline of the Processing errors chart.

This is a setting in the Grafana chart options, on how to display nulls (as zero or null). I've updated it across the board to not display null values, for consistency.

Unzipped bag transfers appear as transfer_type in the Average number of files chart. And I think Zipped bag transfers don't even appear.

Oh, good catch. Sadly, zipped and unzipped bag transfers don't have a transfer type set in the database. I've added the "unknown" type for them (change in archivematica, not the Grafana dashboard).

This is more of a question than a problem: how should a blue bucket like this be interpreted?

I'm not sure...if you hover over it you can see the value. It looks like it's slightly about zero, which seems like it shouldn't happen.

The histograms aren't intended to give precise values so much as show where the activity is. I've revised the heatmaps visually, and rounded the values (since we're just looking at counts), which should stop that from happening again.

I couldn't populate the HTTP Errors chart. Maybe it's the _total suffix in the expression: django_http_exceptions_total_by_type_total ?

I think it just only populates in the case of an http error. Did you have any?

cole commented 4 years ago

Forgot to say thanks @replaceafill for the very helpful review!

replaceafill commented 4 years ago

I tested again and I can see the dashboard fully populated now. The color change helped me understand the heatmaps better :+1: I think the only odd thing remaining is the file format charts not displaying the first transfer.