wp-xyz / corona

Plots and analyzes daily data of the Covid-19 pandemic
MIT License
20 stars 9 forks source link

Global stat? #37

Closed fredvs closed 3 years ago

fredvs commented 3 years ago

What a useful project! Surely the most clear and easy to use that I know.

Could it be possible to have also a global result? I mean for all the planet and by continent?

Thanks.

Fre;D

wp-xyz commented 3 years ago

A global result will soon be available in the work-in-progress version in branch "mapping" since it loads the entire JHU data set in contrast to the current master version.

JHU does not provide a list with the assigment of countries to continents, and I am too lazy to create one on my own. So sorry about that...

fredvs commented 3 years ago

Hello Wp.

A global result will soon be available in the work-in-progress version in branch "mapping" since it loads the entire JHU data set in contrast to the current master version

Ha, ok, I will follow it.

JHU does not provide a list with the assigment of countries to continents, and I am too lazy to create one on my own. So sorry about that...

No problem (but if you want I can take a look for it, just say me what you need).

fredvs commented 3 years ago

Here list of countries by continent in csv form: country-and-continent-codes-list-csv_csv.csv.zip

wp-xyz commented 3 years ago

Thanks.

wp-xyz commented 3 years ago

The current version in the mapping branch now contains a top-level "world" node plus "continent" nodes, the countries have been assigned to the corresponding continent nodes according to your list. Thanks again.

fredvs commented 3 years ago

Congrats!

I will deeply look at it tonight.

Write you later.

Fre;D

fredvs commented 3 years ago

About the world-map and selection of countries with mouse.

What widget are you using for this (works perfectly)?

wp-xyz commented 3 years ago

In TAChart there is a new "polygon series" which I copied and renamed into the corona sources so that the project can be compiled also with the release version. These polygons represent the features in the map (countries, states etc). Polygons can consist of several parts or can contain holes - I wrote a wiki about it recently.

All the mouse handling is built into TAChart, there's not much to be done (except for tweaking the polygon series such that clicks inside the polygons are accepted, not only clicks on the polygon vertices.

The data for the polygons are provided by the TcGeoMap class which creates a reader for kml files. The structure is such that in principle also other geo formats (shp, geojson) can be integrated, but I did not go this far.

Maybe I'll add also maps for the continents. Depends whether I find decent files. The map files usually have a too fine resolution. For simplification I use the tool at https://mapshaper.org - this drops too-close polygon points; unlike other tools this one is clever enough to avoid generating gaps and overlaps between the countries. The re-saved, simplified maps are loaded into the application QGIS which I use only to write a kml file. The kml file sometimes must be edited manually, e.g. when the country names are not the same as in the JHU files. I also added a "GeoID" field which is unique to each country/state etc and is attached also to the polygon series. This way the series can be identifed by the GeoMap class.

I am thinking about adding the geo routines to TAChart, but I am not sure about the general structure. Therefore, this exercise in the corona application is important from this point of view.

fredvs commented 3 years ago

Wow.

Huh, are you the creator/contributor of TAChart?

(I did not yet deeply test new-corona app, I will do it tonight).

fredvs commented 3 years ago

Hello.

I try to compile corona-mapping with Linux 64 bit, fpc 3.2.0 and Lazarus 2.0.10 but get this error (see picture):

error-corona

wp-xyz commented 3 years ago

Oh yes. I usually work with trunk and only occasionally compile with other versions. I did not notice that TTreeView has many useful changes in trunk now. I added a class helper for TTreeNode to fix the issue for 2.0.10.

Yes, I am the current maintainer of TAChart.

fredvs commented 3 years ago

I added a class helper for TTreeNode to fix the issue for 2.0.10.

Yep, compiles + runs perfectly now. Super wow, impressive and very clean.

Huh, did you see the difference between continents (max y of rich vs others)? But this is a other discussion.

Yes, I am the current maintainer of TAChart.

Congrats, excellent widget.

Fre;D

fredvs commented 3 years ago

Huh, did you see the difference between continents (max y of rich vs others)?

Not really rich vs others, but Africa, Asia and Oceania do much better than the others. But, re-yes, it is a other discussion.

fredvs commented 3 years ago

Hello.

Perfect, deep test done, the earth-map works like charm, fluid and without problems. There are very few little details here on Linux that dont look ok, for example the legends that are too long and dont fit on the screen, also for some check-boxes.

Also the hint in the bottom chart is always visible (the timer dont hide it).

Capture d’écran_2021-03-12_01-36-58

Anyway, superb work.

Fre;D

fredvs commented 3 years ago

Last thing.

I see that Australia, China and some African countries are in white and the legend says white = (no data) Is it true that for all that countries there are no data (or not reliable)?

wp-xyz commented 3 years ago

There are very few little details here on Linux that dont look ok, for example the legends that are too long and dont fit on the screen, also for some check-boxes.

There should be splitters near such space-critical things so that you can adjust the size of the corresponding control as needed. Splitter positions are saved in the ini file so that you will be annoyed only once.

BTW it's painfully hard to write a self-layouting program which is correct for every theme and every widgetset even when anchoring and autosizing are active...

There is an issue with the y axis title of the time-series chart which is too long when also the map is displayed. I am planning to use a word-wrap function to split the title into several lines, depending on the space available

Also the hint in the bottom chart is always visible (the timer dont hide it).

I am not sure about the chart hints, I think they are created by the application but by the chart and do not follow the conventions all the time. I added the hints to give the user an idea how to use the hidden parts of the program which do not have a place in the menu or toolbar. If you find the hints annoying you can always turn them off in the "Configuration" menu (not working at the moment for the map hints, but this will follow - or I'll find another place for the country info hints)

see that Australia, China and some African countries are in white and the legend says white = (no data) Is it true that for all that countries there are no data (or not reliable)?

I have a problem when the JHU files report "0" cases: does it mean "no data" or "zero value"? I decided to name it "no data" and have in mind that "no" could also mean "zero".

But anyway, Australia and China did have non-zero values in the past. Did you notice the scrollbar underneath the map? It allows you to scroll back into the corona history. When you scroll back China about one year you'll see province Hubei getting a green color - which is a ridiculously low incidence compared to our current values. Do you remember the dramatic reports in these old days? When you select China/Hubei in the tree for the time-series chart and activate the "Date indicator" you see a purple line for the date displayed in the map.


There was another question by you about the world data not being the sum of the continent values, but I can't find it any more (I hate the confusing notification system of github...). Well, the "new cases" and "cumulative cases" in the tree nodes for which JHU does not provide values were calculated exactly as the sum of the child node values, so this should be correct. But there are also the "normalized new cases" and these, in fact, do not add up. Here the new cases are are divided by the population of that region and multiplied by 100,000 -- i.e. this is the number of new cases normalized to a population of 100,000. And these numbers are not additive: a/b + c/d is not the same as (a+c)/(b+d). Suppose a region A having 100,000 inhabitants and 1 new case - the incidence (normalized new cases) is 1 here. In another region, B, there is also 1 new case, but there are only 1000 inhabitants -- the incidence is 100 (1/1000*100000). The naive sum for the combined region, A + B, is 1+100 but the total incidence of both regions would be 2 new cases for 100,000 + 1,000 = 101,000 inhabitants which corresponds to an incidence value slightly below 2.

fredvs commented 3 years ago

BTW it's painfully hard to write a self-layouting program which is correct for every theme and every widgetset even when anchoring and autosizing are active...

Yes I know, same for me with MSEgui/fpGUI (and in my last projects I dont use anchor anymore for some widgets but code adapted for each OS).

If you find the hints annoying you can always turn them off in the "Configuration" menu

Ooops, indeed, I did not see it, so ok, perfect.

There was another question by you about the world data not being the sum of the continent values, but I can't find it any more

Huh, yes, I removed it after. If you take a look at my previous image, Time series data is set to "Normalized cumulative" "Deaths" so I was thinking that the global stat was for this too. But after I realized that it is the ratio deaths/100k inhabitants, so all is ok.

Fre;D

fredvs commented 3 years ago

Your new geochart is very impressive. And it ask lot of things too.

For example, all that covid-deaths, what represent it as percentage of total deaths (all causes) in same period?

But, yes, I agree, there are lot of things to compare and a never-ended "I want more".

Anyway, nice project.

wp-xyz commented 3 years ago

If you take a look at my previous image, Time series data is set to "Normalized cumulative" "Deaths" so I was thinking that the global stat was for this too.

This is a problem with the current gui. I think it is not very intuitive at first glance that the upper groupbox refers to the map and the lower groupbox refers to the time series. I did try to move them into the chart panels, but this looks unbalanced. And there's a problem also with the treeview since it primarily determines which time series is shown, but it has also an effect on which map is seen.

wp-xyz commented 3 years ago

For example, all that covid-deaths, what represent it as percentage of total deaths (all causes) in same period?

I think it would also be interesting to plot hospitality rate, count of intensive station beds available, and so on... Unfortunately the Johns Hopkins University files do not contain this information.

fredvs commented 3 years ago

If you take a look at my previous image, Time series data is set to "Normalized cumulative" "Deaths" so I was thinking that the global stat was for this too.

This is a problem with the current gui. I think it is not very intuitive at first glance that the upper groupbox refers to the map and the lower groupbox refers to the time series. I did try to move them into the chart panels, but this looks unbalanced. And there's a problem also with the treeview since it primarily determines which time series is shown, but it has also an effect on which map is seen.

Imho, the "Map Data" combo-box could be inside the map tab. And maybe no more left panel but only the tabwidget in all the page and all what was in the left panel integrated into the time-serie panel.

Also, to reduce the length of legend, maybe use 10k instead of 10,000 (but this is detail).

I think it would also be interesting to plot hospitality rate, count of intensive station beds available, and so on...

Yes and maybe also to plot the new births in same period, this to relativize things.

wp-xyz commented 3 years ago

Imho, the "Map Data" combo-box could be inside the map tab. And maybe no more left panel but only the tabwidget in all the page and all what was in the left panel integrated into the time-serie panel.

Normally this would be right, but the setup information for the map chart and the time series chart require a much different amount of space. When I do as you propose the treeview would not longer fit on the form when both charts are visible and the form height is 600 (that's my limit - I have VMs in which I cannot control the main window height, and then large forms are very difficult to handle)

An alternative would be to separate map and time series charts on different tabs of the pagecontrol. But I think it is very nice to have both of them on the same screen in particular when scrolling through the time history.

fredvs commented 3 years ago

Imho, the "Map Data" combo-box could be inside the map tab. And maybe no more left panel but only the tabwidget in all the page and all what was in the left panel integrated into the time-serie panel.

Normally this would be right, but the setup information for the map chart and the time series chart require a much different amount of space. When I do as you propose the treeview would not longer fit on the form when both charts are visible and the form height is 600 (that's my limit - I have VMs in which I cannot control the main window height, and then large forms are very difficult to handle)

An alternative would be to separate map and time series charts on different tabs of the pagecontrol. But I think it is very nice to have both of them on the same screen in particular when scrolling through the time history.

Yes you are right, now after playing a few with it, it is ok (apart maybe the length of legends and checklistbox) For the width-size of checklistbox, I know it is difficult to a adapt to the length of the biggest item. Maybe there is a auto-size in LCL checklistbox?

wp-xyz commented 3 years ago

apart maybe the length of [...] checklistbox

As I said there is a splitter at the left of the checklistbox and another splitter between the left parameter panel and the page control.

fredvs commented 3 years ago

As I said there is a splitter at the left of the checklistbox and another splitter between the left parameter panel and the page control.

Ha, so much simpler, so for the first load of corona, just set the position of the listchekbox-spliter to make listchekbox a few bigger for GTK2 vs win32. This to have a nice first look like this: Capture d’écran_2021-03-12_22-30-03

Ok, I let you in peace and congrats for this impressive project (that gives lot of interrogations). And your TAMapChart is wonderful.

Fre;D