search

reustle / covid19japan-data

https://data.covid19japan.com
MIT License
49 stars 16 forks source link

we just hit the 5 million cells limit !!! #79

Open ggouaillardet opened 3 years ago

ggouaillardet commented 3 years ago

We cannot add more cells otherwise it would be more than 5 million cells, and this is not supported.

I guess the easiest way to move forward is to create a new spreadsheet and have generate.js and friends merge both files.

Right now, we're stuck :-(

ggouaillardet commented 3 years ago

tagging @reustle @liquidx to make sure you got that blocker issue.

liquidx commented 3 years ago

Thanks, it's odd because I was able to update it but it was super slow ...

liquidx commented 3 years ago

I'm deleting some columns and pruning some cells, but maybe tonight I might change the format of the Osaka or Tokyo tabs to compress the information

liquidx commented 3 years ago

I just quickly pushed a change that I wanted to do anyway, but this forces me to do it right now.

In every sheet, I changed the blank column that comes after "Status" into "Count". The generate script will now use that number to calculate how many patients that row represents. So instead of adding 300 rows for 300 patients, you can add one single row and add "300" to the count column.

If we start using this format, I think we should be able to reduce the number of rows we are using. It means we will lose some ability to annotate each patient with additional information, but I think this should at least make the sheet more scalable and might make data entry a bit easier.

If we need to expand the patients back up for some prefectures, it should be straightforward to do.

liquidx commented 3 years ago
Screen Shot 2020-12-23 at 21 18 46

Here is an example of how it looks in the Osaka tab, which is where I'm starting to use this format

ggouaillardet commented 3 years ago

sounds great!

what about the Deceased patients? can we use the new 'Count' column? or do we still need one entry per patient?

liquidx commented 3 years ago

Right now I haven't tested it on deceased patients yet - I'm looking at testing that a bit before I think it's safe to use. Luckily deceased are less, so that's ok if right now it's still one per-row.

liquidx commented 3 years ago

I pushed a small fix to make it also work for deceased patients, so it looks like it's working now.

The spreadsheet also properly accounts for this format change.