djay / covidthailand

Thailand Covid testing and case data gathered and combined from various sources for others to download or view
126 stars 15 forks source link

have colab and/or github.dev env ready try #206

Open djay opened 2 years ago

djay commented 2 years ago

Would have all the data loaded so you can explore it with some example plots in a notebook or give you a starter development environment to add new plots or run pytest or add a scraper.

djay commented 2 years ago

@kahnwong is this something you could do?

kahnwong commented 2 years ago

tried out

# Tests Plots
df = scrape_and_combine()
save_tests_plots(df)

from covid_plot.py but got

---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/kahnwong/.pyenv/versions/3.8.6/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/Users/kahnwong/Git/covidthailand/covid_data_dash.py", line 123, in dash_daily
    assert df[df['Recovered'] == 0.0].empty
AssertionError
"""

The above exception was the direct cause of the following exception:

AssertionError                            Traceback (most recent call last)
/var/folders/mc/1bzxsp8x6x1dl12nd62j64s40000gn/T/ipykernel_90632/3364214411.py in <module>
      8 
      9 # Tests Plots
---> 10 df = scrape_and_combine()
     11 save_tests_plots(df)

~/Git/covidthailand/covid_data.py in scrape_and_combine()
    263         en_situation = en_situation.get()
    264 
--> 265         dash_daily = dash_daily.get()
    266         dash_ages = dash_ages.get()
    267         dash_by_province = dash_by_province.get()

~/.pyenv/versions/3.8.6/lib/python3.8/multiprocessing/pool.py in get(self, timeout)
    769             return self._value
    770         else:
--> 771             raise self._value
    772 
    773     def _set(self, i, obj):

AssertionError: 

I guess it's better to read from a static dump. Although the "release" link on the readme seems to be broken 🤔

djay commented 2 years ago

@kahnwong I think a notebook is more useful for helping with plotting rather than scraping so downloading the csv release makes sense as thats the fastest way. Scraping I think is best done using an IDE and via pytest? Otherwise you need to get the 1.5G docs download...

I tested the link on https://github.com/djay/covidthailand#running-just-plots-or-latest-files and it seems to work fine for me. Would it be something people open in colab or they have to check out and run locally?

kahnwong commented 2 years ago

I could do both: make a local nb but also make it available on colab too. Code drift should be minimal since it's mostly a wrapper function

kahnwong commented 2 years ago

pr: https://github.com/djay/covidthailand/pull/219

WIP