refactor CI using provision-with-micromamba

[jGaboardi]

This PR updates the environment setup in unittests.yml to take advantage of provision-with-micromamba. It is based on the current CI schema in spaghetti where runtime is reduced by approx. half.

[jGaboardi]

@knaaptime Here you go as a follow up that will supercede #303.

[jGaboardi]

Got a weird error on my fork here

1 Run python geosnap/tests/_dl_data.py
2   python geosnap/tests/_dl_data.py
3  shell: C:\Program Files\Git\bin\bash.EXE -l {0}
4  env:
5    COMBO_DATA: 
6 /c/Users/runneradmin/.bash_profile: line 1: micromamba: command not found
7 Traceback (most recent call last):
8  File "geosnap/tests/_dl_data.py", line 2, in <module>
9    import wget
10 ModuleNotFoundError: No module named 'wget'
11 Error: Process completed with exit code 1.

[knaaptime]

Got a weird error on my fork here

there's an environmental variable set in the repo to download some private data from a masked url. probably just failing because yours isnt set

[knaaptime]

although it could be that wget is included with miniconda but not micromamba and needs to be installed separately?

[jGaboardi]

Got a weird error on my fork here

there's an environmental variable set in the repo to download some private data from a masked url. probably just failing because yours isnt set

Maybe that, but I also had made other mistakes 🙃

[jGaboardi]

there's an environmental variable set in the repo to download some private data from a masked url. probably just failing because yours isnt set

Is this maybe why I couldn't run tests locally?

[jGaboardi]

Getting further along now. Current failure during the notebook tests here:

69------------------ 
70riverside = Community.from_ltdb(msa_fips="40140") 
71------------------ 
72 
73--------------------------------------------------------------------------- 
74FileNotFoundError Traceback (most recent call last) 
75<ipython-input-1-71c3a92354ad> in <module> 
76----> 1 riverside = Community.from_ltdb(msa_fips="40140") 
77 
78~/work/geosnap/geosnap/geosnap/_community.py in from_ltdb(cls, state_fips, county_fips, msa_fips, fips, boundary, years) 
79 1518 else: 
80 1519 gdf = _from_db( 
81-> 1520 data=datasets.ltdb(), 
82 1521 state_fips=state_fips, 
83 1522 county_fips=county_fips, 
84 
85~/work/geosnap/geosnap/geosnap/_data.py in ltdb(self) 
86 384 """ 
87 385 try: 
88--> 386 return pd.read_parquet(pathlib.Path(data_dir, "ltdb.parquet")) 
89 387 except KeyError: 
90 388 print( 
91 
92~/micromamba/envs/test/lib/python3.7/site-packages/pandas/io/parquet.py in read_parquet(path, engine, columns, use_nullable_dtypes, **kwargs) 
93 458 impl = get_engine(engine) 
94 459 return impl.read( 
95--> 460 path, columns=columns, use_nullable_dtypes=use_nullable_dtypes, **kwargs 
96 461 ) 
97 
98~/micromamba/envs/test/lib/python3.7/site-packages/pandas/io/parquet.py in read(self, path, columns, use_nullable_dtypes, storage_options, **kwargs) 
99 216 kwargs.pop("filesystem", None), 
100 217 storage_options=storage_options, 
101--> 218 mode="rb", 
102 219 ) 
103 220 try: 
104 
105~/micromamba/envs/test/lib/python3.7/site-packages/pandas/io/parquet.py in _get_path_or_handle(path, fs, storage_options, mode, is_dir) 
106 80 # fsspec resources can also point to directories 
107 81 # this branch is used for example when reading from non-fsspec URLs 
108---> 82 handles = get_handle(path_or_handle, mode, is_text=False) 
109 83 fs = None 
110 84 path_or_handle = handles.handle 
111 
112~/micromamba/envs/test/lib/python3.7/site-packages/pandas/io/common.py in get_handle(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options) 
113 649 else: 
114 650 # Binary mode 
115--> 651 handle = open(handle, ioargs.mode) 
116 652 handles.append(handle) 
117 653 
118 
119FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/.local/share/geosnap/ltdb.parquet'

[jGaboardi]

@knaaptime Seems there is some issue getting the LTDB data in 02_creating_community_datasets.ipynb (which I am also having the same trouble with locally) with a FileNotFoundError being raised here. Thoughts?

[jGaboardi]

@knaaptime Any thoughts on getting this passing?

[knaaptime]

i think we could probably skip the notebook tests here. The line that's failing is trying to use ltdb data, which we have to download privately. For the main tests, that happens in this script, but i dont know why the data dont seem to be available to the next process. As an alternative, we can keep the notebook examples in a separate repo and have that test separately

[jGaboardi]

i think we could probably skip the notebook tests here. The line that's failing is trying to use ltdb data, which we have to download privately. For the main tests, that happens in this script, but i dont know why the data dont seem to be available to the next process. As an alternative, we can keep the notebook examples in a separate repo and have that test separately

OK. Well then I think this PR is good to go if you like it.

[codecov[bot]]

Codecov Report

Merging #304 (9345fde) into master (3d290e5) will increase coverage by 0.2%. The diff coverage is 100.0%.

@@           Coverage Diff            @@
##           master    #304     +/-   ##
========================================
+ Coverage    68.9%   69.1%   +0.2%     
========================================
  Files          14      14             
  Lines        1193    1195      +2     
========================================
+ Hits          822     826      +4     
+ Misses        371     369      -2     

Impacted Files	Coverage Δ
geosnap/_community.py	73.0% <100.0%> (+0.1%)	:arrow_up:
geosnap/analyze/analytics.py	85.9% <0.0%> (+1.0%)	:arrow_up: