azavea / noaa-hydro-data

NOAA Phase 2 Hydrological Data Processing
11 stars 3 forks source link

Added parquet benchmarks for features per day #78

Closed vlulla closed 2 years ago

vlulla commented 2 years ago

Now we have same queries for zarr and parquet.

Overview

The parquet and zarr benchmarks now have the same queries over the same time ranges.

Checklist

vlulla commented 2 years ago

Hmmm...I copy/pasted into a new ipynb only the necessary input blocks (1, 2, 3, 4, 7, 8, and 14) and the results dataframe (from block 14) states that there were only 1787 nb_reaches (which corresponds to nb_reaches in the zarr queries). What am I missing?

lewfish commented 2 years ago

Hmmm...I copy/pasted into a new ipynb only the necessary input blocks (1, 2, 3, 4, 7, 8, and 14) and the results dataframe (from block 14) states that there were only 1787 nb_reaches (which corresponds to nb_reaches in the zarr queries). What am I missing?

I think that's the value it's supposed to have. What did you expect it to be?

vlulla commented 2 years ago

Oh I see. I have to remove the comment above the block! Will do...

vlulla commented 2 years ago

Had to do git commit --amend because my jupyter notebook has stopped working locally and it did not export the python script! So, manually fixed it manually.

vlulla commented 2 years ago

@lewfish I did a git rebase -i to squash multiple commits on my local branch and git now says that my local branch and origin have diverged. Is it alright to do a git push --force now?

Removed the extraneous verbiage from the first comment!

lewfish commented 2 years ago

Is it alright to do a git push --force now?

yes

vlulla commented 2 years ago

Fixed everything you suggested! Can you please review it?

vlulla commented 2 years ago

Squashed the two commits into one. Thanks for the patient review!