search

cagov / caldata-mdsa-caltrans-pems

CalData's MDSA project with Caltrans on Performance Measurement System (PeMS) data
https://cagov.github.io/caldata-mdsa-caltrans-pems/
MIT License
7 stars 0 forks source link

Review of Poetry-based Virtual Environments #468

Closed YanlinQi-BIA closed 5 days ago

YanlinQi-BIA commented 2 weeks ago

Task Background

This PR implements updates to the Oracle puller and uploader components, focusing on managing dependencies and configurations in poetry.lock and pyproject.toml.

Order of Review

  1. pyproject.toml - Updated dependencies for Oracle puller.

  2. poetry.lock - Reflects dependency updates.

  3. oracle_puller/ - Contains main logic updates for the Oracle puller.

  4. uploader/ - Contains main logic updates for the uploader.

YanlinQi-BIA commented 1 week ago

This looks good to me. Per our discussion yesterday on writing parquet files as an intermediate format, does it make sense to also include pyarrow in the oracle_puller environment?

Certainly, will do.

YanlinQi-BIA commented 1 week ago

This looks good to me. Per our discussion yesterday on writing parquet files as an intermediate format, does it make sense to also include pyarrow in the oracle_puller environment?

Certainly, will do.

pyarrow has been added to the oracle_puller environment.