search

astronomy-commons / hipscat-import

HiPSCat import - generate HiPSCat-partitioned catalogs
https://hipscat-import.readthedocs.io
BSD 3-Clause "New" or "Revised" License
5 stars 3 forks source link

Improve CSV reading kwargs #245

Closed delucchi-cmu closed 5 months ago

delucchi-cmu commented 6 months ago

Change Description

Improve handling of kwargs for reading parquet schema (to use for column names/types), and for reading CSV file.

codecov[bot] commented 6 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.41%. Comparing base (97562d0) to head (abb7652).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #245 +/- ## ========================================== - Coverage 99.75% 99.41% -0.34% ========================================== Files 24 24 Lines 1204 1201 -3 ========================================== - Hits 1201 1194 -7 - Misses 3 7 +4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

hombit commented 6 months ago

I like this PR, but actually #231 was not only about the case when we use schema_file, but primary about type derivation, see the code snippet: https://github.com/astronomy-commons/hipscat-import/issues/231#issuecomment-1957752200