Use Glob in PUT file paths and Allow for Multi File Upload

andrusha / snowflake-rs

Snowflake API libraries for Rust

Apache License 2.0

31 stars 26 forks source link

Use Glob in PUT file paths and Allow for Multi File Upload #13

Closed colin99d closed 8 months ago

colin99d commented 10 months ago

This is a work in progress, I would be very thankful for any feedback.

By adding the GLOB package, file paths like the one below are now allowed: api.exec(&format!("PUT file://{} @%OSCAR_AGE_MALE;", "/Users/colin/test-snowflake/data/*.csv"))

This PR uses Tokio tasks to manage spawn a thread for each individual file.

This PR also moves libraries only required for the PUT command into the file feature

colin99d commented 9 months ago

FYI: Until the limit to concurrency is implemented this PR is not ready for review. I am more than happy for your comments. But please know some of this code is stuff I plan to clean on my own (right now this is all being coded while I am on planes and busses).

colin99d commented 9 months ago

@andrusha besides the two questions I had this PR is good to go. I decided to create a way for the user to define the file size and concurrency thresholds. Let me know your thoughts!

colin99d commented 8 months ago

lgtm

Спасибо!