danzafar / tidyspark

tidyspark: a tidyverse implementation of SparkR built for simplicity, elegance, and ease of use.
Other
22 stars 0 forks source link

Write tests/polish read-write functions #30

Closed danzafar closed 4 years ago

danzafar commented 4 years ago

There are a lot of functions here. Need to make sure the docs are really great on these and they work with all the options.

Still need to implement:

Note: I did a lot of lifting to get JDBC functions working and tested with a local implementation of mysql and postgres, but since these tests will not run in CI I am limiting my dev work there and will pick it up when I'm setting up CI.

danzafar commented 4 years ago

with 5fca754 I added some nice write documentation. Very important.

danzafar commented 4 years ago

got all the functions, tests, and docs for reading and writing to files. Just need to finish up bucketing with saveAsTable, insertInto, and make a function that reads from hive table.

danzafar commented 4 years ago

I gave up on the bucketing feature in spark_write_table but got everything else going :)

danzafar commented 4 years ago

going to sleep on this and probably take a look tomorrow before merging