Copying large data with splaklyr?

gettyimages / docker-spark

Docker build for Apache Spark

MIT License

679 stars 369 forks source link

Copying large data with splaklyr? #53

Open ignacio82 opened 5 years ago

ignacio82 commented 5 years ago

I'm trying to learn more about spark and to do so I installed it using this docker-compose.yml. I was able to copy a small table to it but I cannot copy a large table. I'm not sure how to debug this problem, nor if this is an appropriate place to ask for help. But in any case, if someone here can help me I would really appreciate it.

OneCricketeer commented 4 years ago

Curious if you tried the same with Spark outside of Docker, The stacktrace there indicates an error within Sparklyr array access, not this container