CODAIT / r4ml

Scalable R for Machine Learning
Apache License 2.0
42 stars 13 forks source link

[HYDRAR-206] adding ability to pass on arguments to sparkr.session #13

Closed iyounus closed 7 years ago

iyounus commented 7 years ago

This PR adds the ability to pass on arguments from r4ml.session to sparkr.session. This will allow user to set enableHiveSupport = FALSE. BTW, for sparkr.session, the defaults is enableHiveSupport = TRUE.

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
R4ML-CI commented 7 years ago

Build triggered.

R4ML-CI commented 7 years ago

Build success. All unit tests passed.

R4ML-CI commented 7 years ago

Build triggered.

R4ML-CI commented 7 years ago

Build success. All unit tests passed.

aloknsingh commented 7 years ago

@adrian555 can we test it just manually starting the session using enableHive... = FALSE and make sure there is not metastore dir created? I think i can't see any other way of doing it what do you think? please let us know

R4ML-CI commented 7 years ago

Build triggered.

iyounus commented 7 years ago

I just added another test for enableHiveSupport. The first test I added was to just have any test for zzz, and then plan was to add more tests in the future, because no tests existed for r4ml.session. For enableHiveSupport, sparkR adds spark.sql.catalogImplementation to config and set it to "hive". When we disable hive support, this variable no longer exists in the config. So, I've added a test accordingly.

R4ML-CI commented 7 years ago

Build success. All unit tests passed.

R4ML-CI commented 7 years ago

Build triggered.

R4ML-CI commented 7 years ago

Build success. All unit tests passed.