Closed tzolov closed 8 years ago
Thanks, it sucks that we try to keep same plugin for all ambari versions but then they just rename python methods. I wonder if we should make a tweak to resolve if HdfsDirectory or HdfsResource should be used.
@trisberg do you have comments on this?
ah, ok this PR handles both HdfsDirectory and HdfsResource. Nice!
Spring XD currently supports HDP 2.2 so maybe we should wait with this PR until we update XD to support HDP 2.3
@jvalkeal , indeed i've tried to handle the schizmatic HdfsDirectory/HdfsResource change. I've used the the python function to smoothen the transition and keep backward compatibility. I've tested it on small HDP2.3 cluster and it seems to be working. I've tested both manual (Ambari Wizard) and Blueprint API installations. Here is my Blueprint. Note that current implementation uses the Stack version to decide which Hdfs API to use. This is suboptimal as the change is Ambari related (e.g from 2.0 to 2.1 - AMBARI-11034) and should be Stack agnostic.
@trisberg this is a valid point. It depends though how long it might take for this update to come. Perhaps in the meantime until the update happens we can map internally HDP2.3 to HDP2.2. Have not tested this yet.
@tzolov I'd rather not go there since we have customers that run a variety of workloads, very difficult to support unless it is officially supported by XD. The plan right now is for XD 1.3 to support HDP 2.3 and PHD 3.3 some time after Spring One which is in September.
Fair point
ok and kafka config breaks with hdp2.3, so it needs to be fixed first anyway.
closing this pr. @tzolov I picked related changes from this pr to fix for #5, thx mate!
@jvalkeal YW
produces a new rpm artefact:
spring-xd-plugin-hdp23-1.2-2.noarch.rpm
Summary:
spring-xd-plugin-hdp23
and build task:hdp23Rpm
HdfsDirectory
byHdfsResource
. I've modified to params.py and springxd.py to support the new API and still provide backaward compatibility