'Creating a Custom StreamSets Destination' tutorial problems

[JamesTreleaven]

I wish to create a 'StreamSets Destination' to interface with my firms software, so I am walking through the 'Creating a Custom StreamSets Destination' tutorial. When I get to the 'Extract the tarball to SDC’s user-libs directory, restart SDC, and you should see the sample stages in the stage library' step ... I don't see 'Sample' or 'sample' listed in the 'Select Destination to connect' dropdown.

Some details:

• I have downloaded and built version 3.7.1 of datacollector, datacollector-api, datacollector-plugin-api ... and pulled the latest version of datacollector-edge from git.
• I untarred the tarballs all into ~/work. I also 'created a new custom stage project using the Maven archetype' in ~/work creating the ~/work/samplestage directory.
• After I built datacollector, I did 'cp ~/work/samplestage/target/samplestage-1.0-SNAPSHOT.jar ~/work/datacollector/dist/target/streamsets-datacollector-3.7.1/streamsets-datacollector-3.7.1/user-libs' to copy the jar into the datacollector tree.
• I then ran datacollector with "~/work/datacollector/dist/target/streamsets-datacollector-3.7.1/streamsets-datacollector-3.7.1/bin/streamsets dc"
• I then logged into streamsets as 'admin/admin', created a new 'Data Collector Pipeline', performed 'Select Origin->Directory' to create an origin, then pulled down the 'Select Destination to connect' dropdown ... in which I could not find 'Sample' or 'sample'.

[JamesTreleaven]

Additionally, I am working on Ubuntu 18.04 with Java version 1.8.0_191.

[metadaddy]

Hi @JamesTreleaven - did you set archetypeVersion in the mvn archetype:generate command to 3.7.1?

[metadaddy]

I think I figured out your problem, @JamesTreleaven. You copied the jar file to ~/work/datacollector/dist/target/streamsets-datacollector-3.7.1/streamsets-datacollector-3.7.1/user-libs rather than extracting the tarball (as mentioned in the tutorial). The jar needs to be in a specific directory path under the user-libs directory for Data Collector to pick it up. You can fix this by doing:

mkdir -p ~/work/datacollector/dist/target/streamsets-datacollector-3.7.1/streamsets-datacollector-3.7.1/user-libs/samplestage/lib
mv ~/work/datacollector/dist/target/streamsets-datacollector-3.7.1/streamsets-datacollector-3.7.1/user-libs/samplestage-1.0-SNAPSHOT.jar ~/work/datacollector/dist/target/streamsets-datacollector-3.7.1/streamsets-datacollector-3.7.1/user-libs/samplestage/lib/samplestage-1.0-SNAPSHOT.jar

[JamesTreleaven]

Thank you very much @metadaddy, the fix you provided has solved my problem. My apologies - I should have followed the instructions more closely!

[metadaddy]

No worries, @JamesTreleaven!