Great work & publishing to Maven

[MrPowers]

Great work on this project. Looks really useful. I will help you get users ;)

Are you publishing this to Maven yet?

Once we get this lib in Maven, I can submit a README PR with an installation / quick start section that grabs the reader's attention and motivates them to start using the lib.

It might be cool to give this lib a single-word name - something like kyuubi would be pretty sweet ;)

Thanks for all your open source Spark contributions. You do a lot to help the community!

[yaooqinn]

Thanks, @MrPowers

It's not in the maven repo yet, if you are interested in this. I can add you as a maintainer of this repo. And we can start by naming it.

[MrPowers]

@yaooqinn - sounds like a good plan to me :)

[yaooqinn]

I have sent the invitation

[MrPowers]

@yaooqinn - what should we call the repo? Naruto, Ibara, looks like we have lots of good options. Anything short and that's easy to pronounce works for me!

[yaooqinn]

yea, I also like a short and clear name. Too many good options might be a bad thing :) to reach a final decision.

Let's create a list of candidates

• konoha - https://naruto.fandom.com/wiki/Konohagakure
• itachi https://naruto.fandom.com/wiki/Itachi_Uchiha
• sharingan https://naruto.fandom.com/wiki/Sharingan
• pending for your suggestions

[MrPowers]

Let's go with itachi - this guy looks awesome. I am going to have to watch this series ;)

[yaooqinn]

done 👍🏼

[MrPowers]

@yaooqinn - I haven't used Maven a lot, but am excited to try something different than SBT.

Should I just install Maven on my machine? Can you tell me the command to run the test suite? Thank you!

[yaooqinn]

Yeah, for now, you need to install maven locally first and run the command in the ci https://github.com/yaooqinn/itachi/blob/master/.github/workflows/ci.yml

BTW, I'm OK that we switch to SBT or something else, or support both build tools like spark

[yaooqinn]

Shall we move this work under a more public space for people to reach? like awesome-spark?

[yaooqinn]

cc @zero323 WDYT？

[MrPowers]

@yaooqinn - I publish my open source projects to Maven directly with the sbt-sonatype plugin, more details here. You'll need to do some work on your end if you'd like to publish, namely the Sonatype and GPG setup steps.

I can submit a pull request with the info that needs to be added to this repo if you'd like to publish via sbt-sonatype.

Here's an example of where you'll be able to find the JAR files in Maven once they're published.

[yaooqinn]

yes, thanks for your very detailed suggestion. I will check that soon @MrPowers

[zero323]

Shall we move this work under a more public space for people to reach? like awesome-spark?

WDYT @eliasah @oluies

[oluies]

Shall we move this work under a more public space for people to reach? like awesome-spark?

WDYT @eliasah @oluies

Looks intresting, can it be made avaliable at Spark Packages?

[MrPowers]

@oluies - thanks for commenting!

Spark Packages has unfortunately been broken for a few years and looks like it's abandonware.

I had to shift all my stuff off Spark packages, but it's still causing my users confusion. I'll bring this up to the Databricks product team and see if they can either archive the sbt-spark-package repo or start maintaining it again. Clear messaging either way would be helpful for the community.

[MrPowers]

@yaooqinn - Just a friendly reminder to get your Maven namespace setup 😉 It'll be good for you to have one of these in any case.

Here are the instructions.

I'd recommend just getting com.yaooqinn to keep it simple.

I'll help you with all the hard work of getting the plugins setup, so the actual publishing part will just be a one command process.

[zero323]

I'd recommend just getting com.yaooqinn to keep it simple.

That might be the way to go. However, if you want to use org.awesomespark (thinking about it as something like community hub) I am fine with that ‒ just ping directly on matrix.org or keybase.

[yaooqinn]

@yaooqinn - Just a friendly reminder to get your Maven namespace setup 😉 It'll be good for you to have one of these in any case.

Sorry for the delay. It’s the Chinese New Year Festival now.😬

I'd recommend just getting com.yaooqinn to keep it simple.

I filed a New Project ticket in SonaType JIRA w/ com.github.yaooqinn % itachi a few days ago （also add mrpowers）and I’ll make some time to get the rest done in the next 2 or 3 days

FYI, https://issues.sonatype.org/browse/OSSRH-64476

[MrPowers]

@yaooqinn - Happy New Year!!!!

[yaooqinn]

Hi @MrPowers, I've followed your instructions and succeeded in pushing a SNAPSHOT but failed for a RELEASE. See: https://github.com/yaooqinn/itachi/actions?query=workflow%3ARelease

And Happy New Year to ALL.

[yaooqinn]

Oh, it seems to succeed at last with a 'long' retry :)

[MrPowers]

@yaooqinn - great job getting the JAR published. That's huge, we can start getting users now.

I updated the README to explain how to add itachi to a project. Let me know what you think.

Let's map out some additional work for this project. Would be cool to get some other open source contributors too - this is a great learning project!

[yaooqinn]

Let's map out some additional work for this project. Would be cool to get some other open source contributors too

I wonder if there is a way to get all those rejected PRs for adding extra SQL functions to Spark main from Github. Then we can contact the authors of these PRs and see if they're willing to help and contribute to the project. Or we could just make an announcement on Spark's mailing list.

[MrPowers]

@yaooqinn - just FYI, I'm starting to post to Reddit and will write other content to let people know about itachi.

Reaching out on the PRs and letting ppl know that some functions are probably better fits for itachi is another great idea.

Will keep marketing and trying to get users ;)

[MrPowers]

Also, closing this issue cause the JAR file is in Maven now!

[yaooqinn]

thanks a lot for your help!