Issue 71 - Clojure deps

[gigasquid]

This moves the project from using lein to clojure deps

See issue: https://github.com/cnuernber/libpython-clj/issues/71

Converted and tested so far:

• Compile java classes
• Run tests
• Create jar
• Install local jar (and tested from another project)
• Dockerfile and running from nrepl (tested from emacs/cider)
• Working with CI (Travis)

Still to do:

• have someone test out doing actually deployment

[jjtolton]

I'm pretty excited for this!

[cnuernber]

This looks ready to land to me. All I use from leiningen is 'install snapshot', 'run a rep', 'install to clojars' all of which look to be supported. Also 'lein clean' I guess.

What further work is necessary to land this PR?

[gigasquid]

I think everything is ready for a final review @cnuernber and @jjtolton Please take a 👀 and let me know any feedback 😸

[gigasquid]

This looks ready to land to me. All I use from leiningen is 'install snapshot', 'run a rep', 'install to clojars' all of which look to be supported. Also 'lein clean' I guess.

What further work is necessary to land this PR?

Good feedback. I will add a clean alias. The clean is baked into all the other build tasks too. The only thing after that would be for you to maybe do a test deploy to clojars to make sure that works.

[gigasquid]

Feedback has been implemented. It's ready when you feel comfortable to merge.

[cnuernber]

Off we go! Thanks, this is great!

[cnuernber]

Now, the last thing which I didn't notice until I started using it is that you now have to know that javac is a root dependency before repl or anyting else. Previously leiningen would automatically track this dependency.

Is there a way with the current system to track this dependency or would that require boot or a makefile?

[gigasquid]

Yes - I was looking at rolling the javac into repl and tests by adding those to the build.clj this morning but wasn't sure if there was a better way to handle it. I'll ask around and see if I can come up with a good solution.

[cnuernber]

This is pretty close honestly. Closer than I thought you could get. At least it should be repl; we can provide a test script under scripts.

Hopefully we can also eliminate some of this duplication and then I could move more projects to this pathway. Can deps include other deps?

[gigasquid]

"Can deps include other deps?" - I'm not sure of the question. Could you please elaborate?

[cnuernber]

Well, there is a lot of structure and repetition in the deps.edn file - if I want to convert another project like for example tech.datatype (which also includes java) to this system it would be nice to abstract a lot of that. Note that tech.datatype is already partially deps based.

[jjtolton]

If I'm understanding correctly, you're trying to achieve the benefits of this deps.edn in other projects without needing to duplicate?

One experiment I tried was moving the aliases to my ~/.clojure/deps.edn profile. That worked well for clean and javac.

I'm guessing if the build.clj were deployed on git or clojars then those local files could be removed and the commands mostly or entirely available with ~/.clojure/deps.edn. Would require some R&D.

[cnuernber]

Hmm. I think this is why the clojure core libs are built with straight pom.xml files (those can reference each other). I wonder if there is a way to pull this off with a combination of boot and deps.edn. deps.edn were never meant to build a build system.

[jjtolton]

True story. I feel like a command line tool of some sort is a missing piece of the puzzle.

[gigasquid]

I do have some ideas:

• to make repl and test contain javac
• to separate out the build dependency and paths to it's own alias to reduce repetitiveness

I'll try to get a PR up in the next few days. We can then take a look at what it brings.

[cnuernber]

I think this pathway is objectively a bit worse than what we had before.

The original project.clj was 12 lines. That was sufficient to work well with all of the IDE's and editors in the clojure ecosystem and it allowed a clear release process with very little effort.

The deps.edn file that replaced it is 89 lines and it appears we also need a makefile or some other system of working with this.

I now want to do a release and that process is not clear and doesn't work with my encrypted credentials.gpg.

It isn't clear to me that this actually addresses one of the original desires which is to be able to include libpython-clj as a git dependency in another project that is based on deps.edn. I feel like this cannot be done at this time regardless due to the java class dependencies.

AFAICT, deps.edn only work well for pure-clojure projects. They aren't ready for mixed java/clojure projects.

When I deploy, I get:

chrisn@chrisn-lt-2:~/dev/cnuernber/libpython-clj$ clj -A:deploy
Execution error (ExceptionInfo) at badigeon.exec/exec (exec.clj:38).
Error while signing

Full report at:
/tmp/clojure-1373218383250084000.edn

I feel like the wins from this are minor, faster repl and such is a nice to have but it isn't a core win. If we could include this project as a git dependency in another project that would be a major win. The costs are nontrivial as of yet.

I can work around all of this by creating a new project.clj that uses the deps lein pathway. The project needs an actual build system so that is either leiningen or boot.

[cnuernber]

Well, I guess leiningen or boot or ant or makefiles or something along all those lines. Something I haven't seen explored is ant with deps.edn.

[gigasquid]

@cnuernber you might want to explore uncommenting out the section for prompting for credentials https://github.com/cnuernber/libpython-clj/blob/master/dev/src/build.clj#L179 - Sorry, I couldn't test that part out for you.

I definitely want the project to have the right tools to do tasks in the best way possible for the project. If a combo of deps, lein, or make is what works - then that is the best way.

I'm going to step back and let you and @jjtolton take customizing and getting the project setup working the best for your needs. If you have any particular questions, I'd be happy to ask around and research different options of doing this for you.

Thank you for both for your openness to embrace deps for the project. I truly think it will be a good base to build on for the future.

[gigasquid]

I did want to leave a few options here for you to consider for your future path:

• Collapse the dependency map for the build into it's own alias and then call it composed with the task. This will reduce duplication. Example:

  {:dev { :extra-deps {badigeon/badigeon {:git/url "https://github.com/EwenG/badigeon.git"
                                                                     :sha     "c5d7d8f9c44fee2f193ef924cdf8a485aee539c5"}
                                  ch.qos.logback/logback-classic {:mvn/version "1.1.3"}}
                     ;;; to supress noisy logs with logback file
            :extra-paths ["dev/resources"]
  :javac   {:main-opts ["-e" "(load-file,\"dev/src/build.clj\"),(build/clean)"]}

Then to call the javac with the dev dependencies would be clj -A:dev:javac The other aliases could be shorted that way as well.

• You could further modify build.clj to run tests and start up repl with javac. Although, it you go a makefile route - this might not be needed

• A final option is to break out the java portion of libpython-cl to another repo. I would depend on how often it changes, and how you want to manage it, but it would make some things easier, like the original point of including by sha from tools deps in other libraries and programs like nextjournal.

Just thoughts - I'll leave it for you to consider for your research :)

[cnuernber]

Thanks :-). We will discuss this on Clojurians. This is an interesting option and I have learned a lot from this deps.edn.