Closed peter closed 9 years ago
yogthos
commented
9 years ago I don't believe this is Luminus specific and if it works locally, then It could be an issue with the URL for Heroku according to this. Check to make sure that the DATABASE_URL variable is available in the environment on the server with the correct jdbc URL.
peter
commented
9 years ago The database URL (DATABASE_URL) is set automatically by Heroku when the PostgreSQL addon is created. Also, I am able to connect to PostgreSQL directly like this:
(require '[clojure.java.jdbc :as db]) (db/insert! (System/getenv "DATABASE_URL") :posts {:subject "clojure works on heroku"})
If it matters, my project.clj dependencies are currently these:
[org.clojure/java.jdbc "0.3.2"]
[postgresql "9.1-901.jdbc4"](from https://devcenter.heroku.com/articles/clojure-web-application)
So is the issue maybe with yesql or some other library since direct JDBC access works?
peter
commented
9 years ago Another issue I have with Heroku deployments is that "heroku run lein repl" takes forever to start up and seems to time out about every second time.
yogthos
commented
9 years ago It sounds like the issue might be somewhere else, I doubt it's yesql since it just piggybacks on clojure.java.jdbc. Using lein repl isn't the recommended way to run the app. The default Procfile sets it up to use the main function from the compiled project, e.g:
web: java $JVM_OPTS -cp target/app.jar clojure.main -m app.core
``It might be an issue with environ that's used within Luminus. The library will try to resolve environment variables and should generate :database-url key for you if there's a DATABASE_URL available as a shell variable.
peter
commented
9 years ago I found the issue. It turns out db-spec needs to be a string for the username:password type URL that Heroku uses to work. It would be great if you could update your Heroku deployment documentation to reflect this.
For details, check out the get-connection function in clojure.java.jdbc:
This works:
(DriverManager/getConnection "jdbc:postgresql://ec2-54-217-202-108.eu-west-1.compute.amazonaws.com:5432/dcql6a1c91v5up" (as-properties {:user "uorytkslaggjoj", :password "
But this doesn't:
(DriverManager/getConnection (System/getenv "DATABASE_URL"))
peter
commented
9 years ago NOTE: the db-spec needs to be changed to a string in db/migrations.clj as well as it's duplicated there.
peter
commented
9 years ago And now for the next stumbling block on Heroku... Migrations run fine locally on postgresql (lein run migrate) but not on Heroku (heroku run lein run migrate). I get this exception:
INFO: Starting migrations
Jul 12, 2015 10:00:07 AM clojure.tools.logging$eval106$fn110 invoke
INFO: Ending migrations
Exception in thread "main" java.lang.NullPointerException, compiling:(/tmp/form-init1050664351735383545.clj:1:73)
at clojure.lang.Compiler.load(Compiler.java:7239)
at clojure.lang.Compiler.loadFile(Compiler.java:7165)
at clojure.main$load_script.invoke(main.clj:275)
at clojure.main$init_opt.invoke(main.clj:280)
at clojure.main$initialize.invoke(main.clj:308)
at clojure.main$null_opt.invoke(main.clj:343)
at clojure.main$main.doInvoke(main.clj:421)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:383)
at clojure.lang.AFn.applyToHelper(AFn.java:156)
at clojure.lang.Var.applyTo(Var.java:700)
at clojure.main.main(main.java:37)
Caused by: java.lang.NullPointerException
at org.postgresql.jdbc2.AbstractJdbc2Statement.replaceProcessing(AbstractJdbc2Statement.java:829)
at org.postgresql.jdbc2.AbstractJdbc2Statement.
yogthos
commented
9 years ago Ah, I'll definitely update the docs for Heroku connection url. You should be able to run migrations from the compiled jar by running java -jar target/app.jar migrate.
yogthos
commented
9 years ago Out of curiosity, how are you populating the db-url currently. Locally, you would populate it in profiles.clj where you have
{:provided {:env {:database-url "jdbc:postgresql://localhost/app_dev?user=gallery&password=pictures"}}}The string in the DATABASE_URL on Heroku should match this, e.g:
"jdbc:postgresql://ec2-54-217-202-108.eu-west-1.compute.amazonaws.com:5432/dcql6a1c91v5up?user=uorytkslaggjoj&password=yourpass"The migrations.clj namespace creates a map that should work with get-connection:
{:store :database
:db {:connection-uri (:database-url env)}}The clojure.java.jdbc accepts the :connection-uri key with the raw connection string.
peter
commented
9 years ago I am passing in the Heroku postgresql URL string directly as the db-spec (or db) instead of a map. That should work with jdbc/get-connection. However, when I run the migrations, I get this:
peter@Peters-MacBook-Air ~/src/marklunds]$ heroku run java -jar target/marklunds.jar migrate
Running java -jar target/marklunds.jar migrate attached to terminal... up, run.6593
2015-07-12 13:22:17.254:INFO::main: Logging initialized @14975ms
Jul 12, 2015 1:22:19 PM clojure.tools.logging$eval36$fn40 invoke
INFO: Starting migrations
Jul 12, 2015 1:22:19 PM clojure.tools.logging$eval36$fn40 invoke
INFO: creating migration table 'schema_migrations'
Jul 12, 2015 1:22:19 PM clojure.tools.logging$eval36$fn__40 invoke
INFO: Ending migrations
Exception in thread "main" java.lang.NullPointerException
at org.postgresql.jdbc2.AbstractJdbc2Statement.replaceProcessing(AbstractJdbc2Statement.java:829)
at org.postgresql.jdbc2.AbstractJdbc2Statement.
As you can see it successfully creates the schema_migrations table but then it falls over.
yogthos
commented
9 years ago The documentation for get-connection states that you should be passing a map pointing to the uri though:
Raw:
:connection-uri (required) a String
Passed directly to DriverManager/getConnectionYou can also pass a string to jdbc/get-connection though according to the same documentation:
" String: subprotocol://user:password@host:post/subname An optional prefix of jdbc: is allowed."
In fact, as I wrote above, that's what you need to do on Heroku for the Heroku DATABASE_URL to be parsed.
Here is my corresponding REPL session where migrations fail:
marklunds.core=> (require '[clojure.java.jdbc :as sql]) nil marklunds.core=> (require '[marklunds.db.core :as db]) nil marklunds.core=> (sql/get-connection db/db-spec)
marklunds.core=> (require '[migratus.core :as migratus]) nil marklunds.core=> (migratus/up {:store :database :db db/db-spec} []) Jul 12, 2015 1:30:10 PM clojure.tools.logging$eval47$fn51 invoke INFO: Starting migrations Jul 12, 2015 1:30:10 PM clojure.tools.logging$eval47$fn51 invoke INFO: Ending migrations NullPointerException org.postgresql.jdbc2.AbstractJdbc2Statement.replaceProcessing (AbstractJdbc2Statement.java:829)
yogthos
commented
9 years ago I tested with both approaches locally and you're right either a string or a map will work. I'm not sure why the map isn't working for you on heroku as it's the same URL string in both cases though.
yogthos
commented
9 years ago @pupeno out of curiosity have you run into anything similar with your heroku setup? @peter and have you had any luck? :)
pupeno
commented
9 years ago Yes, I believe I have run into this problem. I had some workarounds which I intended to integrate back into mainstream but I never got around to it. This is in my to-do list as I plan on using Heroku.
pupeno
commented
9 years ago For example, my patch for Korma accepts both, jdbc as well as Heroku-like URIs: https://github.com/korma/Korma/pull/316
yogthos
commented
9 years ago looks like we might need a function in db.core to handle different kinds of urls then
pupeno
commented
9 years ago @yogthos actually, many libraries should handle those URLs, so I created this little library: https://github.com/carouselapps/to-jdbc-uri
yogthos
commented
9 years ago Good idea, if using the library to parse different style URIs would be the best approach. If it works with both Heroku style and jdbc URIs then I think it would make sense to update Luminus to use it.
pupeno
commented
9 years ago Yes, that's the goal of the library. I wonder whether it should be used by Luminus or by the other libraries, like migratus, yesql, korma, etc. Having it in more than one place is not a problem as it's a no-op for proper jdbc URIs.
It works for Heroku and JDBC at the moment. I'm open to add more.
yogthos
commented
9 years ago I think it would probably be easier to start using it in Luminus first, as it doesn't require buy in from the library maintainers.
pupeno
commented
9 years ago Agreed.
J. Pablo Fernández pupeno@pupeno.com http://pupeno.com On Jul 15, 2015 15:25, "Dmitri Sotnikov" notifications@github.com wrote:
I think it would probably be easier to start using it in Luminus first, as it doesn't require buy in from the library maintainers.
— Reply to this email directly or view it on GitHub https://github.com/luminus-framework/luminus-template/issues/136#issuecomment-121632859 .
pupeno
commented
9 years ago yogthos
commented
9 years ago @pupeno oh by the way does this lib do the same thing by any chance? :) https://github.com/weavejester/hanami
pupeno
commented
9 years ago Yeah, it kinda does. I intend to-jdbc-uri to be more generic than just Heroku though.
yogthos
commented
9 years ago I think that's a good plan. :)
yogthos
commented
9 years ago @peter the problem should be solved in the latest template thanks to the library from @pupeno that parses both the Heroku and JDBC style URLs. The following steps should resolve the issue:
add the dependency:
[to-jdbc-uri "0.1.0"]require:
[to-jdbc-uri.core :refer [to-jdbc-uri]]then set the :connection-uri as follows:
:connection-uri (to-jdbc-uri (:database-url env))I'm going to close this one as it should be working now.
This may not be a problem with the Luminus framework as such but there seems to be issues with Heroku deployment. I keep getting "SQLException No suitable driver found" (DriverManager.java:689). I created my luminus app with +postgres and I have tried a variety of different postgresql driver versions to no avail. Any idea what the issue could be? The apps work great for me locally on my Macbook.