Closed lightbody closed 5 years ago
With the help of @jameskleeh, I was able to accomplish this without any changes to the source of this project:
import io.micronaut.configuration.jdbc.hikari.DatasourceConfiguration;
import io.micronaut.context.event.BeanInitializedEventListener;
import io.micronaut.context.event.BeanInitializingEvent;
import javax.inject.Singleton;
@Singleton
public class UnixSocketDatasourceConfigurationListener implements BeanInitializedEventListener<DatasourceConfiguration> {
private static final String REF = "://socket@";
@Override
public DatasourceConfiguration onInitialized(BeanInitializingEvent<DatasourceConfiguration> event) {
DatasourceConfiguration config = event.getBean();
// check if the URL appears to point to a unix socket
String url = config.getUrl();
int start = url.indexOf(REF);
if (start != -1) {
String socketPath = url.substring(start + REF.length());
config.setUrl(url.substring(0, start) + ":///");
config.addDataSourceProperty("socketFactory", UnixSocketFactory.class.getName());
config.addDataSourceProperty("socketFactoryArg", socketPath);
}
return config;
}
}
import jnr.unixsocket.UnixSocketAddress;
import jnr.unixsocket.UnixSocketChannel;
import javax.net.SocketFactory;
import java.io.File;
import java.io.IOException;
import java.net.InetAddress;
import java.net.Socket;
import java.net.UnknownHostException;
public class UnixSocketFactory extends SocketFactory {
private String socket;
public UnixSocketFactory(String socket) {
this.socket = socket;
}
@Override
public Socket createSocket() throws IOException {
return UnixSocketChannel.open(new UnixSocketAddress(new File(socket))).socket();
}
@Override
public Socket createSocket(String s, int i) throws IOException, UnknownHostException {
throw new UnsupportedOperationException();
}
@Override
public Socket createSocket(String s, int i, InetAddress inetAddress, int i1) throws IOException, UnknownHostException {
throw new UnsupportedOperationException();
}
@Override
public Socket createSocket(InetAddress inetAddress, int i) throws IOException {
throw new UnsupportedOperationException();
}
@Override
public Socket createSocket(InetAddress inetAddress, int i, InetAddress inetAddress1, int i1) throws IOException {
throw new UnsupportedOperationException();
}
}
With these two classes, your application.yml file can look like this:
datasources:
default:
url: jdbc:postgresql://socket@/path/to/project-id:us-central1:instance-name/.s.PGSQL.5432
username: postgres
password: password
As you can see, I introduced a simple new convention that looks for "socket@" and establishes the UnixSocketFactory if so. Otherwise, it keeps the same Hikari default behavior that connects to hosts via TCP/IP.
On Google Cloud Run the socket path will start with /cloudsql/
and for local development you can use the Google Cloud SQL Proxy like so to keep the same path:
./cloud_sql_proxy -dir /cloudsql
Feel free to close this out... or use it as inspiration for an improvement to this project. I suspect others who try to use Cloud Run + Cloud SQL will bump into similar issues.
Quick update: this works exactly the same in Google App Engine (Flexible).
FYI for anyone else looking for a socket factory: Cloud SQL JDBC SocketFactory
@kurtisvg Oh wow wish I found that a few weeks ago :)
For an app on App Engine Java 11, I was also facing similar issues. I followed your approach, but simplified it a bit. We can use Google's own unix socket factory instead of creating our own.
In my application.yaml I use a standard JDBC URL:
datasources:
default:
url: jdbc:postgresql:///my-db
dialect: POSTGRES
username: postgres
password: xxxxxxx
jpa:
default:
properties:
hibernate:
bytecode:
provider: none
hbm2ddl:
auto: update
And in my bean initialization listener, I've hard coded things like the Cloud SQL Instance name, but we could get that info from the JDBC URL and parse it to make things cleaner and reusable:
import io.micronaut.configuration.jdbc.hikari.DatasourceConfiguration
import io.micronaut.context.event.BeanInitializedEventListener
import io.micronaut.context.event.BeanInitializingEvent
import com.google.cloud.sql.postgres.SocketFactory
import javax.inject.Singleton
@Singleton
class CloudSqlPostgresqlDatasourceConfigurationListener implements BeanInitializedEventListener<DatasourceConfiguration> {
DatasourceConfiguration onInitialized(BeanInitializingEvent<DatasourceConfiguration> event) {
DatasourceConfiguration config = event.getBean()
config.setUrl("jdbc:postgresql:///")
config.addDataSourceProperty("socketFactory", SocketFactory.class.getName())
config.addDataSourceProperty("cloudSqlInstance", "gae-2nd-gen:us-central1:fruits-db")
return config
}
}
For anyone that runs into this post, and is using MySQL for cloud SQL and also using cloud run, and wants it to work with GraalVM, native-image, this was my solution:
I used socat to map to tcp socket to the cloudsql unix socket, like so:
url: jdbc:mariadb://127.0.0.1:3306/feedback?autoreconnect=true&useSSL=false
# socat and run.sh script based on:
#https://discourse.metabase.com/t/has-anyone-deployed-metabase-to-google-cloud-run/8217/6
FROM oracle/graalvm-ce:20.1.0-java8 as graalvm
RUN gu install native-image
COPY . /home/app/feedback-history-svc
WORKDIR /home/app/feedback-history-svc
RUN native-image --no-server -cp build/libs/feedback-history-svc-*-all.jar
RUN chmod +x run.sh
FROM frolvlad/alpine-glibc
RUN apk update && apk add libstdc++ && apk add coreutils && apk add socat
EXPOSE 8087
COPY --from=graalvm /home/app/feedback-history-svc/run.sh /app/run.sh
COPY --from=graalvm /home/app/feedback-history-svc/feedback-history-svc /app/feedback-history-svc
CMD ["/app/run.sh"]
#!/bin/sh
# Start the proxy, if $CLOUD_SQL_INSTANCE is set
if [[ "${CLOUD_SQL_INSTANCE}" ]]; then
# : is an argument separator for socat, so set up symlink to Cloud SQL UNIX socket without :
ln -s /cloudsql/$CLOUD_SQL_INSTANCE /app/mysql.sock
# Forward TCP:3306 to Cloud SQL Unix socket
nohup socat -d -d TCP4-LISTEN:3306,fork UNIX-CONNECT:/app/mysql.sock &
fi
# Start the server
/app/feedback-history-svc
Sorry to bring this old question back, but, are you also facing long times to start the Micronaut app in Cloud run with this socket set up? In my case, a simple Micronaut app with just a DB connection takes approx 5 seconds to start in cloud run, while using an external database (no socket required) takes only 2 seconds.
I've decided to use managed Google Cloud Run + a Google SQL Postgres database for my deployment. The docs are pretty sparse, but the way you can integrate the two securely without having to keep your database open to the public is to follow the directions outlined here.
As you can see, the trick is that Cloud Run bakes in a little unix socket that proxies the connection to the database. You can also recreate this same behavior by running the proxy locally.
Unix sockets aren't native to Java and thus the
SocketFactory
needs to be overrides. At first Google's docs suggested to me that their provided SocketFactory was what I needed.But after lots of testing, I've realized it does more than I need, such as authentication that has actually already been done once the proxy is established. Really all I needed was to override the
SocketFactory
and return ajnr.unixsocket.UnixSocket
pointing to:/cloudsql/project-id:us-central1:instance-name/.s.PGSQL.5432
And that does the trick, both for local development and in Cloud Run. Woo hoo!
Unfortunately, my problem is that the Micronaut SQL factories don't seem to give me the control I need to establish a custom
SocketFactory
. I'm pretty new to Micronaut, so I'm guessing it is very possible that it can be done via configuration, but I've yet to figure out how to invokeHikariConfig.addDataSourceProperty("socketFactory", "my.custom.SocketFactory")
viaapplication.yml
.So this issue could be one of a few things:
Thanks!