michael-yuji / xc

FreeBSD container engine
Other
87 stars 9 forks source link

Java's InetAddress.getLocalHost() doesn't work in Jail #12

Closed j1cken closed 9 months ago

j1cken commented 10 months ago

After some googling and seeing other people having the same problem I have learned there's no access to a loopback device in a Jail and therefore there isn't something like 127.0.0.1 available.

What's the recommended way to resolve the following if you have these calls in a dependent library that queries the loopback device?

Here's the link to InetAddress JavaDoc

torben@beast ~/d/xc (main)> doas xc run --network default --vnet quay.io/keycloak/keycloak:23.0 -- /opt/keycloak/bin/kc.sh start-dev --db dev-mem --hostname 192.168.17.11
required_cleanerce: []
Updating the configuration and installing your custom providers, if any. Please wait.
2023-12-19 15:55:50,467 INFO  [io.quarkus.deployment.QuarkusAugmentor] (main) Quarkus augmentation completed in 4235ms
2023-12-19 15:55:51,399 INFO  [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: <unset>, Hostname: 192.168.17.11, Strict HTTPS: false, Path: <request>, Strict BackChannel: false, Admin URL: <unset>, Admin: <request>, Port: -1, Proxied: false
2023-12-19 15:55:52,660 WARN  [io.quarkus.agroal.runtime.DataSources] (main) Datasource <default> enables XA but transaction recovery is not enabled. Please enable transaction recovery by setting quarkus.transaction-manager.enable-recovery=true, otherwise data may be lost if the application is terminated abruptly
2023-12-19 15:55:53,047 WARN  [org.infinispan.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
2023-12-19 15:55:53,097 WARN  [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
2023-12-19 15:55:53,157 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2023-12-19 15:55:53,459 INFO  [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: node_138829, Site name: null
2023-12-19 15:55:53,462 INFO  [org.keycloak.broker.provider.AbstractIdentityProviderMapper] (main) Registering class org.keycloak.broker.provider.mappersync.ConfigSyncEventListener
2023-12-19 15:55:55,785 INFO  [org.keycloak.quarkus.runtime.storage.legacy.liquibase.QuarkusJpaUpdaterProvider] (main) Initializing database schema. Using changelog META-INF/jpa-changelog-master.xml

UPDATE SUMMARY
Run:                        117
Previously run:               0
Filtered out:                 0
-------------------------------
Total change sets:          117

2023-12-19 15:55:58,705 WARN  [com.arjuna.ats.arjuna] (main) ARJUNA012210: Unable to use InetAddress.getLocalHost() to resolve address.
2023-12-19 15:55:58,730 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: Failed to start server in (development) mode
2023-12-19 15:55:58,731 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) ERROR: ARJUNA012359: SocketProcessId.getpid could not get unique port.
2023-12-19 15:55:58,731 ERROR [org.keycloak.quarkus.runtime.cli.ExecutionExceptionHandler] (main) For more details run the same command passing the '--verbose' option. Also you can use '--help' to see the details about the usage of the particular command.
Crest commented 10 months ago

Jails can have three kind of network virtualisation: inherit, alias, and vnet. Inherit provides normal access to the parent (in this case host) network stack. Alias is the traditional one which uses the parent network stack, but is restricted to a subset of the host network addresses. Since each network stack can have only one loopback interface with 127.0.0.1/8 on jails are normally not allowed to use the host network stack's 127.0.0.1 address. To allow applications hardwired to use the loopback to still work the kernel rewrites address into the jail's first alias address, but application can find out that they have been tricked which is what appears to have happened to you. The newest and most flexibel configuration is provide a jail it's own network stack with everything that goes with this (network interfaces, IP addresses, routing tables, firewalls, etc.). Since such jail can't share interfaces with the host it has to be connected to the outside world instead of sharing the hosts network connection e.g. using a software bridge and one and of an epair interface pair, (dynamic) routing, SR-IOV virtual functions etc. This is more complex to setup, but vnet enabled jails get their own loopback interface with their own loopback addresses (127.0.0.1, ::1).

elliptic-kitty commented 10 months ago

Good news is I can run this container, bad news is only if I first assign an ip address of 127.0.0.1/8 to the loopback interface of the Jail.

I wonder if it'll be a good idea to by default add 127.0.0.1/8 to lo0 for all VNET jails

elliptic-kitty commented 10 months ago

@j1cken fyi, the command to run this will be doas xc run --ip 'lo0|127.0.0.1/8' --vnet quay.io/keycloak/keycloak:23.0 -- /opt/keycloak/bin/kc.sh start-dev --db dev-mem

j1cken commented 9 months ago

@Crest Thx for elaborating! I've always tried to use vnet though because without it I got another error that I should actually report in another issue.

@elliptic-kitty Thx, that works like a charm!

The only thing I've added to your command is the --network option to be able to access the container from the outside and therefore epairB gets added to the container. Automatic IP assignment works with both --network and --ip option!

doas xc run --network default --ip 'lo0|127.0.0.1/8' --vnet quay.io/keycloak/keycloak:23.0 -- /opt/keycloak/bin/kc.sh start-dev --db dev-mem

required_cleanerce: []
Updating the configuration and installing your custom providers, if any. Please wait.
2024-01-12 18:50:11,458 INFO  [io.quarkus.deployment.QuarkusAugmentor] (main) Quarkus augmentation completed in 4417ms
2024-01-12 18:50:12,556 INFO  [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: <unset>, Hostname: <request>, Strict HTTPS: false, Path: <request>, Strict BackChannel: false, Admin URL: <unset>, Admin: <request>, Port: -1, Proxied: false
2024-01-12 18:50:13,661 WARN  [io.quarkus.agroal.runtime.DataSources] (main) Datasource <default> enables XA but transaction recovery is not enabled. Please enable transaction recovery by setting quarkus.transaction-manager.enable-recovery=true, otherwise data may be lost if the application is terminated abruptly
2024-01-12 18:50:14,088 WARN  [org.infinispan.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
2024-01-12 18:50:14,143 WARN  [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
2024-01-12 18:50:14,199 INFO  [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2024-01-12 18:50:16,897 INFO  [org.keycloak.quarkus.runtime.storage.legacy.liquibase.QuarkusJpaUpdaterProvider] (main) Initializing database schema. Using changelog META-INF/jpa-changelog-master.xml

UPDATE SUMMARY
Run:                        117
Previously run:               0
Filtered out:                 0
-------------------------------
Total change sets:          117

2024-01-12 18:50:19,945 WARN  [com.arjuna.ats.arjuna] (main) ARJUNA012210: Unable to use InetAddress.getLocalHost() to resolve address.
2024-01-12 18:50:20,039 INFO  [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: node_609891, Site name: null
2024-01-12 18:50:20,122 INFO  [org.keycloak.broker.provider.AbstractIdentityProviderMapper] (main) Registering class org.keycloak.broker.provider.mappersync.ConfigSyncEventListener
2024-01-12 18:50:20,149 INFO  [org.keycloak.services] (main) KC-SERVICES0050: Initializing master realm
2024-01-12 18:50:21,211 INFO  [io.quarkus] (main) Keycloak 23.0.3 on JVM (powered by Quarkus 3.2.9.Final) started in 9.660s. Listening on: http://0.0.0.0:8080
2024-01-12 18:50:21,211 INFO  [io.quarkus] (main) Profile dev activated.
2024-01-12 18:50:21,211 INFO  [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-h2, jdbc-mariadb, jdbc-mssql, jdbc-mysql, jdbc-oracle, jdbc-postgresql, keycloak, logging-gelf, micrometer, narayana-jta, reactive-routes, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, smallrye-health, vertx]
2024-01-12 18:50:21,214 WARN  [org.keycloak.quarkus.runtime.KeycloakMain] (main) Running the server in development mode. DO NOT use this configuration in production.

xcd output:

2024-01-12T18:50:05.570916Z  INFO xc::container: starting jail name="xc-31bab68e2505"
ifconfig: SIOCSIFVNET: File exists
2024-01-12T18:50:05.613516Z  INFO xc::container: /sbin/ifconfig -j 5 lo0 inet 127.0.0.1/8 alias
2024-01-12T18:50:05.641309Z  INFO xc::container: /sbin/ifconfig -j 5 epair0b inet 192.168.17.11/24 alias
add net default: gateway 192.168.17.254
2024-01-12T18:50:05.656190Z  INFO xc::container::runner: spawn: Jexec {
    arg0: "/opt/keycloak/bin/kc.sh",
    args: [
        "start-dev",
        "--db",
        "dev-mem",
    ],
    envs: {},
    uid: None,
    gid: None,
    user: None,
    group: None,
    output_mode: Terminal,
    notify: None,
    work_dir: None,
}
j1cken commented 9 months ago

@elliptic-kitty regarding adding 127.0.0.1/8 by default ... do you want to use this issue for it or create a new one? At least for my case I have a working config option that fixes my problem. So I guess it's kind of an RFE rather a bug to add the loopback IP by default. WDYT?

elliptic-kitty commented 9 months ago

@j1cken implemented at https://github.com/michael-yuji/xc/commit/664aadc86ba04e377499851476938b48030c38f6