Jsonschema2Pojo does not follow "301 Moved Permanently"

[marcmesh]

I'm trying to generate Pojos for this schema: https://raw.githubusercontent.com/asyncapi/spec-json-schemas/master/schemas/2.6.0.json All of the references sub-schemas are referenced via http:// but then HTTP-redirecting to https:// For example:

❯ curl http://asyncapi.com/definitions/2.6.0/servers.json -v
*   Trying 34.148.19.16:80...
* Connected to asyncapi.com (34.148.19.16) port 80 (#0)
> GET /definitions/2.6.0/servers.json HTTP/1.1
> Host: asyncapi.com
> User-Agent: curl/7.87.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Location: https://asyncapi.com/definitions/2.6.0/servers.json
< Server: Netlify
< X-Nf-Request-Id: 01GX9KHHRQFWKVXKNHRKDPFH18
< Date: Wed, 05 Apr 2023 21:00:42 GMT
< Content-Length: 66
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host asyncapi.com left intact
Redirecting to https://asyncapi.com/definitions/2.6.0/servers.json%

But Jsonschema2Pojo is not following this Redirect:

❯ jsonschema2pojo -s 2.6.0.json -t jsonschema-out -log trace
Reading schema: file:/Users/marc/Downloads/asyncapi/2.6.0.json
Adding _260
Adding _260.Asyncapi
Reading schema: http://asyncapi.com/definitions/2.6.0/info.json
Exception in thread "main" java.lang.IllegalArgumentException: Error parsing document: http://asyncapi.com/definitions/2.6.0/info.json
    at org.jsonschema2pojo.ContentResolver.resolve(ContentResolver.java:78)
    at org.jsonschema2pojo.SchemaStore.create(SchemaStore.java:66)
    at org.jsonschema2pojo.SchemaStore.create(SchemaStore.java:144)
    at org.jsonschema2pojo.rules.SchemaRule.apply(SchemaRule.java:69)
    at org.jsonschema2pojo.rules.SchemaRule.apply(SchemaRule.java:38)
    at org.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:85)
    at org.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:42)
    at org.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:70)
    at org.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:38)
    at org.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:120)
    at org.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:65)
    at org.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:86)
    at org.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:41)
    at org.jsonschema2pojo.rules.SchemaRule.apply(SchemaRule.java:83)
    at org.jsonschema2pojo.rules.SchemaRule.apply(SchemaRule.java:76)
    at org.jsonschema2pojo.rules.SchemaRule.apply(SchemaRule.java:38)
    at org.jsonschema2pojo.SchemaMapper.generate(SchemaMapper.java:90)
    at org.jsonschema2pojo.Jsonschema2Pojo.generate(Jsonschema2Pojo.java:79)
    at org.jsonschema2pojo.cli.Jsonschema2PojoCLI.main(Jsonschema2PojoCLI.java:58)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'Redirecting': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (URL); line: 1, column: 13]
    at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:2418)
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:759)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3693)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2781)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:907)
    at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:793)
    at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4759)
    at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3173)
    at org.jsonschema2pojo.ContentResolver.resolve(ContentResolver.java:76)
    ... 18 more

[unkish]

Hi

At present JsonSchema2Pojo delegates reading/parsing of JSON schema to FasterXML/Jackson: https://github.com/joelittlejohn/jsonschema2pojo/blob/80d827db93e4cc3848c4f297ebbfe4b3fd936272/jsonschema2pojo-core/src/main/java/org/jsonschema2pojo/ContentResolver.java#L76 which in turn most probably is using standard java URL::openStream/HttpURLConnection that does not support redirects between different protocols: https://github.com/openjdk/jdk/blob/c67bbcea92919fea9b6f7bbcde8ba4488289d174/src/java.base/share/classes/sun/net/www/protocol/http/HttpURLConnection.java#L991-L996

Please see also:

• https://stackoverflow.com/questions/1884230/httpurlconnection-doesnt-follow-redirect-from-http-to-https
• https://bugs.openjdk.org/browse/JDK-4620571 (there's also rationale provided by Java engineers as to why it is that way)

[coiouhkc]

Hi @marcmesh , have you managed to make it work? I'm running into the same problem with exactly same JSON spec... The only workaround i could imagine so far would be to download all the JSON specs using bash+curl and then use some sed replace magic..

[unkish]

You could create a custom RuleFactory that would create SchemaStore with customized ContentResolver smht. like:

    public class CustomRuleFactory extends RuleFactory {

        @Override
        public void setSchemaStore(SchemaStore schemaStore) {
            super.setSchemaStore(new SchemaStore(
                    new ContentResolver() {
                        @Override
                        public JsonNode resolve(URI uri) {
                            return super.resolve(URI.create(uri.toString().replace("http://", "https://")));
                        }
                    },
                    getLogger()));
        }
    }

And configure CLI/plugin to use custom rulefactory

[coiouhkc]

I ended up just cloning the spec from https://github.com/asyncapi/spec-json-schemas, then replaced the http references with local ones.

[smoya]

Does this native HTTP Java client support HSTS. If enabled in asyncapi.com website, and supported by this client, the issue could be fixed.

[unkish]

I'm afraid it does not (don't know if HttpURLConnection could qualify as a "client").

Why not create an issue on asyncapi asking to replace http with https ?

[coiouhkc]

Already done (kinda) so. See https://asyncapi.slack.com/archives/C0230UAM6R3/p1697557159905629. Should stay http instead of https due to reasons explained in linked articles (btw. OpenAPI is using http as well).