NewDocument doesn't follow references

[stevenh]

The documentation for NewDocument states:

This function will automatically follow (meaning load) any file or remote references that are found anywhere. Which means recursively also, like a spider, it will follow every reference found, local or remote.

However this appears to not be the case as using it with remote reference fails e.g.

func TestNewDocument(t *testing.T) {
    spec, err := os.ReadFile("openapi.yaml")
    require.NoError(t, err)

    document, err := libopenapi.NewDocument(spec)
    require.NoError(t, err)

    v3Model, errs := document.BuildV3Model()
    if len(errs) > 0 {
        t.Fatalf("cannot create v3 model from document: %d errors reported\n%s", len(errs), errors.Join(errs...))
    }
}

Errors / failure

{"time":"2024-05-24T15:54:56.976510707+01:00","level":"ERROR","msg":"unable to open the rolodex file, check specification references and base path","file":"https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml","error":"rolodex has no file systems configured, cannot open 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml'. Add a BaseURL or BasePath to your configuration so the rolodex knows how to resolve references"}
{"time":"2024-05-24T15:54:56.98663067+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 97, col 23"}
{"time":"2024-05-24T15:54:56.986686429+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 60, col 23"}
{"time":"2024-05-24T15:54:56.988067365+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 146, col 23"}
{"time":"2024-05-24T15:54:56.988848963+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 128, col 23"}
{"time":"2024-05-24T15:54:56.99099516+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 174, col 23"}
{"time":"2024-05-24T15:54:56.991513782+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 211, col 23"}
{"time":"2024-05-24T15:54:56.993294829+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 343, col 23"}
{"time":"2024-05-24T15:54:56.993561555+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 270, col 23"}
{"time":"2024-05-24T15:54:56.996231398+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 375, col 23"}
{"time":"2024-05-24T15:54:56.996368872+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 411, col 23"}
{"time":"2024-05-24T15:54:56.997371034+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 503, col 23"}
{"time":"2024-05-24T15:54:56.997641807+01:00","level":"ERROR","msg":"error building path item: schema build failed: reference 'https://opensource.zalando.com/restful-api-guidelines/models/problem-1.0.1.yaml#/Problem' cannot be found at line 480, col 23"}
--- FAIL: TestNewDocument (0.24s)
   api/api_test.go:43: cannot create v3 model from document: 1 errors reported
        component '#/Problem' does not exist in the specification

Changing this to the following fixes the issue:

func TestNewDocument(t *testing.T) {
    spec, err := os.ReadFile("openapi.yaml")
    require.NoError(t, err)

    config := &datamodel.DocumentConfiguration{
        AllowFileReferences:   true,
        AllowRemoteReferences: true,
    }

    document, err := libopenapi.NewDocumentWithConfiguration(petstore, config)
    require.NoError(t, err)

    v3Model, errs := document.BuildV3Model()
    if len(errs) > 0 {
        t.Fatalf("cannot create v3 model from document: %d errors reported\n%s", len(errs), errors.Join(errs...))
    }
}

The issue seems to be that in BuildV3Model d.config is nil so gets setup with:

    if d.config == nil {
        d.config = &datamodel.DocumentConfiguration{
            AllowFileReferences:   false,
            AllowRemoteReferences: false,
        }
    }

[daveshanley]

Docs need to be updated. The default behavior is to NOT follow references automatically now.