search

Azure / data-api-builder

Data API builder provides modern REST and GraphQL endpoints to your Azure Databases and on-prem stores.
https://aka.ms/dab/docs
MIT License
933 stars 194 forks source link

[Bug]: Odd config file requirements to get "complex" cosmosdb_nosql to work #2335

Open sander1095 opened 3 months ago

sander1095 commented 3 months ago

What happened?

I have the following cosmosdb_nosql data setup with a database and 2 containers, conferences and talks:

data setup - **database** - conferences - ```json [ { "endDate": "2022-06-01", "id": "072d1638-a763-49f7-850f-763eed777837", "location": "", "mainTag": "", "name": "CONFERENCE", "startDate": "2022-06-01", "url": "https://example.com/", "year": 2022, "talks": [ { "abstract": "EXAMPLE", "mainTag": "", "tags": [], "talkId": "0c8c7101-26ee-42e9-9596-32ec1d229c0b", "talkLength": 50, "talkTime": null, "title": "EXAMPLE" } ] } ] - talks - ```json [ { "id": "f45a36c0-4031-4779-9461-eeedf37a7b74", "mainTag": "Software Development", "title": "EXAMPLE" } ] ```

The important thing to notice is that there are 3 models, of which 2 need to be exposed as their own entities in DAB:

I read the docs which tell me I need to define a graphql schema, on top of the dab-config.json, which I imagine has to look like this:

GraphQL schema ```graphql type Conference @model { id: ID! name: String! location: String year: Int! startDate: String endDate: String url: String mainTag: String talks: [ConferenceTalk!]! } type ConferenceTalk { talkId: ID! title: String! abstract: String mainTag: String talkTime: String talkLength: Int tags: [String] } type Talk @model { id: ID! title: String! mainTag: String } ```

Which leads me to my dab-config.json, which looks like this.

dab-config.json ```json { "$schema": "https://github.com/Azure/data-api-builder/releases/download/v1.2.10/dab.draft.schema.json", "data-source": { "database-type": "cosmosdb_nosql", "connection-string": "@env('COSMOSDB_CONNECTIONSTRING')", "options": { "database": "database", "container": null, "schema": "schema.gql" } }, "runtime": { "rest": { "enabled": false, "path": "/api", "request-body-strict": true }, "graphql": { "enabled": true, "path": "/graphql", "allow-introspection": true }, "host": { "cors": { "origins": [], "allow-credentials": false }, "authentication": { "provider": "StaticWebApps" }, "mode": "development" } }, "entities": { "Conference": { "source": { "object": "conferences" }, "graphql": { "enabled": true, "type": { "singular": "Conference", "plural": "Conferences" } }, "rest": { "enabled": false }, "permissions": [ { "role": "anonymous", "actions": [ { "action": "read" } ] } ] }, "Talk": { "source": { "object": "talks" }, "graphql": { "enabled": true, "type": { "singular": "Talk", "plural": "Talks" } }, "rest": { "enabled": false }, "permissions": [ { "role": "anonymous", "actions": [ { "action": "read" } ] } ] }, "ConferenceTalk": { "source": { "object": "WE_DO_NOT_EXPOSE_THIS_BUT_NEED_IT_TO_GET_DAB_TO_WORK" }, "graphql": { "enabled": false }, "rest": { "enabled": false }, "permissions": [ { "role": "anonymous", "actions": [ { "action": "read" } ] } ] } } } ```

The problem

I struggled with this for a long while. In order to get both ConferenceTalk and Talk to work, I must define ConferenceTalk as an entity in dab-config.json, which is what the error also says. However, this doesn't make sense because ConferenceTalk is not its own entity and should not need to be exposed.

Expected solution

I should not need to define entities like ConferenceTalk in dab-config.json for objects that are part of main/parent entities. What if I had objects 10 layers deep? Would I need to define all of them as entities in dab-config.json? This would lead to a huge dab-config.json with filler data, just like ConferenceTalk is now.

Extra

Please improve the docs about defining graphql yourself for cosmosdb. They're very limited. I have no idea why/if I need to define @model, for example.

Version

Microsoft.DataApiBuilder 1.2.10+c7ca8db8558a63919c530e454c8f18b45d5b931c

What database are you using?

CosmosDB NoSQL

What hosting model are you using?

Local (including CLI)

Which API approach are you accessing DAB through?

GraphQL

Relevant log output

Azure.DataApiBuilder.Service.Startup[0]
      Unable to complete runtime initialization. Refer to exception for error details.
      Azure.DataApiBuilder.Service.Exceptions.DataApiBuilderException: The entity 'ConferenceTalk' was not found in the runtime config.
         at Azure.DataApiBuilder.Core.Services.MetadataProviders.CosmosSqlMetadataProvider.AssertIfEntityIsAvailableInConfig(String entityName) in /_/src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:line 296
         at Azure.DataApiBuilder.Core.Services.MetadataProviders.CosmosSqlMetadataProvider.ProcessSchema(IReadOnlyList`1 fields, Dictionary`2 schemaDocument, String currentPath, IncrementingInteger tableCounter, EntityDbPolicyCosmosModel parentEntity, HashSet`1 visitedEntities) in /_/src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:line 228
         at Azure.DataApiBuilder.Core.Services.MetadataProviders.CosmosSqlMetadataProvider.ParseSchemaGraphQLFieldsForJoins() in /_/src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:line 180
         at Azure.DataApiBuilder.Core.Services.MetadataProviders.CosmosSqlMetadataProvider..ctor(RuntimeConfigProvider runtimeConfigProvider, IFileSystem fileSystem) in /_/src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs:line 85
         at Azure.DataApiBuilder.Core.Services.MetadataProviders.MetadataProviderFactory..ctor(RuntimeConfigProvider runtimeConfigProvider, IAbstractQueryManagerFactory queryManagerFactory, ILogger`1 logger, IFileSystem fileSystem, Boolean isValidateOnly) in /_/src/Core/Services/MetadataProviders/MetadataProviderFactory.cs:line 32
         at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
         at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
         at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
         at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(ServiceIdentifier serviceIdentifier)
         at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
         at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)    
         at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
         at Azure.DataApiBuilder.Service.Startup.PerformOnConfigChangeAsync(IApplicationBuilder app) in /_/src/Service/Startup.cs:line 620
fail: Azure.DataApiBuilder.Service.Startup[0]
      Could not initialize the engine with the runtime config file: dab-config.json
info: Microsoft.Hosting.Lifetime[0]
      Application is shutting down...
fail: Microsoft.Extensions.Hosting.Internal.Host[11]
      Hosting failed to start
      System.Threading.Tasks.TaskCanceledException: A task was canceled.
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
         at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
         at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)
Unable to launch the Data API builder engine.
Error: Failed to start the engine.

Code of Conduct

sander1095 commented 3 months ago

I've created a reproducable repo here: https://github.com/sander1095/dab-graphql-csharp-client

sajeetharan commented 2 months ago

@sander1095 thanks for raising the issue, we will check and get back!

golfalot commented 1 month ago

where does the fix for this sit on the priority list please ? Or is there a less convoluted workaround. Thanks.

sourabh1007 commented 3 weeks ago

Since you're working with a NoSQL database where the schema can vary with each record, DAB is unable to predict the sub-entities present. Therefore, it is recommended to explicitly configure the entities you're interested in within the configuration. This will ensure that DAB generates the GraphQL API only for those specified entities.

golfalot commented 3 weeks ago

Hi @sourabh1007 thank you for the technical explanation. I would however like to challenge, that from a usability point of view, it feels counter intuitive when notionally one has already provided a .gql schema defining all child class/type sub-entities relationships.

Being required to define sub-entities in config feels like an unnecessary barrier to entry for using this fabulous dab tool.