microsoft / kiota

OpenAPI based HTTP Client code generator
https://aka.ms/kiota/docs
MIT License
2.83k stars 194 forks source link

kiota uses too much memory generating the msgraph beta go sdk #1743

Closed jaym closed 2 years ago

jaym commented 2 years ago

I'm running on a linux machine with 32GB, of which at least 28GB was available.

I ran:

hidi transform -d https://raw.githubusercontent.com/microsoftgraph/msgraph-metadata/master/openapi/beta/openapi.yaml -c msgraph.json -o output.yaml
kiota -l go -d output.yaml -c GraphServiceClient -n msgraphbetasdkgo  -o client --ll Trace

where msgraph.json is from https://gist.github.com/jaym/6cbb56b2405274bd169d76b2d4afb3f0

When kiota runs, it dies after a minute or so because it gets SIGKILL'd for using up all the memory:

fish: Job 1, 'kiota -l go -d output2.yaml -c…' terminated by signal SIGKILL (Forced quit)

dmesg shows the reason:

[1149689.168193] Out of memory: Killed process 3477026 (kiota) total-vm:38182588kB, anon-rss:26873152kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:54136kB oom_score_adj:0

Here's the tail of the output. There are a lot more of those not found warnings:

warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.managementIntent not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.managementTemplateCollection not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.managementTemplate not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.managementTemplateStep not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.managementTemplateStepVersion not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.tenantGroup not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.tenant not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.tenantCustomizedInformation not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.tenantDetailedInformation not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.tenantTag not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.windowsDeviceMalwareState not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.windowsProtectionState not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.managedTenants.managementTemplateStepDeployment not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.search.searchAnswer not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.externalConnectors.connectionOperation not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.externalConnectors.connectionQuota not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.externalConnectors.externalConnection not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.externalConnectors.externalGroup not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.externalConnectors.externalItem not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.externalConnectors.schema not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.externalConnectors.identity not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.windowsUpdates.windows not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.windowsUpdates.updatableAsset not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.windowsUpdates.catalog not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.windowsUpdates.catalogEntry not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.windowsUpdates.deployment not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.windowsUpdates.deploymentAudience not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.windowsUpdates.resourceConnection not found in the OpenAPI document.
warn: Kiota.Builder.KiotaBuilder[0]
      Discriminator microsoft.graph.windowsUpdates.updates not found in the OpenAPI document.
trce: Kiota.Builder.KiotaBuilder[0]
      Creating method Get of Kiota.Builder.CodeType
trce: Kiota.Builder.KiotaBuilder[0]
      Creating method CreateGetRequestInformation of Kiota.Builder.CodeType
dbug: Kiota.Builder.KiotaBuilder[0]
      CreateRequestBuilderClass 00:00:00.3151952
dbug: Kiota.Builder.KiotaBuilder[0]
      MapTypeDefinitions 00:00:00.1679643
trce: Kiota.Builder.KiotaBuilder[0]
      0ms: Created source model with 11 classes
dbug: Kiota.Builder.KiotaBuilder[0]
      step 4 - create source model - took 00:00:00.4835702
baywet commented 2 years ago

closing as duplicate of #1744 as I wasn't able to reproduce the memory issue either. Will re-open depending on the development of the other issue.