search

Azure / autorest.powershell

AutoRest PowerShell Generator
MIT License
111 stars 77 forks source link

[THREAD] PowerShell Generator - Quick Questions #153

Open fearthecowboy opened 5 years ago

fearthecowboy commented 5 years ago

If you have some quick questions about the PowerShell Generator, ask 'em here.

If you have an actual bug to report, go ahead and file an issue.

fearthecowboy commented 5 years ago

Getting Updates

We're deep in development now, and new builds will be published every time I merge a commit, so if you want to get the latest version of the PowerShell generator, you can just run

autorest --reset and AutoRest will remove the existing version, and will reinstall the latest beta the next time you use it.

darrelmiller commented 5 years ago

I am getting the following error:

ERROR (UnknownJsonType/LLCS2000): Schema '.schemas:4.allOf.1' is declared with invalid type 'undefined'
    - code-model-v3.yaml:1:0
C:\Users\darrmi\.autorest\@microsoft.azure_autorest.csharp-v2@1.0.106\node_modules\@microsoft.azure\autorest.csharp-v2\dist\plugin-llcsharp.js - {}
FATAL: Error: Unknown Model. Fatal.

I think it is related to this schema,

    "schemas": {
      "microsoft.graph.directory": {
        "allOf": [
          {
            "$ref": "#/components/schemas/microsoft.graph.entity"
          },
          {
            "title": "directory",
            "type": "object",
            "properties": {
              "deletedItems": {
                "type": "array",
                "items": {
                  "$ref": "#/components/schemas/microsoft.graph.directoryObject"
                }
              }
            }
          }

The full OpenAPI can be found here: https://graphslice.azurewebsites.net/v1.0/$openapi?tags=directory.directory&openapiversion=3

The referenced 'microsoft.graph.entity' has type: object. Are refs in schemas supported?

fearthecowboy commented 5 years ago

absolutely; they should be.

I'll have @NelsonDaniel take a look this morning.

NelsonDaniel commented 5 years ago

@darrelmiller, I just fixed the issue. Run: autorest --reset. That will reinstall the latest beta.

japj commented 5 years ago

Hello, I just installed the latest autorest@beta and am getting errors on https://github.com/MicrosoftDocs/vsts-rest-api-specs/blob/master/specification/account/4.1/accounts.json (the first file of the azure devops swagger rest api specs - 4.1 is TFS2018 Update3)

ERROR: Schema violation: Additional properties not allowed: x-ms-vss-area
- file:///D:/GitHub/vsts-rest-api-specs/specification/account/4.1/accounts.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-vss-resource-area-id
ERROR: Schema violation: Data does not match any schemas from 'anyOf'
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-override-version
ERROR: Schema violation: Additional properties not allowed: x-ms-vss-resource
ERROR: Schema violation: Additional properties not allowed: x-ms-vss-method

Similar errors with the 5.0 and 5.1 version of the file.

This is with:

japj commented 5 years ago

I’m not entirely sure, but can (most) of the Schema violations be solved by adding relevant entries to https://github.com/Azure/autorest/blob/master/schema/swagger-extensions.json?

I also assume the ($) at the end of filename was perhaps intended to show the line number of the error?

fearthecowboy commented 5 years ago

Ah, yes, if you have x-ms-* attributes that aren't known, it'll stop pretty quick.

Adding them to swagger-extensions would fix it.

darrelmiller commented 5 years ago

@NelsonDaniel Looks like I managed to bork my Autorest installation image Any suggestions? I tried npm uninstall but that doesn't solve the issue.

NelsonDaniel commented 5 years ago

@darrelmiller, it looks like you are using -input-file instead of --input-file with a double dash. Try that and let me know. In any case, I am going to take a look into cleaning up those error messages, there should've been a warning about the single dash usage.

darrelmiller commented 5 years ago

@NelsonDaniel /me hangs head in shame. Thank you. And it generated stuff! No more errors! /cc @deepak2016

japj commented 5 years ago

Adding them to swagger-extensions would fix it.

Is there a way to figure out what the “official” specification of these attributes should be (as they are x-ms specific), or should I just treat them all as strings?

Probably someone from the AzureDevOps team would know this, maybe @DarqueWarrior has a contact?

DarqueWarrior commented 5 years ago

Just sent some internal emails this morning.

fearthecowboy commented 5 years ago

@darrelmiller ProtTip™️ - If you ever have a borked install of autorest, just run autorest --reset ... that will remove all the plugins and install the latest autorest-core.

Hmm. Weird that it didn't yell at the single dash. That should have triggered the "Don't use old switches" message. sigh

DarqueWarrior commented 5 years ago

From what I was able to get from internal there is work being down on other issues that might trip up the generator. vsts-rest-api-specs/issues/152

Not sure if this is useful or not, but this doc describes some of the x-ms extensions

fearthecowboy commented 5 years ago

@DarqueWarrior -- yeah, operationId is required to be unique in the document.

deepak2016 commented 5 years ago

Hello, I'm hitting following error: 

PS D:\workspace\powershell\graph> autorest --powershell --input-file:./openapi-user.json
AutoRest code generation utility [version: 3.0.5155; node: v10.15.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
   Loading AutoRest core      'C:\Users\deagrawa\.autorest\@microsoft.azure_autorest-core@3.0.5378\node_modules\@microsoft.azure\autorest-core\dist' (3.0.5378)
   Installing AutoRest extension '@microsoft.azure/autorest.powershell' (beta)
   Installed AutoRest extension '@microsoft.azure/autorest.powershell' (beta->1.0.189)
   Installing AutoRest extension '@microsoft.azure/autorest.remodeler' (beta)
   Installed AutoRest extension '@microsoft.azure/autorest.remodeler' (beta->1.0.132)
   Installing AutoRest extension '@microsoft.azure/autorest.csharp-v2' (beta)
   Installed AutoRest extension '@microsoft.azure/autorest.csharp-v2' (beta->1.0.145)
ERROR: Syntax Error Encountered:  Unexpected token N in JSON
    - file:///D:/workspace/powershell/graph/openapi-user.json:1976:18
ERROR: Syntax Error Encountered:  Unexpected token N in JSON
    - file:///D:/workspace/powershell/graph/openapi-user.json:1976:18
--------------------------------------------------------------------

WARNING (UndefinedTypeWithSchema): The schema 'schemas:4' with a undefined type and using allOf is a bit ambigious. This has been auto-corrected to 'type:object'
    - oai3-subset-schema-reduced doc... ($)

WARNING (UndefinedTypeWithSchema): The schema 'schemas:15' with a undefined type and using allOf is a bit ambigious. This has been auto-corrected to 'type:object'
    - oai3-subset-schema-reduced doc... ($)

WARNING (UndefinedTypeWithSchema): The schema 'schemas:91' with a undefined type and using allOf is a bit ambigious. This has been auto-corrected to 'type:object'
    - oai3-subset-schema-reduced doc... ($)

WARNING (UndefinedTypeWithSchema): The schema 'schemas:344' with a undefined type and using allOf is a bit ambigious. This has been auto-corrected to 'type:object'
    - oai3-subset-schema-reduced doc... ($)
C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\dist\plugin-remodeler.js - FAILURE  {} Error: Invalid Reference schemas:1142 -- #/components/schemas/schemas:1142
    at Object.dereference (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\common.ts:52:15)
    at Remodeler.dereference (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:35:12)
    at Remodeler.copySchema (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:228:76)
    at Remodeler.add (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:288:12)
    at Remodeler.add (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:283:27)
    at Remodeler.refOrAdd (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:325:17)
    at Remodeler.copySchema (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:250:36)
    at Remodeler.add (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:288:12)
    at Remodeler.add (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:283:27)
    at Remodeler.refOrAdd (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:325:17)
PLUGIN FAILURE: Invalid Reference schemas:1142 -- #/components/schemas/schemas:1142, Error: Invalid Reference schemas:1142 -- #/components/schemas/schemas:1142
    at Object.dereference (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\common.ts:52:15)
    at Remodeler.dereference (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:35:12)
    at Remodeler.copySchema (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:228:76)
    at Remodeler.add (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:288:12)
    at Remodeler.add (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:283:27)
    at Remodeler.refOrAdd (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:325:17)
    at Remodeler.copySchema (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:250:36)
    at Remodeler.add (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:288:12)
    at Remodeler.add (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:283:27)
    at Remodeler.refOrAdd (C:\Users\deagrawa\.autorest\@microsoft.azure_autorest.remodeler@1.0.132\node_modules\@microsoft.azure\autorest.remodeler\src\remodeler.ts:325:17), {}
FATAL: Error: Invalid Reference schemas:1142 -- #/components/schemas/schemas:1142
C:\Users\deagrawa\.autorest\@microsoft.azure_autorest-core@3.0.5378\node_modules\@microsoft.azure\autorest-core\dist\lib\pipeline\pipeline.js - FAILURE {}
Process() cancelled due to exception : Plugin remodeler reported failure. / Error: Plugin remodeler reported failure.
    at C:\Users\deagrawa\.autorest\@microsoft.azure_autorest-core@3.0.5378\node_modules\@microsoft.azure\autorest-core\dist\lib\pipeline\plugins\external.js:26:19

Seems to be caused because of following section of input openapi file:

 "microsoft.graph.geoCoordinates": {
        "title": "geoCoordinates",
        "type": "object",
        "properties": {
          "altitude": {
            "anyOf": [
              {
                "type": "number"
              },
              {
                "type": "string"
              },
              {
                "enum": [
                  "-INF",
                  "INF",
                  NaN
                ]
              }
            ],
            "format": "double",
            "nullable": true
          },
          "latitude": {
            "anyOf": [
              {
                "type": "number"
              },
              {
                "type": "string"
              },
              {
                "enum": [
                  "-INF",
                  "INF",
                  NaN
                ]
              }
            ],
            "format": "double",
            "nullable": true
          },
          "longitude": {
            "anyOf": [
              {
                "type": "number"
              },
              {
                "type": "string"
              },
              {
                "enum": [
                  "-INF",
                  "INF",
                  NaN
                ]
              }
            ],
            "format": "double",
            "nullable": true
          }
        },
        "example": {
          "altitude": "Double",
          "latitude": "Double",
          "longitude": "Double"
        }
      },
deepak2016 commented 5 years ago

One more error, I am encountering is: 

PS D:\workspace\powershell\graph> autorest --powershell --input-file:./openapi-user.json
AutoRest code generation utility [version: 3.0.5155; node: v10.15.0]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
   Loading AutoRest core      'C:\Users\deagrawa\.autorest\@microsoft.azure_autorest-core@3.0.5378\node_modules\@microsoft.azure\autorest-core\dist' (3.0.5378)
   Loading AutoRest extension '@microsoft.azure/autorest.powershell' (beta->1.0.189)
   Loading AutoRest extension '@microsoft.azure/autorest.remodeler' (beta->1.0.132)
   Loading AutoRest extension '@microsoft.azure/autorest.csharp-v2' (beta->1.0.145)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Data does not match any schemas from 'oneOf'
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Data does not match any schemas from 'oneOf'
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Data does not match any schemas from 'oneOf'
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Data does not match any schemas from 'oneOf'
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-operation-type
    - file:///D:/workspace/powershell/graph/openapi-user.json ($)
C:\Users\deagrawa\.autorest\@microsoft.azure_autorest-core@3.0.5378\node_modules\@microsoft.azure\autorest-core\dist\lib\pipeline\pipeline.js - FAILURE {"exitCode":1}
Process() cancelled due to exception : [OperationAbortedException] Error occurred. Exiting. / Error: [OperationAbortedException] Error occurred. Exiting.
    at C:\Users\deagrawa\.autorest\@microsoft.azure_autorest-core@3.0.5378\node_modules\@microsoft.azure\autorest-core\dist\lib\pipeline\schema-validation.js:30:19
    at process._tickCallback (internal/process/next_tick.js:68:7)

caused by sections like below in open api input file:

"parameters": [
          {
            "in": "path",
            "name": "driveItem-id",
            "description": "key: driveItem-id",
            "required": true,
            "type": "string",
            "x-ms-docs-key-type": "driveItem"
          },
fearthecowboy commented 5 years ago

We don't have x-ms-docs-operation-type in our schema. (all x-ms-* extensions should be in our schema that it checks against) I can add x-ms-docs-operation-type

as for 

"anyOf": [
              {
                "type": "number"
              },
              {
                "type": "string"
              },
              {
                "enum": [
                  "-INF",
                  "INF",
                  NaN
                ]
              }
            ],

We don't have support for anyOf at this point -- I suggest you just reduce that to "type": "number"

fearthecowboy commented 5 years ago

I'm going to turn OpenAPI3 schema validation to Warning for now -- I'm not sure x- extensions are enabled on all of the nodes where it should be allowed. (Not that OAI has published an actual valid schema yet, right @darrelmiller ? .... 😄 )

deepak2016 commented 5 years ago

Please find attached open api yml file I'm using to generate powershell module using autorest. For below openAPI snippet, seems that Concrete class is not implementing methods from IMicrosoftGraphList interface, however, IMicrosoftGraphList interface is inheriting from IMicrosoftGraphBaseItem. Due to which generated powershell module compilation is failing.

 microsoft.graph.list:
      allOf:
        - $ref: '#/components/schemas/microsoft.graph.baseItem'
        - title: list
          type: object
          properties:
            displayName:
              type: string
              nullable: true
            list:
              $ref: '#/components/schemas/microsoft.graph.listInfo'
            sharepointIds:
              $ref: '#/components/schemas/microsoft.graph.sharepointIds'
            system:
              $ref: '#/components/schemas/microsoft.graph.systemFacet'
            columns:
              type: array
              items:
                $ref: '#/components/schemas/microsoft.graph.columnDefinition'
            contentTypes:
              type: array
              items:
                $ref: '#/components/schemas/microsoft.graph.contentType'
            drive:
              $ref: '#/components/schemas/microsoft.graph.drive'
            items:
              type: array
              items:
                $ref: '#/components/schemas/microsoft.graph.listItem'

@darrelmiller @fearthecowboy openapi.zip

https://gist.github.com/deepak2016/493b4f37f57e2acfd89542bec18460b2

fearthecowboy commented 5 years ago

@deepak2016 -- we'll look into this today.

Just a curiosity, why have an allOf that uses an inline model, and then no definition in the model itself?

Conceptually, you end up with a class that inherits two models, but defines nothing of it's own, rather than having a class that inherits a model, and defines more properties on top.

The PowerShell generator should handle this because we use composition, but I know that the older language generators that do inheritance end up making a dogs breakfast of it.

deepak2016 commented 5 years ago

@fearthecowboy this is the corresponding generated cs file: https://gist.github.com/deepak2016/98ff473e6721dfa089bf7fc09cb17668

If you see class MicrosoftGraphList implements IMicrosoftGraphList and IMicrosoftGraphList extends IMicrosoftGraphBaseItem. However class MicrosoftGraphList does not extend MicrosoftGraphBseItem. I'm not sure, if that is an expected/correct class hierarchy.

deepak2016 commented 5 years ago

@fearthecowboy I tried autorest generation again today and now compilation is going through fine. Thanks for the quick fix for issue.

deepak2016 commented 5 years ago

When trying to run the command after loading powershell moduel, I'm not able to send parameter to the command.

PS D:\workspace\powershell\graph [Subset of Microsoft Graph API]> Invoke-MeContactGetPhoto
Invoke-MeContactGetPhoto : Object reference not set to an instance of an object.
At line:1 char:1
+ Invoke-MeContactGetPhoto
+ ~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Invoke-MeContac...ontactsGetPhoto], NullReferenceException
+ FullyQualifiedErrorId : Sample.API.Cmdlets.InvokeMeContactGetPhoto_ContactsGetPhoto

PS D:\workspace\powershell\graph [Subset of Microsoft Graph API]> Invoke-MeContactGetPhoto -ContactId abc
Invoke-MeContactGetPhoto : Object reference not set to an instance of an object.
At line:1 char:1
+ Invoke-MeContactGetPhoto -ContactId abc
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Invoke-MeContac...tPhotoContactId], NullReferenceException
+ FullyQualifiedErrorId : Sample.API.Cmdlets.InvokeMeContactGetPhoto_ContactsGetPhotoContactId

This is the link to generated powershell code to invoke command: https://gist.github.com/deepak2016/f416a9f65c8c23c42fa153e9d93cc79d

Not sure if there is any code missing to accept input parameter?

fearthecowboy commented 5 years ago

@deepak2016 -- we have a couple large updates coming in the next few days; a refactoring of how parameters and properties are being handled. Should probably clean up a few things, provided I can finish debugging it soon...

deepak2016 commented 5 years ago

@fearthecowboy looking forward for the updates.

darrelmiller commented 5 years ago

@fearthecowboy The JSON Schema for OAS V3 is soooo close I can smell it https://github.com/OAI/OpenAPI-Specification/pull/1270#issuecomment-475313106

deepak2016 commented 5 years ago

@fearthecowboy has the updates been released? I am facing issue while compiling the generated module. 

PS D:\workspace\powershell\graph> .\generated\build-module.ps1
Creating isolated process...
Cleaning build folders...
Compiling module...
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphDriveAllof1.json.cs(56,217): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphDriveAllof1.json.cs(56,248): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphDriveAllof1.json.cs(59,225): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphDriveAllof1.json.cs(59,256): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(54,232): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(54,263): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(55,247): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(55,278): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(58,216): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(58,247): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
D:\workspace\powershell\graph\generated\build-module.ps1 : Compilation failed.
At D:\workspace\powershell\graph\generated\build-module.ps1:70 char:3
+   Write-Error 'Compilation failed.'
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,build-module.ps1
NakulSabharwal commented 5 years ago

@fearthecowboy I am also facing this issue while compiling. Any update on its fix ? 

PS D:\workspace\powershell\graph> .\generated\build-module.ps1
Creating isolated process...
Cleaning build folders...
Compiling module...
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphDriveAllof1.json.cs(56,217): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphDriveAllof1.json.cs(56,248): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphDriveAllof1.json.cs(59,225): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphDriveAllof1.json.cs(59,256): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(54,232): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(54,263): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(55,247): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(55,278): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(58,216): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
generated\api\Sample\API\Models\ComponentsSchemasMicrosoftGraphListAllof1.json.cs(58,247): error CS1061: 'IMicrosoftGraphSystemFacet' does not contain a definition for 'Linq' and no accessible extension method 'Linq' accepting a first argument of type 'IMicrosoftGraphSystemFacet' could be found (are you missing a using directive or an assembly reference?) [D:\workspace\powershell\graph\generated\Partial Graph API.csproj]
D:\workspace\powershell\graph\generated\build-module.ps1 : Compilation failed.
At D:\workspace\powershell\graph\generated\build-module.ps1:70 char:3
+   Write-Error 'Compilation failed.'
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,build-module.ps1
darrenjrobinson commented 5 years ago

This may seem like a rudimentary question, but where should Directives be placed for generation of PowerShell Modules ? I built a simple API and AutoRest successfully builds the PowerShell Module, but names it Invoke-Cmdletname when I'd like it to be Get-Cmdletname

Parameter Rename as detailed here https://github.com/Azure/autorest/blob/master/docs/powershell/options.md#Tweak-The-Way-It-Generates-Cmdlets has likely syntax.

When adding the Directive to the Literate Configuration document I don't seem to be able to get the syntax right for specifying the cmdlet Verb. The error returned is:

Process() cancelled due to exception : path.match is not a function / TypeError: path.match is not a function

Thinking I'm missing something quite basic and just need to be pointed in the right direction. Is there an example somewhere for Verb Rename?

fearthecowboy commented 5 years ago

@darrenjrobinson - Are you using a configuration file?

AutoRest allows extensive configuration thru it's configuration file, in which you stop putting stuff on the cmdline, and stick it in a file.

You can use .json, .yaml or .md (markdown literate configuration) for your file.

Simplest make a yaml file :

save as configuration.yaml

# your swagger file
input-file: foo.json

powershell: 
  namespace: MyCompany.API
  output-folder: out

directive:
  - where: 
      subject: VirtualMachine
      verb: Get
    set: 
      subject: VM

Then from the command line: autorest configuration.yaml

We use the markdown literate configuration file format, which is a markdown file. This has the virtue of being both docs and configuration, and when you're navigating in github, the readme.md is rendered automatically, and autorest will look for readme.md in the current folder, so you don't even have to specify it on the cmdline

The only difference is that configuration is in YAML code blocks in the markdown file.

# Sample Configuration

_the following line is required!_
> see https://aka.ms/autorest

``` yaml
# your swagger file
input-file: foo.json

powershell: 
  namespace: MyCompany.API
  output-folder: out

directive:
  - where: 
      subject: VirtualMachine
      verb: Get
    set: 
      subject: VM 
```
darrenjrobinson commented 5 years ago

Hey @fearthecowboy yes I'm using a Config file.

Even using a module wide directive (at the top of the config file) such as the following fails. Also tried RegEx versions. 

directive:
  - where:
      verb: invoke
    set:
      verb: get

the output is

Process() cancelled due to exception : path.match is not a function / TypeError: path.match is not a function at isAbsolute (/node_modules/@microsoft.azure/uri/dist/main.js:71:19) at Object.ResolveUri (/node_modules/@microsoft.azure/uri/dist/main.js:149:9) at ConfigurationView.ResolveAsPath (C:\Users\Darren Robinson.autorest\@microsoft.azure_autorest-core@3.0.5402\node_modules\@microsoft.azure\autorest-core\dist\lib\configuration.js:236:22) at linq_es2015_1.From.Select.each (C:\Users\Darren Robinson.autorest\@microsoft.azure_autorest-core@3.0.5402\node_modules\@microsoft.azure\autorest-core\dist\lib\configuration.js:305:34) at EnumerableImpl.Select [as _factory] (/node_modules/linq-es2015/lib/generators.js:29:15) at Select.next () at EnumerableImpl.ToArray (/node_modules/linq-es2015/lib/linq.js:372:18) at ConfigurationView.get InputFileUris [as InputFileUris] (C:\Users\Darren Robinson.autorest\@microsoft.azure_autorest-core@3.0.5402\node_modules\@microsoft.azure\autorest-core\dist\lib\configuration.js:306:14) at Object.get (C:\Users\Darren Robinson.autorest\@microsoft.azure_autorest-core@3.0.5402\node_modules\@microsoft.azure\autorest-core\dist\lib\configuration.js:223:82) at processInternal (C:\Users\Darren Robinson.autorest\@microsoft.azure_autorest-core@3.0.5402\node_modules\@microsoft.azure\autorest-core\dist\lib\autorest-core.js:103:26) at process._tickCallback (internal/process/next_tick.js:68:7) TypeError: path.match is not a function

fearthecowboy commented 5 years ago

Can you give us a concrete repro? I'll have to have @NelsonDaniel take a look at it.

darrenjrobinson commented 5 years ago

@NelsonDaniel @fearthecowboy

It a very simple API that I quickly built to help me get up to speed with AutoRest. A very simple Get. Here is the OpenAPI Definition that AutoRest successfully generates the module for but with the Verb Invoke, that I'm trying to change the verb to Get through the Config file.

swagger: '2.0'
info:
  title: BOFH
  version: 1.0.0
host: my.webhost.com.au
basePath: /
schemes:
  - https
  - http
paths:
  /api/excuse:
    get:
      operationId: /api/excuse/get
      produces:
        - application/json
      consumes: []
      parameters: []
      description: Get a BOFH Excuse
      responses:
        '200':
          description: Success operation
          schema:
            $ref: "#/definitions/GetExcuseResponseModel"
      security:
        - apikeyQuery: []
definitions:
  GetExcuseResponseModel:
    type: object
    properties:
      excuse:
        type: string
      excuseGenerated:
        type: string
securityDefinitions:
  apikeyQuery:
    type: apiKey
    name: code
    in: query
JustinGrote commented 5 years ago

Great work on this. I'm stepping through getting it to work for the LogicMonitor API since it has a swagger specification. One thing is that they have a convoluted authentication system requiring a complicated signature. I noticed that your example shows how to add custom functionality via C#, but sadly I'm a powershell guy :)

I did manage to use HttpPipelinePrepend to get what I want using your Gist example, but is there a better way to add this authorization to the commands?

Get-Alertlist -HttpPipelinePrepend { param($req, $callback, $next ) 
    <# Use TLS 1.2 #>
    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

    <# account info #>
    $accessId = 'oiajweoriajweorijafake'
    $accessKey = 'L+C-9Sp2w2R)wA4Ufakefakefake'
    $company = 'fakecompany'

    $httpVerb = [string]$req.Method
    $resourcePath = $req.RequestUri.PathAndQuery

    <# Get current time in milliseconds #>
    $epoch = [Math]::Round((New-TimeSpan -start (Get-Date -Date "1/1/1970") -end (Get-Date).ToUniversalTime()).TotalMilliseconds)

    <# Concatenate Request Details #>
    $requestVars = $httpVerb + $epoch + $resourcePath
    write-verbose "Request Variables: $requestVars"

    <# Construct Signature #>
    $hmac = New-Object System.Security.Cryptography.HMACSHA256
    $hmac.Key = [Text.Encoding]::UTF8.GetBytes($accessKey)
    $signatureBytes = $hmac.ComputeHash([Text.Encoding]::UTF8.GetBytes($requestVars))
    $signatureHex = [System.BitConverter]::ToString($signatureBytes) -replace '-'
    $signature = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($signatureHex.ToLower()))

    <# Construct Headers #>
    $auth = 'LMv1 ' + $accessId + ':' + $signature + ':' + $epoch
    $req.headers.Authorization = $auth

    # call the next step in the Pipeline
    $responseTask = $next.SendAsync($req, $callback)

    # wait on the responseTask
    $response = $responseTask.Result

    # let the cmdlet finish it's job
    return $responseTask
 }
rmustard commented 5 years ago

Is anyone able to generate a powershell API from https://github.com/MicrosoftDocs/vsts-rest-api-specs? I have tried removing any "x-ms-*" attributes, but still get the "anyOf" error.

ERROR: Schema violation: Data does not match any schemas from 'anyOf'
    - file:///C:/Users/rmustard/Documents/GitHub/vsts-rest-api-specs/specification/account/4.1/accounts.json ($)

Full output included here.

PS ~\GitHub\vsts-rest-api-specs\specification\account\4.1> autorest --powershell --input-file:accounts.json
AutoRest code generation utility [version: 3.0.5195; node: v10.15.1, max-memory: 8192 gb]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
   Loading AutoRest core      'C:\Users\rmustard\.autorest\@microsoft.azure_autorest-core@3.0.5445\node_modules\@microsoft.azure\autorest-core\dist' (3.0.5445)
   Installing AutoRest extension '@microsoft.azure/autorest.powershell' (beta)
   Installed AutoRest extension '@microsoft.azure/autorest.powershell' (beta->2.0.423)
   Installing AutoRest extension '@microsoft.azure/autorest.remodeler' (beta)
   Installed AutoRest extension '@microsoft.azure/autorest.remodeler' (beta->2.0.254)
   Installing AutoRest extension '@microsoft.azure/autorest.csharp-v2' (beta)
   Installed AutoRest extension '@microsoft.azure/autorest.csharp-v2' (beta->2.0.276)
ERROR: Schema violation: Additional properties not allowed: x-ms-vss-area
    - file:///C:/Users/rmustard/Documents/GitHub/vsts-rest-api-specs/specification/account/4.1/accounts.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-vss-resource-area-id
    - file:///C:/Users/rmustard/Documents/GitHub/vsts-rest-api-specs/specification/account/4.1/accounts.json ($)
ERROR: Schema violation: Data does not match any schemas from 'anyOf'
    - file:///C:/Users/rmustard/Documents/GitHub/vsts-rest-api-specs/specification/account/4.1/accounts.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-docs-override-version
    - file:///C:/Users/rmustard/Documents/GitHub/vsts-rest-api-specs/specification/account/4.1/accounts.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-vss-resource
    - file:///C:/Users/rmustard/Documents/GitHub/vsts-rest-api-specs/specification/account/4.1/accounts.json ($)
ERROR: Schema violation: Additional properties not allowed: x-ms-vss-method
    - file:///C:/Users/rmustard/Documents/GitHub/vsts-rest-api-specs/specification/account/4.1/accounts.json ($)
C:\Users\rmustard\.autorest\@microsoft.azure_autorest-core@3.0.5445\node_modules\@microsoft.azure\autorest-core\dist\lib\pipeline\pipeline.js - FAILURE {"exitCode":1}
Process() cancelled due to exception : [OperationAbortedException] Error occurred. Exiting. / Error: [OperationAbortedException] Error occurred. Exiting.
    at C:\Users\rmustard\.autorest\@microsoft.azure_autorest-core@3.0.5445\node_modules\@microsoft.azure\autorest-core\dist\lib\pipeline\schema-validation.js:30:19
    at process._tickCallback (internal/process/next_tick.js:68:7)
kevinblumenfeld commented 5 years ago

Referencing issue: https://github.com/Azure/autorest.powershell/issues/385 ERROR (UnknownJsonType/LLCS2000): Schema 'BasicAuthApplicationName' is declared with invalid type 'undefined'

ghost commented 5 years ago

First off LOVE PowerShell and LOVE what y'all are working on here. Our API requires an api key in the request header in order to authenticate. Is this supported? If so could you provide an example of how you would populate the api key?

"securityDefinitions": {
        "x-api-key": {
            "name": "x-api-key",
            "type": "apiKey",
            "in": "header"
        }
    },
fearthecowboy commented 5 years ago

@epanipinto-jc -- we currently don't handle adding auth data automatically. An example how to add it manually https://github.com/Azure/autorest/tree/master/docs/powershell/samples/timeswire

ghost commented 5 years ago

@fearthecowboy

Not being too familiar with C# it took a bit to figure out but changing the namespace and updating this section to append to the header vs query string appears to have worked. Thanks for the tip and quick response!

protected async Task<HttpResponseMessage> AddApiKey(HttpRequestMessage request, IEventListener callback, ISendAsync next) { // add on the apikey we'll cheat and pull it from the environment :D request.Headers.Add("x-api-key",System.Environment.GetEnvironmentVariable("xApiKey")); // let it go on. return await next.SendAsync(request, callback); }

ghost commented 4 years ago

It appears the generator only create cmdlets from items under the paths property within the OAS spec. Is there a way to create cmdlets for items that exist under definitions property that would essentially be New-* that would assist in the creation of objects?

An example of what I would like to do is I have a path that is used for the creation of new users, lets call that cmdlet Invoke-UserPost. That endpoint takes in specific properties that define a user which is defined within the definitions section of the OAS spec. Ideally the generator would create a function called New-User based upon that definition that would return the user as an object so that I can pipe it to the first command. New-User -FirstName:('Bruce') -LastName:('Wayne') | Invoke-UserPost

fearthecowboy commented 4 years ago

We originally were going to make something like that, but it ended up being less-than-awesome.

The fallback at this point is that all the objects have typeconverters that allow you to pass in a PSObject or HashTable that looks like the object

ie, 


$user = @{ FirstName="Bruce"; LastName="Wayne" } 

$user | Invoke-UserPost

As a matter of fact, the typeconverters that get generated are pretty damn awesome. They will go to great lengths to adapt the input to match the use

ghost commented 4 years ago

Ah bummer... I see how that could be complex. I was hoping to leverage the parameter validation and structure already in the OAS spec to remove redundant work. I stumbled on another way to kinda accomplish this, that is pretty manual, but i'm still trying to get it to work. I'm curious if you have any thoughts on this.

Import-Module ('TestApi.psd1') -Force
$NewUser = [TestApi.Models.UserPost]::new()
$NewUser.Firstname = 'Bruce'
$NewUser.Lastname = 'Wayne'
$NewUser.Email = 'NotBatman@WayneCorp.com'
$NewPhoneNumber = [TestApi.Models.UserPostPhoneNumbersItem]::new()
$NewPhoneNumber.Number = 'Work'
$NewPhoneNumber.Type = '123456789'
$NewUser.PhoneNumbers = $NewPhoneNumber
$NewUser | Invoke-UserPost
fearthecowboy commented 4 years ago

What's the difference between that and

Import-Module ('TestApi.psd1') -Force
$NewUser =  @{
  Firstname = 'Bruce'
  Lastname = 'Wayne'
  Email = 'NotBatman@WayneCorp.com'
  PhoneNumbers = @( @{
    Number = 'Work'
    Type = '123456789'
   })
}

$NewUser | Invoke-UserPost

# you can also cast it to force the typeconverter to run and give a typed obecjt
$TypedNewUser = [TestApi.Models.UserPost] $NewUser

Validation is currently only leveraged on _expanded parameter variants (which explode out all the body properties into powershell parameters)

fearthecowboy commented 4 years ago

FWIW, You can also call the Validate method on an object an have it give the validation errors for it (which are derived from the original OAI doc)

We didn't end up using that in the model objects, but it should be usable. 

$NewUser = [TestApi.Models.UserPost] @{
  Firstname = 'Bruce'
  Lastname = 'Wayne'
  Email = 'NotBatman@WayneCorp.com'
  PhoneNumbers = @( @{
    Number = 'Work'
    Type = '123456789'
   })
}

$NewUser.Validate( $listener)

the $listener needs to implement the IEventListener interface ... I don't have a sample for that, but you could spelunk the code and find it.

peombwa commented 4 years ago

Is there a clear guide that shows how to use profiles in conjunction with tag:all-api-versions to generate modules that targets two different API versions?

My overall goal is to generate single PowerShell module that can support two API versions. I'm generating my cmdlets using an OpenAPI v3 document, and my API has two versions, beta and v1.0 - https://graph.microsoft.com/{API-VERSION}/.

darrelmiller commented 4 years ago

@fearthecowboy Do you have any docs/examples available for this? We would like to use this for Microsoft Graph to switch between V1.0 and beta.

fearthecowboy commented 4 years ago

I don't have a doc for that at this point.

API version is a PATH parameter? .... hmm. I gotta go check how that would impact it.

We should probably talk about how the different API versions work; it may be possible to craft something you can use as a profile; but it may require some tweaking to get it there.

Can we meet on teams (sometime early tomorrow?).

darrelmiller commented 4 years ago

Sure, happy to meet and discuss.

James-Allerton commented 4 years ago

Hi, I'm completely new to autorest and have tried it for the first time but when running the following command: autorest --powershell --input-file=vecta.json I get the Error: FATAL: Error: Operation http-operation:0 : /api/Activities/{activityId} has no servers entry (Are you missing 'host' or 'servers' information in your source file?) Error: Plugin remodeler reported failure.

The Swagger JSON in the vecta.json file is from: https://api.vecta.net/swagger/V0/swagger.json

I don't get this error when running autorest --csharp --input-file=vecta.json autorest --python --input-file=vecta.json

They both generate without errors, which make me think it's something specific to the powershell generator but i just wanted to check if anyone could see something i'm missing?

Thank you for any suggestions.