Autorest failing with ModuleNotFoundError: No module named 'm2r'

[koderjoker]

Facing issues while trying to generate Azure CLI code

Repro steps:

autorest —reset
autorest --az C:\Users\<USER>\azure-rest-api-specs\specification\redisenterprise\resource-manager\readme.md --azure-cli-folder=C:\Users\<USER>\azure-cli --target-mode=core --compatible-level=track2 --version=3.6.1

Output

AutoRest code generation utility [cli version: 3.6.1; node: v14.18.1]
(C) 2018 Microsoft Corporation.
https://aka.ms/autorest
info    | AutoRest core version selected from configuration: 3.6.1.
info    |    Loading AutoRest core      'C:\Users\<USER>\.autorest\@autorestcore@3.6.1\nodemodules\@autorest\core\dist' (3.6.1)
info    |    Loading AutoRest extension '@autorest/az' (latest->1.8.0)
info    |    Loading AutoRest extension '@autorest/python' (5.4.0->5.4.0)
info    |    Loading AutoRest extension '@autorest/clicommon' (0.6.2->0.6.2)
info    |    Loading AutoRest extension '@autorest/modelerfour' (4.15.421->4.15.421)
Executing: C:\Users\<USER>\.autorest\@autorest_python@5.4.0\node_modules\@autorest\python\venv\Scripts\python.exe -m autorest.jsonrpc.server
warning | IgnoredPropertyNextToRef | Semantic violation: Sibling values alongside $ref will be ignored. See https://github.com/Azure/autorest/blob/main/docs/openapi/howto/$ref-siblings.md for allowed values (components > schemas > ClusterProperties > properties > provisioningState)
  keys: [ 'readOnly' ]
    - file:///C:/Users/<USER>/azure-rest-api-specs/specification/redisenterprise/resource-manager/Microsoft.Cache/stable/2022-01-01/redisenterprise.json:1346:9
warning | IgnoredPropertyNextToRef | Semantic violation: Sibling values alongside $ref will be ignored. See https://github.com/Azure/autorest/blob/main/docs/openapi/howto/$ref-siblings.md for allowed values (components > schemas > ClusterProperties > properties > resourceState)
  keys: [ 'readOnly' ]
    - file:///C:/Users/<USER>/azure-rest-api-specs/specification/redisenterprise/resource-manager/Microsoft.Cache/stable/2022-01-01/redisenterprise.json:1351:9
warning | IgnoredPropertyNextToRef | Semantic violation: Sibling values alongside $ref will be ignored. See https://github.com/Azure/autorest/blob/main/docs/openapi/howto/$ref-siblings.md for allowed values (components > schemas > DatabaseProperties > properties > provisioningState)
  keys: [ 'readOnly' ]
    - file:///C:/Users/<USER>/azure-rest-api-specs/specification/redisenterprise/resource-manager/Microsoft.Cache/stable/2022-01-01/redisenterprise.json:1442:9
warning | IgnoredPropertyNextToRef | Semantic violation: Sibling values alongside $ref will be ignored. See https://github.com/Azure/autorest/blob/main/docs/openapi/howto/$ref-siblings.md for allowed values (components > schemas > DatabaseProperties > properties > resourceState)
  keys: [ 'readOnly' ]
    - file:///C:/Users/<USER>/azure-rest-api-specs/specification/redisenterprise/resource-manager/Microsoft.Cache/stable/2022-01-01/redisenterprise.json:1447:9
warning | PreCheck/AllOfWhenYouMeantRef | Schema 'ProxyResource' is using an 'allOf' instead of a $ref. This creates a wasteful anonymous type when generating code.
warning | PreCheck/CheckDuplicateSchemas | Checking for duplicate schemas, this could take a (long) while.  Run with --verbose for more detail.
API Exception: {'type': 'ModuleNotFoundError', 'args': ("No module named 'm2r'",), 'message': "No module named 'm2r'"}
Traceback (most recent call last):
  File "C:\Users\<USER>\.autorest\@autorest_python@5.4.0\node_modules\@autorest\python\venv\lib\site-packages\jsonrpc\manager.py", line 113, in _get_responses
    result = method(*request.args, **request.kwargs)
  File "C:\Users\<USER>\.autorest\@autorest_python@5.4.0\node_modules\@autorest\python\autorest\jsonrpc\server.py", line 35, in Process
    from ..m2r import M2R as PluginToLoad
  File "C:\Users\<USER>\.autorest\@autorest_python@5.4.0\node_modules\@autorest\python\autorest\m2r\__init__.py", line 11, in <module>
    import m2r
ModuleNotFoundError: No module named 'm2r'
warning | cli.auto-parameter-hidden is not true, skip visibility cleaner
info    | NOTE: You are currently generating based on track2 SDK
fatal   | Error: --az.extensions should not be null!
fatal   | Process() cancelled due to failure
  Error: Server error

As per discussion with the autorest team, the code generator has pinned older versions of @autorest/python version and @autorest/modelerfour, causing such errors.

Forcing it to use the latest versions

autorest --az C:\Users\<USER>\azure-rest-api-specs\specification\redisenterprise\resource-manager\readme.md --azure-cli-folder=C:\Users\<USER>\azure-cli --target-mode=core --compatible-level=track2 --version=3.8.4 --use=@autorest/python@5.16.0 --use=@autorest/modelerfour@~4.23.0 --debug --verbose

breaks its functionality with errors such as

debug   | [9.23 s] python/m2r - START inputs = 2
debug   | [9.23 s] [_main__.Process:32] Autorest called process with plugin_name 'm2r' and session_id: 'session8'
PLUGIN FAILURE: unknown tag !<!AnyObjectSchema> in "code-model-v4.yaml" at line 1148, column 3:
      objects:
      ^, YAMLException: unknown tag !<!AnyObjectSchema> in "code-model-v4.yaml" at line 1148, column 3:
      objects:
      ^
    at generateError (C:\Users\<USER>\.autorest\@autorest_clicommon@0.6.2\node_modules\js-yaml\lib\js-yaml\loader.js:167:10)

debug   | [15.20 s] Emitting 'python-files' at file:///C:/Users/<USER>/azure-rest-api-specs/specification/redisenterprise/resource-manager/generated/models/_init_.py
debug   | [15.20 s] python/codegen/emitter - END [0.01 s][93 MB]
fatal   | Process() cancelled due to exception : Plugin cli-prenamer reported failure. / Error: Plugin cli-prenamer reported failure.
    at C:\Users\<USER>\.autorest\@autorestcore@3.8.4\node_modules\@autorest\core\dist\src_lib_autorest-corets.js:2792:19
    at ScheduleNode (C:\Users\<USER>\.autorest\@autorestcore@3.8.4\node_modules\@autorest\core\dist\src_lib_autorest-corets.js:1338:29)
error   |   Error: Plugin cli-prenamer reported failure.
error   | Autorest completed with an error. If you think the error message is unclear, or is a bug, please declare an issues at https://github.com/Azure/autorest/issues with the error message you are seeing.
debug   | [15.20 s] Shutting Down.
debug   | [15.20 s] Exiting.

Could someone please help out with this?

[jitghosh]

Blocked on identical issue. Any progress on a resolution ? @koderjoker did you get any updates ?

[koderjoker]

@jitghosh hopped on call with the Azure CLI team but weren't able to find the root cause. Tried everything possible recommended here https://github.com/Azure/autorest.az#take-logz-extension-as-a-dry-run. In the end copied the generated code folder from them since generation worked on their system

[richardnnn]

@jitghosh @koderjoker Tried the solution mentioned in https://github.com/Azure/autorest.az/issues/882 and confirmed it worked. Note there will be further dependency issues, google them and install the right version.