search

kubernetes-client / gen

Common generator scripts for all client libraries
Apache License 2.0
148 stars 145 forks source link

Update maven from 3.5 to 3.6.3 and required dependencies #184

Closed abelsromero closed 2 years ago

abelsromero commented 3 years ago

This PR begun with the intent to update the maven version used 3.5 (2018-06-21) on java8 by latest 3.6.3 (2019-11-25) in java11. But this broke other things and ended up requiring several changes in all generators.

Now, all but csharp work, which fails with the following error. I suspect it could be the ammount of types in the k8s API is too much.

[INFO] /tmp/.autorest/@autorest_core@3.0.6375/node_modules/@autorest/core/dist/src_lib_autorest-core_ts.js - FAILURE {}
[INFO] /tmp/.autorest/@autorest_core@3.0.6375/node_modules/@autorest/core/dist/src_lib_autorest-core_ts.js - FAILURE {}
[INFO] Process() cancelled due to exception : Maximum call stack size exceeded / RangeError: Maximum call stack size exceeded
[INFO]   RangeError: Maximum call stack size exceeded
[INFO]     at mergeYamls (/tmp/.autorest/@autorest_core@3.0.6375/node_modules/@autorest/core/dist/src_lib_autorest-core_ts.js:10534:18)
[INFO]     at processTicksAndRejections (internal/process/task_queues.js:93:5)
[INFO]     at async LoadLiterateSwagger (/tmp/.autorest/@autorest_core@3.0.6375/node_modules/@autorest/core/dist/src_lib_autorest-core_ts.js:3872:18)
[INFO]     at async LoadLiterateSwaggers (/tmp/.autorest/@autorest_core@3.0.6375/node_modules/@autorest/core/dist/src_lib_autorest-core_ts.js:3902:29)
[INFO]     at async /tmp/.autorest/@autorest_core@3.0.6375/node_modules/@autorest/core/dist/src_lib_autorest-core_ts.js:3926:26
[INFO]     at async ScheduleNode (/tmp/.autorest/@autorest_core@3.0.6375/node_modules/@autorest/core/dist/src_lib_autorest-core_ts.js:2543:33)

Note also that CSharp does not currently work neither because it downloads latest version of autorest which requires a newer version of dotnet.

We could try to use the openapi maven plugin, based on this example, there's support for csharp https://openapi-generator.tech/docs/usage/, I cannot validate further than checking than some code is being generated. wdyt, should I replace current autorest usage?

Said, that this PR required the following changes:

k8s-ci-robot commented 3 years ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: abelsromero To complete the pull request process, please assign brendandburns after the PR has been reviewed. You can assign the PR to them by writing /assign @brendandburns in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/kubernetes-client/gen/blob/master/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
brendandburns commented 3 years ago

To fix csharp code-gen we need to pin at version 2 so we should set npm i autorest@2.0.4413

see: https://github.com/kubernetes-client/csharp/issues/563

We can do that in this PR or a separate PR

brendandburns commented 3 years ago

We need to validate that this doesn't introduce any code differences in the generated clients before this can be merged.

Ideally this would be a pre-merge check, but sadly it's not currently added as a test.

abelsromero commented 3 years ago

We can do that in this PR or a separate PR

I can give it a try in this same PR.

abelsromero commented 3 years ago

Running with v2.0.4413 resulted in the following error

There is a new version of AutoRest available (3.1.2).
 > You can install the newer version with with npm install -g autorest@latest

Starting @microsoft.azure/autorest-core from /tmp/.autorest/@microsoft.azure_autorest-core@2.0.4417
   Loading AutoRest core      '/tmp/.autorest/@microsoft.azure_autorest-core@2.0.4417/node_modules/@microsoft.azure/autorest-core/dist' (2.0.4417)
No configuration found at 'file:///'.
   Loading local AutoRest extension '@microsoft.azure/autorest.csharp' (/node_modules/@microsoft.azure/autorest.csharp)
   Loading local AutoRest extension '@microsoft.azure/autorest.modeler' (/node_modules/@microsoft.azure/autorest.modeler)
events.js:292
      throw er; // Unhandled 'error' event
      ^

Error: write EPIPE
    at afterWriteDispatched (internal/stream_base_commons.js:156:25)
    at writeGeneric (internal/stream_base_commons.js:147:3)
    at Socket._writeGeneric (net.js:785:11)
    at Socket._write (net.js:797:8)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at Socket.Writable.write (internal/streams/writable.js:303:10)
    at Writable.write [as _write] (/tmp/.autorest/@microsoft.azure_autorest-core@2.0.4417/node_modules/@microsoft.azure/autorest-core/dist/lib/pipeline/plugin-endpoint.js:38:31)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at Writable.write (internal/streams/writable.js:303:10)
    at StreamMessageWriter.write (/node_modules/vscode-jsonrpc/lib/messageWriter.js:82:27)
    at /node_modules/vscode-jsonrpc/lib/main.js:761:35
    at new Promise (<anonymous>)
    at Object.sendRequest (/node_modules/vscode-jsonrpc/lib/main.js:751:26)
    at Object.GetPluginNames (/tmp/.autorest/@microsoft.azure_autorest-core@2.0.4417/node_modules/@microsoft.azure/autorest-core/dist/lib/pipeline/plugin-endpoint.js:78:86)
    at AutoRestExtension.GetPluginNames (/tmp/.autorest/@microsoft.azure_autorest-core@2.0.4417/node_modules/@microsoft.azure/autorest-core/dist/lib/pipeline/plugin-endpoint.js:116:31)
    at Function.FromChildProcess (/tmp/.autorest/@microsoft.azure_autorest-core@2.0.4417/node_modules/@microsoft.azure/autorest-core/dist/lib/pipeline/plugin-endpoint.js:110:41)
Emitted 'error' event on Socket instance at:
    at emitErrorNT (internal/streams/destroy.js:106:8)
    at emitErrorCloseNT (internal/streams/destroy.js:74:3)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  errno: -32,
  code: 'EPIPE',
  syscall: 'write'
brendandburns commented 3 years ago

I'll take a look at getting the csharp generation working. There was a commit to do this here:

https://github.com/kubernetes-client/gen/commit/74ee0c60948ecdc4af8613bcb0b658d0fdc1b821

But it got overwritten for some reason.

brendandburns commented 3 years ago

Taking a step back to the beginning, what is the motivation for upgrading to Java8/Maven 3.6? Is there something broken with the existing setup?

abelsromero commented 3 years ago

Taking a step back to the beginning, what is the motivation for upgrading to Java8/Maven 3.6? Is there something broken with the existing setup?

Mainly keeping things with up-to-date and supported versions.

brendandburns commented 3 years ago

Ok, given that there's no pressing need, I think I would like to delay this PR until we get csharp generating correctly. I'll try to get to that soon-ish. Apologies for the delay.

brendandburns commented 3 years ago

Ok, given that there's no pressing need, I think I would like to delay this PR until we get csharp generating correctly. I'll try to get to that soon-ish. Apologies for the delay.

abelsromero commented 3 years ago

Ok, given that there's no pressing need, I think I would like to delay this PR until we get csharp generating correctly. I'll try to get to that soon-ish. Apologies for the delay.

No problem at all. JS it not in my area of expertise, but I can do some more research and share what I find.

abelsromero commented 3 years ago

Tried to run autorest "natively" in my machine and it seems to go further, but finally fails with

DEBUG: [WRITING] ModelTemplate => Models/V1ValidatingWebhookConfigurationList.cs
DEBUG: [WRITING] ModelTemplate => Models/Corev1EventList.cs
DEBUG: [WRITING] ModelTemplate => Models/V1beta1MutatingWebhookConfigurationList.cs
DEBUG: csharp/generate - END
DEBUG: csharp/simplifier - START
No usable version of the libssl was found
FATAL: csharp/simplifier - FAILED
FATAL: Error: [Exception] AutoRest extension '@microsoft.azure/autorest.csharp' terminated.
Process() cancelled due to exception : [Exception] AutoRest extension '@microsoft.azure/autorest.csharp' terminated.

I have libssl installed, but probably not a compatible version, so I'll try to build a Debian VM. I noted that while running node starts listen on ports 12494 (LISTEN) and 32066 (LISTEN), I wonder if it could be related.

brendandburns commented 3 years ago

@abelsromero thanks for pursuing this. Definitely no obligation if you get bored/stuck I will get it done, but the assist is much appreciated!

abelsromero commented 3 years ago

Seeeing the interdependencies between tools and how these prevents other converters. I wonder if it would make sense to have different Dockerfiles. I know it would mean some duplication, but the fact that for one language we have to block the mantainance of of al the rest seems a high cost to me.

brendandburns commented 3 years ago

@abelsromero probably good observation. We could split them out and have them all depend on a shared base image, which would largely eliminate the duplication.

If this is blocking some work on java generation, let me know and I will increase the priority of fixing this.

k8s-ci-robot commented 3 years ago

@abelsromero: PR needs rebase.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
k8s-triage-robot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

k8s-triage-robot commented 3 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot commented 2 years ago

@k8s-triage-robot: Closed this PR.

In response to [this](https://github.com/kubernetes-client/gen/pull/184#issuecomment-927021201): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.