[GO] Problem with generating go client code, too many structs re-declared

[wari]

Description

I'm unable to run the following:

java -jar ~/Downloads/openapi-generator-cli-4.0.0-20180728.030217-16.jar generate -i https://products.solace.com/download/PUBSUB_SEMPV2_SCHEMA_JSON -g go

go get -v ./...                                                                                                                                                                                    
github.com/SolaceDev/semp-v2-go
# github.com/SolaceDev/semp-v2-go
./api_msg_vpn.go:149:6: CreateMsgVpnAclProfileOpts redeclared in this block
        previous declaration at ./api_acl_profile.go:41:6
./api_msg_vpn.go:260:6: CreateMsgVpnAclProfileClientConnectExceptionOpts redeclared in this block
        previous declaration at ./api_acl_profile.go:152:6
./api_msg_vpn.go:372:6: CreateMsgVpnAclProfilePublishExceptionOpts redeclared in this block
        previous declaration at ./api_acl_profile.go:264:6
./api_msg_vpn.go:484:6: CreateMsgVpnAclProfileSubscribeExceptionOpts redeclared in this block
        previous declaration at ./api_acl_profile.go:376:6
./api_msg_vpn.go:595:6: CreateMsgVpnAuthorizationGroupOpts redeclared in this block
        previous declaration at ./api_authorization_group.go:41:6
./api_msg_vpn.go:705:6: CreateMsgVpnBridgeOpts redeclared in this block
        previous declaration at ./api_bridge.go:41:6
./api_msg_vpn.go:817:6: CreateMsgVpnBridgeRemoteMsgVpnOpts redeclared in this block
        previous declaration at ./api_bridge.go:153:6
./api_msg_vpn.go:931:6: CreateMsgVpnBridgeRemoteSubscriptionOpts redeclared in this block
        previous declaration at ./api_bridge.go:267:6
./api_msg_vpn.go:1045:6: CreateMsgVpnBridgeTlsTrustedCommonNameOpts redeclared in this block
        previous declaration at ./api_bridge.go:381:6
./api_msg_vpn.go:1157:6: CreateMsgVpnClientProfileOpts redeclared in this block
        previous declaration at ./api_client_profile.go:41:6
./api_msg_vpn.go:1157:6: too many errors

openapi-generator version

openapi-generator-cli-4.0.0-20180728.030217-16.jar including earlier release versions (and swagger 2.4.0 as well)

OpenAPI declaration file content or url

https://products.solace.com/download/PUBSUB_SEMPV2_SCHEMA_JSON

Command line used for generation

java -jar ~/Downloads/openapi-generator-cli-4.0.0-20180728.030217-16.jar generate -i https://products.solace.com/download/PUBSUB_SEMPV2_SCHEMA_JSON -g go

Steps to reproduce

java -jar ~/Downloads/openapi-generator-cli-4.0.0-20180728.030217-16.jar generate -i https://products.solace.com/download/PUBSUB_SEMPV2_SCHEMA_JSON -g go
go get -v ./...

Related issues/PRs

Might be similar: https://github.com/OpenAPITools/openapi-generator/issues/535

Suggest a fix/enhancement

None I can think of at the moment

[tomi77]

It's probably problem with tags. Creates the same code for acl_profile and ms_vpn tags.

[wari]

A lot of the Opts struct are redeclared and it's everywhere, not just in acl_profile and msg_vpn:

$ grep -h "type .* struct" * | grep Opts  | sort | uniq -c | clipcopy
      2 type CreateMsgVpnAclProfileClientConnectExceptionOpts struct {
      2 type CreateMsgVpnAclProfileOpts struct {
      2 type CreateMsgVpnAclProfilePublishExceptionOpts struct {
      2 type CreateMsgVpnAclProfileSubscribeExceptionOpts struct {
      2 type CreateMsgVpnAuthorizationGroupOpts struct {
      2 type CreateMsgVpnBridgeOpts struct {
      2 type CreateMsgVpnBridgeRemoteMsgVpnOpts struct {
      2 type CreateMsgVpnBridgeRemoteSubscriptionOpts struct {
      2 type CreateMsgVpnBridgeTlsTrustedCommonNameOpts struct {
------ Lots of lines removed here ------
      2 type UpdateMsgVpnReplicatedTopicOpts struct {
      2 type UpdateMsgVpnRestDeliveryPointOpts struct {
      2 type UpdateMsgVpnRestDeliveryPointQueueBindingOpts struct {
      2 type UpdateMsgVpnRestDeliveryPointRestConsumerOpts struct {
      2 type UpdateMsgVpnTopicEndpointOpts struct {
$ grep -h "type .* struct" * | sort | uniq -c | grep " 2 " | wc -l        
106

The thought of manually removing those is a headache!

[tomi77]

Workaround is commenting / removing tags. But this is not a solution…

[wing328]

What about an option to only generate code for the 1st tag (assuming there're multiple tags defined in the operation)?

[wing328]

@wari can you please also try another client generator to see if you experience similar issues?

[wari]

Hi @wing328 what is the option to generate the first tag? I've tested this with Swagger (released and unreleased) and I got the same problem as well. I've not reported to swagger yet though. Are there any other client generators I can try? I only know of openapi and swagger

[wing328]

Sorry for client generator, I mean Python, Ruby, PHP client generators instead of the Go client generator.

If you manually clean up the tags to keep the first one only, does it solve your problem?

[wari]

• Python - Running python setup.py test has no failures
• Ruby - rake spec 1056 examples, 0 failures
• Bash - I've tested with this one before without failures.
• In the Java generated code I can see duplicates, but since they are in their own class, this is not a problem, only that the IDE complains. Anyway, gradle test failed when I removed the \@Ignore lines on the test files. Not sure if that helps.

As for removing the duplicates manually, it should work, as the options contain about 1 or 4 options of similar types, for example:

grep type\ GetMsgVpnTopicEndpointsOpts\  struct * -A 5
api_msg_vpn.go:type GetMsgVpnTopicEndpointsOpts struct {
api_msg_vpn.go-    Count optional.Int32
api_msg_vpn.go-    Cursor optional.String
api_msg_vpn.go-    Where optional.Interface
api_msg_vpn.go-    Select_ optional.Interface
api_msg_vpn.go-}
--
api_topic_endpoint.go:type GetMsgVpnTopicEndpointsOpts struct {
api_topic_endpoint.go-    Count optional.Int32
api_topic_endpoint.go-    Cursor optional.String
api_topic_endpoint.go-    Where optional.Interface
api_topic_endpoint.go-    Select_ optional.Interface
api_topic_endpoint.go-}

[wari]

Just wondering if this will get fixed or I will have to remove the duplicates manually whenever I generate code from this particular API.

[wing328]

@wari I'll take another look later today

[ity42]

Same symptoms here with another 3.0 spec file. Further example needed for analysis ? (sorry, I am a beginner to openapi-generator only able to provide another example, not any further insights)

[wari]

Thanks, @ity42 for reporting, I know I'm not the only here with this issue. Do you have the URL of the YAML or JSON file?

[tomi77]

Whats about this: Create *Opts structs in model_*_opts.go files. for example:

• GetAboutUserOpts move to model_about_user_opts.go
• GetAboutUserMsgVpnOpts move to model_about_user_msg_vpn_opts.go
• GetAboutUserMsgVpnsOpts move to model_about_user_msg_vpns_opts.go and so on…

[ity42]

@wari: sorry, indirect link only; another example is: psd2-api 1.2 Update 2018-08-17.yaml on https://www.berlin-group.org/nextgenpsd2-downloads (I just tried it with openapi-generator-cli-3.3.1-20181008.101417-13.jar, still same symptoms)

[akutz]

I'm still hitting this issue with the Golang generator for the HAProxy dataplane API. You can download their OpenAPI spec file from the top of the linked page (a direct link isn't working).

Total

$ grep -h "type .* struct" *.go | grep Opts | sort | uniq -c | awk '{s+=$1} END {print s}'
255

Details

$ grep -h "type .* struct" *.go | grep Opts | sort | uniq -c | grep -v '^[[:space:]]\{0,\}1'
      4 type CreateAclOpts struct {
      2 type CreateBackendOpts struct {
      3 type CreateBackendSwitchingRuleOpts struct {
      3 type CreateBindOpts struct {
      4 type CreateFilterOpts struct {
      2 type CreateFrontendOpts struct {
      4 type CreateHTTPRequestRuleOpts struct {
      4 type CreateHTTPResponseRuleOpts struct {
      4 type CreateLogTargetOpts struct {
      3 type CreateServerOpts struct {
      3 type CreateServerSwitchingRuleOpts struct {
      3 type CreateStickRuleOpts struct {
      4 type CreateTCPRequestRuleOpts struct {
      4 type CreateTCPResponseRuleOpts struct {
      4 type DeleteAclOpts struct {
      2 type DeleteBackendOpts struct {
      3 type DeleteBackendSwitchingRuleOpts struct {
      3 type DeleteBindOpts struct {
      4 type DeleteFilterOpts struct {
      2 type DeleteFrontendOpts struct {
      4 type DeleteHTTPRequestRuleOpts struct {
      4 type DeleteHTTPResponseRuleOpts struct {
      4 type DeleteLogTargetOpts struct {
      3 type DeleteServerOpts struct {
      3 type DeleteServerSwitchingRuleOpts struct {
      3 type DeleteStickRuleOpts struct {
      4 type DeleteTCPRequestRuleOpts struct {
      4 type DeleteTCPResponseRuleOpts struct {
      4 type GetAclOpts struct {
      4 type GetAclsOpts struct {
      2 type GetBackendOpts struct {
      3 type GetBackendSwitchingRuleOpts struct {
      3 type GetBackendSwitchingRulesOpts struct {
      2 type GetBackendsOpts struct {
      3 type GetBindOpts struct {
      3 type GetBindsOpts struct {
      2 type GetDefaultsOpts struct {
      4 type GetFilterOpts struct {
      4 type GetFiltersOpts struct {
      2 type GetFrontendOpts struct {
      2 type GetFrontendsOpts struct {
      2 type GetGlobalOpts struct {
      2 type GetHAProxyConfigurationOpts struct {
      4 type GetHTTPRequestRuleOpts struct {
      4 type GetHTTPRequestRulesOpts struct {
      4 type GetHTTPResponseRuleOpts struct {
      4 type GetHTTPResponseRulesOpts struct {
      4 type GetLogTargetOpts struct {
      4 type GetLogTargetsOpts struct {
      3 type GetServerOpts struct {
      3 type GetServerSwitchingRuleOpts struct {
      3 type GetServerSwitchingRulesOpts struct {
      3 type GetServersOpts struct {
      3 type GetStickRuleOpts struct {
      3 type GetStickRulesOpts struct {
      4 type GetTCPRequestRuleOpts struct {
      4 type GetTCPRequestRulesOpts struct {
      4 type GetTCPResponseRuleOpts struct {
      4 type GetTCPResponseRulesOpts struct {
      2 type PostHAProxyConfigurationOpts struct {
      4 type ReplaceAclOpts struct {
      2 type ReplaceBackendOpts struct {
      3 type ReplaceBackendSwitchingRuleOpts struct {
      3 type ReplaceBindOpts struct {
      2 type ReplaceDefaultsOpts struct {
      4 type ReplaceFilterOpts struct {
      2 type ReplaceFrontendOpts struct {
      2 type ReplaceGlobalOpts struct {
      4 type ReplaceHTTPRequestRuleOpts struct {
      4 type ReplaceHTTPResponseRuleOpts struct {
      4 type ReplaceLogTargetOpts struct {
      3 type ReplaceServerOpts struct {
      3 type ReplaceServerSwitchingRuleOpts struct {
      3 type ReplaceStickRuleOpts struct {
      4 type ReplaceTCPRequestRuleOpts struct {
      4 type ReplaceTCPResponseRuleOpts struct {

[akutz]

FWIW, I wrote a quick Makefile target to remove the duplicate options and fix any vestigial imports:

remove-dupe-opts: ## Removes duplicate Opt structs from the generated code
    @for struct in $$(grep -h 'type .\{1,\} struct' $(OUTPUT_DIR)/*.go | grep Opts  | sort | uniq -c | grep -v '^      1' | awk '{print $$3}'); do \
      for f in $$(/bin/ls $(OUTPUT_DIR)/*.go); do \
        if grep -qF "type $${struct} struct" "$${f}"; then \
          if eval "test -z \$${$${struct}}"; then \
            echo "skipping first appearance of $${struct} in file $${f}"; \
            eval "export $${struct}=1"; \
          else \
            echo "removing dupe $${struct} from file $${f}"; \
            tr '\n' '\r' <"$${f}" | sed 's~// '"$${struct}"'.\{1,\}type '"$${struct}"' struct {[^}]\{1,\}}~~' | tr '\r' '\n' >"$${f}.tmp"; \
            mv -f "$${f}.tmp" "$${f}"; \
          fi; \
        fi \
      done \
    done
    @$(GOLANGCI_LINT) run -v --no-config --fast=false --fix --disable-all --enable goimports $(OUTPUT_DIR)

[displague]

@wing328 In my case, I deleted the extra tags and that fixed the problem:

https://github.com/t0mk/gometal/pull/31 (https://github.com/t0mk/gometal/pull/31/commits/86e18b77cb27d463be8df8e22965d53645ffba05)

[displague]

The following jq command will remove any extra tags from a Swagger document (keeping the first tag):

cat swagger-v2.json  | jq '. | select(((.paths[][].tags| type=="array"), length) > 1).paths[][].tags |= [.[0]]'

Ideally, I would like to preserve the original tag list as x-tags. Any jq help with that would be appreciated!

[displague]

What about an option to only generate code for the 1st tag (assuming there're multiple tags defined in the operation)?

@wing328 I think an option to enable this behavior would fix the problem. I don't know if this tool should be so opinionated that it always chooses the first tag. Perhaps an optional x-openapitools-tag could offer the preferred tag? If that is not set, then just use the first tag because a client that works is better than one that does not.

[iafilius]

Hi, a me too report and some additional info.

Using mvn version openapi-generator-cli-5.2.0.jar and latest build openapi-generator-cli-5.2.1-20210719.023544-9.jar and have same (and more ) issues.

4.2.2 works BUT to prevent duplicate structs need to use -p enumClassPrefix=true

5.2.0 or 20210719.023544-9 versions seem to ignore it or some other mechanism generates lots of duplicates.

Can't remember i've seen it with an early 5.0.1 version, but that was almost a year ago when tested.

[vendor provided swagger file used] (https://vdc-download.vmware.com/vmwb-repository/dcr-public/d8e1559c-3fd9-4d48-a895-b1036479cea6/9e2aee74-1a4d-46ba-ac14-8694815b8ce2/swagger-4.4.0-dist.json)

java  -jar ../openapi-generator-cli-5.2.1-20210719.023544-9.jar generate --generate-alias-as-model -i ./swagger440_untagged.json -g go -o velocloudsdkgo440openapi521unt \
      --additional-properties=packageName="velocloudsdkgo440openapi521unt",packageVersion="0.0.0.1" \
      -p enumClassPrefix=true \
      --enable-post-process-file

To make it compilable without duplicates I need to modify the vendor provider with the suggested jq statement:

cat swagger-4.4.0-dist.json| jq '. | select(((.paths[][].tags| type=="array"), length) > 1).paths[][].tags |= [.[0]]' > swagger440_untagged.json

And with that it compiles.

Not sure if it affects anything else at this point. update:"Functions are no longer available in respective api collection (defined by tags), which again could be workaround by hardcoding all tags to same value, instead of first tag found. I found another open issue (#9500) which states based on tags different packages should be created".

PS: i need the -p enumClassPrefix=true to hide the issue with generating ModMap's which are not defined anywhere (which happens about just 6 times or so in the swagger spec file), this is for version 4.2.2, 5.2.0 and 5.2.1-20210917

Regards

[gunivan]

Hi, Confirmed that set structPrefix=true then the issue is resolved. I have tested with OpenApiGenerator gradle plugin version 5.3.1