Test `--use-santised-oas` with the OpenAPI generated connectors

[TharmiganK]

Description:

With Ballerina SwanLake Update 10(2201.10.0), a new flag: --use-sanitized-oas has been introduced to sanitise the names given in the OpenAPI specification according to the Ballerina naming conventions. For more information, see Enhance generated record name sanitisation for OAS schema name. This has been introduced as a experimental feature.

Since this is a requirement for this [Proposal] Convert the OAS to Ballerina-friendly naming conventions, we need to test this with the connectors and remove the experimental flag.

Describe your task(s)

• [x] Test the feature with the Ballerina OpenAPI generated connectors given in,
  • [x] the openapi-connectors repository
  • [x] the connectors listed under the Ballerina generated connector modules list
• [x] Identify the issues

[TharmiganK]

A part of testing has been done @lnash94 by checking the Ballerina OpenAPI generated connectors in the openapi-connectors repository. Please find the summary below: (For brevity the connectors with issues has been listed)

• Issues in code generation:

connector	cause of the issue
insightly.custom	The reference in the items is not updated for the component schema generated for this
iris.helpdesk	The reference in the items is not updated for the component schema generated for this
karbon	The references in the items are not updated for the component schema of the modified oas
magento.cart	The references in the items are not updated for the component schema of the modified oas
paylocity	The references in the items are not updated for the component schema of the modified oas
pushcut	The references in the items are not updated for the component schema of the modified oas
quickbase	The references in the items are not updated for the component schema of the modified oas
siemens.iotandstorage.iotfileservice	The references in the items are not updated for the component schema of the modified oas
sinch.sms	The references in the items are not updated for the component schema of the modified oas
vimeo	The references in the items are not updated for the component schema of the modified oas
wordpress	The references in the items are not updated for the component schema of the modified oas
beezup.merchant	The references in the items are not updated for the component schema of the modified oas

• Issues during the compilation of the generated client:

connector	cause of the issue
iris.residuals	The sanitised OAS have incorrect object schemas which ends up in a BadSadError
iris.disputeresponder	The sanitised OAS have incorrect object schemas which ends up in a BadSadError
iris.subscriptions	The sanitised OAS have incorrect object schemas which ends up in a BadSadError
leanix.integrationapi	The sanitised OAS have incorrect object schemas which ends up in a BadSadError
openfigi	The sanitised OAS have incorrect object schemas which ends up in a BadSadError
orbitcrm	The sanitised OAS have incorrect object schemas which ends up in a BadSadError

In addition to these failures it was observed that the names are changed to sentence case even when the name is align with the Ballerina convention with camel case. For example, ChildList is converted to Childlist

[TharmiganK]

With this change, the generation worked without any issues for all the connectors in the openapi-connectors repository.

But found compilation errors during bal build:	connector	cause of the issue
adobe.analytics	a field defined with type true has a default value as false - schema
amadeus.flightoffersprice	header record with required field is used with default value({}) in the resource function - header parameter. The header param has a default value in the schema but it is not populated in the generated type
amadeus.flightofferssearch	Same as amadeus.flightoffersprice
bitly	a query with default integer value is generated with a string default value - query parameter
box	default value for an enum has spelling mistake - default value and there is a missing single backtick token lang warning for this comment
browshot	a query parameter has a name headers which conflicts with the auto generated headers parameter - query parameter
commercetools.cartsordershoppinglists	an operation with path parameter like key=<value> is also mapped to string - operation
commercetools.configuration	Same as commercetools.cartsordershoppinglists
commercetools.customer	Same as commercetools.cartsordershoppinglists
commercetools.customizebehavior	Same as commercetools.cartsordershoppinglists
commercetools.customizedata	Same as commercetools.cartsordershoppinglists
commercetools.pricingdiscount	Same as commercetools.cartsordershoppinglists
commercetools.productcatalog	Same as commercetools.cartsordershoppinglists
covid19	two operations has conflicting resource paths - operation 1 and operation 2
eloqua	a header parameter with default value is not mapped in the generated headers record - header parameter
facebook.ads	two operations has conflicting resource paths - operation 1 and operation 2
hubspot.files	conflicting resource function signatures
kinto	default value for a record is created as an string: "{}" - type schema
pagerduty	default values for the headers are not mapped in the generated headers parameter
trello	conflicting resource function signatures
webscraping.ai	a query parameter has a name headers which conflicts with the auto generated headers parameter - query parameter
whatsapp.business	generated client has a resource function body which calls setPayload with record type which is not supported in the HTTP package
wordnik	a query with default integer value is generated with a string default value - query parameter
xero.accounts	conflicting resource function signatures

[TharmiganK]

With this change, the following issues have been identified in the revamped generated connectors listed under the Ballerina generated connector modules list.

connector	cause of the issue
asana	A field of a included record is overwritten by the same type but including nil - AttachmentResponse.parent
candid	Security schema names are not sanitised - schema
dayforce	The generated query and header parameter names are not sanitised
docusign.dsadmin	The connectionConfig rest parameter is changed to required parameter. The json return is changed to record{}
docusign.dsclick	The connectionConfig rest parameter is changed to required parameter. The object schema with content-type - */* is represented as http:Response
docusign.dsesign	The connectionConfig rest parameter is changed to required parameter. The sanitised path parameter and a query parameter have the same name - langCode in the operation - ConsumerDisclosure_GetConsumerDisclosureEnvelopeIdRecipientIdLangCode. The setPayload method does not support anydata. The object schema with content-type - */* is represented as http:Response
github	The application/vnd.github.object content-type is mapped to http:Response.
openai.assistants	The default value for object - CreateChatCompletionRequest.logit_bias is generated as string - "{}"

[TharmiganK]

Summary

OpenAPI tool side issue:

• [x] Default value is generated as string. The string should be converted to the specific type according to the schema
• [x] Generated header record type does not populated the default values
• [x] Auto generated header parameter name conflicts with the existing query parameters
• [x] ~Security schema names are not sanitized~ - This is a part of headers which are not yet supported by the sanitization feature. This should be addressed via the new ballerina name extension
• [x] The generated type names for query and header parameters are not sanitized
• [x] ~The http:Response for the unsupported content-types should be analyzed~ - This is the correct behaviour

HTTP side issues:

• [x] The setPayload function does not support anydata

Lang side issue:

• [x] ~Getting the following warning: missing single backtick token when back ticks are used in comments~ - Created an issue in the lang side. This not a high priority issue
• [ ] ~Conflicting resource functions with the same method and path parameter but with different function parameters~ - This is not possible since Ballerina does not support function overloading, by design. Need to check whether it is possible to validate this and fallback to remote methods

[TharmiganK]

Closing this issue since the identified issues have been tracked via separate issues