Improve KsqlSerdeFactory for Public API Consumption

[agavra]

If KsqlSerdeFactory is going to become part of our public API then we should give it some love to make it more extensible, e.g.:

• Rather than passing config and SR clients as params, pass them via a Context object, which allows us to add more things to it later without needing to change the signature.
• Consider removing KSqlConfig from the method sigs completely, as this is specific to our serde classes. Instead, pluggable formats can implement Configurable if they need config. (We should limit what config is passed to 3rd party formats just like we do for UDFs e.g. we shouldn't be passing all config that may contain secrets etc).
  • move the type checking version of createSerde out of the interface.

Originally posted by @big-andy-coates in https://github.com/confluentinc/ksql/pull/4437

[big-andy-coates]

We should also look at the Format interface too. The recent work I've done removes any reference to Connect types. But I also think we should plan for extension with our public APIs.

For example, Format currently has two methods for converting between ParsedSchema and List<SingleColumn>. I recently added FormatInfo as a parameter to one of these. In the future that would be a breaking change. Hence we should pass some kind of Context object as a parameter. This provides a way of passing new things to these methods without needing to change the syntax of the method or requiring existing user's custom serde classes to change.

e.g.

  default List<SimpleColumn> toColumns(ParsedSchema schema, ToColumnContext context) {
    throw new KsqlException("Format does not implement Schema Registry support: " + name());
  }

  default ParsedSchema toParsedSchema(List<SimpleColumn> columns, FromColumnContext context) {
    throw new KsqlException("Format does not implement Schema Registry support: " + name());
  }

Initially, both ToColumnContext and FromColumnContext can have a single getter to obtain the FormatInfo. In the future, more getters can be added without requiring a signature change,