search

FIXTradingCommunity / fix-orchestra

Machine readable rules of engagement
Apache License 2.0
72 stars 35 forks source link

[repositoryValidator] xi:include should be supported #112

Open YuvalCohen opened 3 years ago

YuvalCohen commented 3 years ago

There should be a simple way to support xi:include in the xsd repository files.

This is a basic requirement for Orchestra to serve any project which is slightly larger then FIX repository and requires to split the repository to multiple xsd files.

At the moment, repositoryValidator report an error: Attribute 'xml:base' is not allowed to appear in element '...' (name of the element where the xi:include was added)

Further information about this error can be found in:

[https://stackoverflow.com/questions/22774425/attempting-to-connect-xml-files-with-xinclude-attribute-xmlbase-error/22791471#22791471] and way to supress the error can be found in: [http://xerces.apache.org/xerces2-j/features.html#xinclude.fixup-base-uris]

Thanks in advance. The repositoryValidator is a very useful tool.

kleihan commented 3 years ago

Agree with the proposal for the next version of Orchestra. SBE V1.0 supports the import of encodings, see Message Schema - Data encodings - Encoding name - Importing encodings (Chapter 6.5.2.1 in PDF).

donmendelson commented 3 years ago

@YuvalCohen the Orchestra repository schema does both things recommended by that article. It imports xml.xsd and it declares xml:base in top-level elements under <repository>. Unfortunately, Xinclude requires that attribute in every element that you wish to include; you can't include at random locations. Are you trying to include at the top-level, or at lower levels of the schema?

Also, RepositoryValidator sets a switch to make the parser XIncludeAware, another requirement. I'll have to test validator with includes to see if it works as expected.

YuvalCohen commented 3 years ago

@donmendelson The requirement si to be able to split large and complicated repository to multiple files. I believe that it is basic requirement for any future use of Orchestra (i.e. scalable).

At the moment, it seems that repositoryValidator does not like xi:include. We are happy to follow any guidelines or methods that will enable us to have each message, component, and/or codeSet in separated files. So far we tried with xi:include which is not welcome by the repositoryValidator.

donmendelson commented 3 years ago

I altered a unit test for RepositoryValidator as follows and got it to run with an Xinclude:

Modified test resource OrchestraFIXLatest.xml and moved a portion to another file.

  1. Added xmlns:xi="http://www.w3.org/2001/XInclude" to root element.
  2. Replaced <fixr:datatypes> element with <xi:include href="src/test/resources/datatypes.xml"/> (path is tricky with xinclude)
  3. Created datatypes.xml with contents of <fixr:datatypes>
  4. Added namespace to the included file as <fixr:datatypes xmlns:fixr="http://fixprotocol.io/2020/orchestra/repository">

The output did produce some errors or warnings, but they seem to be correct, not a problem with include instructions.

YuvalCohen commented 3 years ago

@donmendelson Thanks. Is there a possibility to write: a (single) message, (a single) component and (a single) codeset in dedicated files (i.e. each in its 'own file') and include all of them in the main repository file?

If so, can you please provide some description about what each of these files needs to contain?

donmendelson commented 3 years ago

@YuvalCohen since XIinclude requires explicit include points to be declared in the schema, what you ask requires a modification. We are about to promote Orchestra version 1.0 to Technical Standard, but we can take your request as a requirement for version 1.1.

donmendelson commented 3 years ago

Now that Orchestra version 1.0 is Technical Standard, we should reconsider all requests for schema changes to become version 1.1.

kleihan commented 1 year ago

Added the following to v1.1 RC1 section 3.1.4 Support for XInclude:

For example, <fixr:datatypes> element can be replaced with the path to the XML file, e.g. <xi:include href="src/test/resources/datatypes.xml"/>. The datatypes then need to be defined in a separate file datatypes.xml that needs to contain the same namespace as the root element of the repository, i.e. <fixr:datatypes xmlns:fixr="http://fixprotocol.io/2023/orchestra/repository">.

kleihan commented 9 months ago

This issue only updates the documentation of V1.1 RC1. There is currently no requirement to separate out individual elements of the top elements supporting XInclude (actors, concepts, sections, categories, messages, groups, components, fields, code sets, datatypes, scenarios). If the need arises, a separate issue should be entered and can be addressed in a later RC of v1.1.

The issue remains open to address the bug in the Repository Validator tool.