Generation of Adapter Coverage matrix

[al-fisher]

The idea here is to auto-generate tables through reflection - not too dissimilar to the implementation columns of BHoM object relation tables in wikis https://github.com/BHoM/Robot_Toolkit/wiki/BHoM-Robot-object-relations https://github.com/BHoM/GSA_Toolkit/wiki/BHoM---GSA-object-relations https://github.com/BHoM/Lusas_Toolkit/wiki/BHoM-Lusas-Object-Relations etc.

i.e.

• Object types supported by each Adapter for
  • Create
  • Read
  • Update
  • Delete

This will enable combining with the Continuous Integration cycle and auto-generation of documentation as part of BHoM API creation #23

In addition potential to add additional information captured through CI Interoperability Testing on a level of granularity refering to each adapter, object type and operation.

@FraserGreenroyd @IsakNaslundBh FYI

[alelom]

Sorry, to make sure I understand, the idea here is to find a way (through reflection) to auto-generate tables like this, for each single toolkit?

[alelom]

If so, I agree it could be really useful.

[al-fisher]

Sorry, to make sure I understand, the idea here is to find a way (through reflection) to auto-generate tables like this, for each single toolkit?

Yes - exactly that!

Individual tables - or also aggregate together into one large table/matrix to help user explore what interoperabilty capability is implemented - sorting by Object Type or Operation across all Adapters say. 🚀

[al-fisher]

Additionally - just to get on your radar also @alelom if not already - we can use the clear organisation of the Test Datasets to expose in real granularity, details of CRUD operations implemented (and/or successful following interoperability testing) for each Type. See https://github.com/BHoM/Test_Toolkit/tree/master/Datasets/TestSets/Structure/Bar as example

[alelom]

@al-fisher I think we need to have a conversation on this. Generally, I think that this automatic generation might not be advisable as a way of documenting the actual CRUD compatibility situation, at least for Update and Delete.

While Create and Read methods are probably safe to take as representatives of the actual support situation, this is less so for Delete, and almost never for Update.

Update would require checking if the Adapter is not making use of the "Create then Delete" mechanism for some or all types.

Delete is yet another case: take GSA for example. There, Delete is supported for virtually any type, but given how the software API works, we don't have a single Delete method for each single type.

This could spark the question "How is this different from the BHoM API generation"? First, we did in fact avoid including Update and Delete methods in there, for this same reason. Secondly, the API is an automatically produced document and people accessing that know it. Producing actual markdown documentation to be included in the Wiki would imply a human-level of attention. Third, actually manually writing that documentation in Wiki form provides the opportunity for the implementer to record notes along the simple tickboxes, for example to say "Update of this object works but only given certain conditions...".

Let's discuss at your convenience.

[al-fisher]

Yes - needs a bit of workshopping around this - agreed. You raise some good points above on why this is non-trivial. I would however keep a short/medium term primary objective that all of the issues you mention above can somehow be addressed such that "what is possible with each adapter" is clearly and transparently documented. A few ideas to muse before discussing in detail.

Delete is supported for virtually any type, but given how the software API works, we don't have a single Delete method for each single type.

If code can not be (or we do not want to) structure to directly reflect what is supported. Quantifying what is possible could therefore be achieved automatically though unit/interoperability testing. Trying and measuring success of deleting each object type through given adapter. These test reports can feed into in some form part of the API or similar

manually writing that documentation in Wiki form provides the opportunity for the implementer to record notes along the simple tickboxes, for example to say "Update of this object works but only given certain conditions...".

Yep - I think this is necessary. Whether it is Wiki - or closely embedded in code as Descriptions, comments and attributes - mining human crafted information to feed into API rather than simply only a "blind" computer generated document is where we need to be, to expand usefulness. I do not see this too differently from the hard work being put into Object and Property Descriptions. Keen we do not, if possible, rely heavily on wiki type platforms for specific functionality reporting long term. Will by definition be out of date. Better for high level documentation, guidance, typical usage. I think. So finding a good balance between:

1. Code architecture.
2. Functionality validation through testing
3. Well crafted descriptions/notation

all scraped and clearly combined together would be my ideal.

Let's chat 😄