Analysis tools/source code readers are buggy

[Nathaniel-Hu]

The current version of the Data Table Generator (in /scripts/) uses a basic (albeit likely buggy) source code reader (SourceCodeReader.hs). This reader is used to extract the data, newtype and class types defined/used in all Haskell scripts stored in /code/.

A more reliable option would be to switch over to using the GHC API to parse the Haskell scripts and extract the data, newtype and class types defined/used. This will most likely require some knowledge in using the GHC API. Some additional resources include the modern toolkit ghc-lib-parser (documentation) and its source code.

Note: the current version of the Data Table Generator (in /scripts/) is currently contained in this branch, and is awaiting merge approval in this pull request. Note: pull request has been merged.

[Ant13731]

Update for this issue: There doesn't really seem to be a lot of documentation on how to actually implement the ghc api, but this stack overflow post might be a decent start. This one might be relevant too.

The SourceCodeReader.hs file has also been split:

• SourceCodeReaderCI.hs for class-instances
• SourceCodeReaderT.hs for data type dependencies

[balacij]

I think @tingyuw is currently experiencing this issue with #3328.

While using GHC tooling to parse the data is feasible, there are other solutions too. Hence I'm switching the ticket from 'enhancement' to 'bug'. For example, the scripts have been working thus far, so they could be properly analyzed for their issues. Alternatively, if/when we 'de-embed Drasil' (#3003), then this ticket should be trivial to solve using our own native tooling.

[balacij]

2211 should also be completed as part of this work.