Persist state across modules with Template Haskell

[noughtmare]

I recently tried to find ways to do this too. Here's my write-up: https://discourse.haskell.org/t/splitting-function-definitions-across-modules/2327

I have since refined it a bit: you can basically use the addModFinalizer and addTopDecls functions to create a top level variable that stores your "module" at the end of the Template Haskell run of each module. Then you can import that variable in another module and use the contents there.

There are some limitations: the data that you want to store must be Liftable or serializable and users must either manually import and export these "module" variables or use implicit exports, because there is no way to add this "module" variable to the export list with Template Haskell (I think it would not be too hard to add such a feature to GHC; I found this ticket: https://gitlab.haskell.org/ghc/ghc/-/issues/1475 but that does more than I need).

I created my splitfuns project to show off these ideas. The auto branch is the most advanced: it allows both manual and automatic "module" management (the example uses the automatic approach, but you can combine the two).

[parsonsmatt]

That is a fascinating approach! Thanks for sharing it.

[noughtmare]

I just tried combining using it with compact and knob to remove the serializable/liftable constraint. That fails with incremental compilation because the compact region can only be deserialized in the same process that created it. Incremental compilation implies that the compiler is run at least twice once process after the other. Which causes segfaults because the second process cannot read the compact region created by the first process. The documentation mentions that you can work around that by using static linking and turning off ASLR, but that doesn't sound user-friendly.

[noughtmare]

I just discovered reifyAnnotations. You can add annotations directly to the module with

addTopDecls [PragmaD (AnnP ModuleAnnotation (SigE yourLiftedValue (ConT ''YourType)))]

and read those annotation from another module using

md <- thisModule
ModuleInfo mds <- reifyModule md
for_ mds $ \md' -> do
  anns <- reifyAnnotations @YourType (AnnLookupModule md')
  ... do something with anns ...

That is probably much more robust and it removes the need for the user to be careful around exporting and importing. It also doesn't require serialization, only that the type you want to share between modules is an instance of Data and Lift (which you might call serialization, but at least they are standard and easily derived). I have updated the auto branch of splitfuns: https://github.com/noughtmare/splitfuns/commit/2dd28b3655bca621a8a617d29a30159e08814bf5