Feature Request: Deferred Rendering

[fbartho]

Purpose

Sometimes you want data to be fetched from a remote server at publish time

Proposed Solution

A. Add an AsyncElement protocol / ElementType

protocol AsyncElement {
   func render(context: PublishingContext) async -> String
}

A benefit of this option is that it's pretty intuitive to the consumer. They should just block/await until their data is ready.

B. Provide a special API to register async queries

struct SomeExampleData: AsyncData {
/// fields from a public REST resources
}
public struct SomeExample: InlineElement {
  func loadData(context: PublishingContext) async throws -> any AsyncData {
    // Can access other member variables
    // Can access URLRequest
    return SomeExampleData(…)
  }
  func render(context: PublishingContext, runtimeData: any AsyncData?) -> String {
   guard let data = runtimeData else {
     return "SomeExampleData Missing"
   }
   // ...
  }
}

Alternatives Considered

A downside of providing any Async APIs is that poor implementation could lead to slow rendering. Each time an Async Component is Encountered the whole subtree of components that are children of that node might be invalidated. Additionally, peer or parent components could be invalidated if their rendering depends on how many children they have. Think how you would implement an Outline component that automatically handles all the headers, but is physically located at the top of the page? Additionally, we probably would want a timeout at some-level to finish rendering despite in-progress async requests.

• Not allowing async-data load, for consumers with this need, they'd have to pre-fetch their data to disk before executing Ignite's build, but this is a bit clunky, particularly if you want the dynamic data to be selected based on the Content in your pages
• I thought about a special element DataLoader that generically had an async method to fetch the data, but this didn't seem to be better than Option A above.

Additional Special Edge Case

If you wanted to build an AutomaticOutline component, you need a way for this component to depend on being run after other Async components have finished running. With either of the options above, we can ensure the data could be provided via an appropriate environment hook (see #2), but it doesn't have something to await against since new outline entries could be discovered when previous async components are rendered.

One way to modify the Option A would be:

protocol AsyncElement {
   // (…other things as before)

   public(get) var asyncRenderGroup: AsyncRenderGroup
   public(get) var asyncRenderRank: Int
}
enum AsyncRenderGroup {
  case `default`
  /// after other async elements in the `default` group
  case last
}
protocol DefaultAsyncElement: AsyncElement {}
extension DefaultAsyncElement {
  let asyncRenderGroup: AsyncRenderGroup = .default
  let asyncRenderRank: Int = 0
}

Obviously, if we go with Option B or some other solution, then a different technique might be needed for this edge case.

[twostraws]

I've been thinking about this. Right now I'm thinking the simplest option that also yields a great deal of power to users is to make page rendering async, so that developers can make their own pages fetch data etc in order to build the output, although the internal elements of Ignite aren't themselves async. This would take a very minor code change to Ignite, wouldn't break any existing pages written before async was introduced, and yet allows folks to build sites from dynamic, remote data just fine.

So, we'd change StaticPage, ContentPage, and TagPage to something like this:

@BlockElementBuilder func body(context: PublishingContext) async -> [BlockElement]

Would that solve a large part of your issue?

[fbartho]

I think that would solve for some use-cases, but would make it impossible to solve others, right?

Eg. If I want a component high up in the page to depend on the rendering of other components lower down in the page, I’m not sure how I would do it (other than hacky string manipulation).

Further, if I want to allow components anywhere in the hierarchy to inject resources into <HEAD>, only if that component needs the stylesheets, an async variant would be very nice.

Additionally, I’d like to make a custom FancyImage component that asynchronously examines a local asset, and dynamically generates different thumbnail images, or uses the image srcset attribute, or hardcodes the asset aspect ratio into the static page (so that there’s no layout flicker).

If I had async components then I could implement that feature as appropriate on a page-by-page basis

Perhaps this is a silly question, but since adding async appears to be a non-breaking change for pages, why wouldn’t allowing async components to coexist with sync components be possible?

[twostraws]

Perhaps this is a silly question, but since adding async appears to be a non-breaking change for pages, why wouldn’t allowing async components to coexist with sync components be possible?

I'm not saying it's impossible, just that making pages async is an initial step forward while we evaluate actual use cases from users. From what I can tell, adding async to pages would be purely additive: users with existing non-async pages won't need to change their code, but users who do want to do some custom code can use the async functionality as needed. If we then see many users reporting need for a larger AsyncComponent solution, that could be added as well – making pages async wouldn't preclude that.

[twostraws]

I haven't heard back further regarding this issue, so unless you can see a problem with adjusting StaticPage, ContentPage, and TagPage to use async, I'm going to go ahead and make that change.

[fbartho]

No objections; I've decided I feel inspired by what you've done here and have been making my own SSG from scratch -- (Driven by challenges threading in EnvironmentValues through the sync workflow in Ignite).

I've managed to make a way for Sync & Async components to be interleaved. I'm sure it'll never be as big a project as Ignite, but I thank you for the inspiration!

(Feel free to close my other requests if you feel so inclined).

[twostraws]

Understood – thank you, and best of luck! 🙌