Theta being concatenated with X?

[cfos3120]

In all your models, the model parameters u_p is concatenated with the grid x as an initial operation in any forward pass (before being passed through an MLP and then Attention Blocks). The model architecture diagram in the paper and readme indicates that theta should be processed as keys and values but wouldn't this conflate it as a query instead?

x = torch.cat([x, u_p.unsqueeze(1).repeat([1, x.shape[1], 1])], dim=-1)

Note: for the ns2d_1100_test.pkl file (rectangular grid with circular cavities), the theta array has a value of all zeros too, what information is this holding? No initial conditions or boundary conditions (inlet outlet) are specified in the model or data either. Were these inferred through training only?

[HaoZhongkai]

Hi, Yes, in our paper, we process theta the same with other input functions and apply attention to it. But later we found that mathematically just concatenating them equals processing theta with attention. Thus concat is an easier code implementation and we test that there is nearly no performance gap. In the NS2d1_channel dataset, we want the model to be able to infer the positions of the obstacles, so theta is an empty array (actually it shows the positions and radius of circles).

best regards, Zhongkai Hao