More Than Two Dimensions

[engsbk]

@caio-davi Thank you for the beneficial discussions so far!

I'm expanding the burgers example to 3D (2D space and 1D time), but things got a bit confusing for me for the initial and boundary conditions.

I try to define them as follows:

# initial condition points
x0 = tf.expand_dims(tf.convert_to_tensor(ux, dtype=tf.float32), -1)
y0 = tf.expand_dims(tf.convert_to_tensor(uy, dtype=tf.float32), -1)
xx0, yy0 = np.meshgrid(x0,y0)
XY0 = np.vstack((np.ravel(xx0), np.ravel(yy0))).T
# X0 = XY0.reshape(100,100) 
X0 = xx0
t0 = tf.zeros(tf.shape(X0), dtype=tf.float32)
print ("t0",t0.shape)
u0 = tf.zeros(tf.shape(t0), dtype=tf.float32)
print ("u0",u0.shape)

# Dirichlet boundary condition points
xlb = tf.expand_dims(xlo * tf.ones(tf.shape(ut), dtype=tf.float32), -1)
xub = tf.expand_dims(xhi * tf.ones(tf.shape(ut), dtype=tf.float32), -1)
ylb = tf.expand_dims(ylo * tf.ones(tf.shape(ut), dtype=tf.float32), -1)
yub = tf.expand_dims(yhi * tf.ones(tf.shape(ut), dtype=tf.float32), -1)
tlb = tf.expand_dims(tf.convert_to_tensor(ut, dtype=tf.float32), -1)
tub = tf.expand_dims(tf.convert_to_tensor(ut, dtype=tf.float32), -1)

uxlb = tf.expand_dims(tf.zeros(tf.shape(ut), dtype=tf.float32), -1) #tf.expand_dims(tf.math.sin(freq * np.pi * tf.convert_to_tensor( ut, dtype=tf.float32)), -1)
uxub = tf.expand_dims(tf.zeros(tf.shape(ut), dtype=tf.float32), -1)
uylb = tf.expand_dims(tf.zeros(tf.shape(ut), dtype=tf.float32), -1) #tf.expand_dims(tf.math.sin(freq * np.pi * tf.convert_to_tensor( ut, dtype=tf.float32)), -1)
uyub = tf.expand_dims(tf.zeros(tf.shape(ut), dtype=tf.float32), -1)

and modified the training functions as follows:

# initial condition points
x0 = tf.expand_dims(tf.convert_to_tensor(ux, dtype=tf.float32), -1)
y0 = tf.expand_dims(tf.convert_to_tensor(uy, dtype=tf.float32), -1)
xx0, yy0 = np.meshgrid(x0,y0)
XY0 = np.vstack((np.ravel(xx0), np.ravel(yy0))).T
# X0 = XY0.reshape(100,100) 
X0 = xx0
t0 = tf.zeros(tf.shape(X0), dtype=tf.float32)
print ("t0",t0.shape)
u0 = tf.zeros(tf.shape(t0), dtype=tf.float32)
print ("u0",u0.shape)

# Dirichlet boundary condition points
xlb = tf.expand_dims(xlo * tf.ones(tf.shape(ut), dtype=tf.float32), -1)
xub = tf.expand_dims(xhi * tf.ones(tf.shape(ut), dtype=tf.float32), -1)
ylb = tf.expand_dims(ylo * tf.ones(tf.shape(ut), dtype=tf.float32), -1)
yub = tf.expand_dims(yhi * tf.ones(tf.shape(ut), dtype=tf.float32), -1)
tlb = tf.expand_dims(tf.convert_to_tensor(ut, dtype=tf.float32), -1)
tub = tf.expand_dims(tf.convert_to_tensor(ut, dtype=tf.float32), -1)

uxlb = tf.expand_dims(tf.zeros(tf.shape(ut), dtype=tf.float32), -1) #tf.expand_dims(tf.math.sin(freq * np.pi * tf.convert_to_tensor( ut, dtype=tf.float32)), -1)
uxub = tf.expand_dims(tf.zeros(tf.shape(ut), dtype=tf.float32), -1)
uylb = tf.expand_dims(tf.zeros(tf.shape(ut), dtype=tf.float32), -1) #tf.expand_dims(tf.math.sin(freq * np.pi * tf.convert_to_tensor( ut, dtype=tf.float32)), -1)
uyub = tf.expand_dims(tf.zeros(tf.shape(ut), dtype=tf.float32), -1)

def loss_grad():
    def _loss(w, b):
        with tf.GradientTape(persistent=True) as tape:
            tape.watch(w)
            tape.watch(b)
            loss_value = loss(w, b)
        trainable_variables = w + b
        grads = tape.gradient(loss_value, trainable_variables)
        return loss_value, grads

    return _loss

def run_swarm(swarm, X):
    new_swarm = []
    for particle in swarm:
        w, b = decode(particle, layer_sizes)
        new_swarm.append(
            multilayer_perceptron(w, b, X_flat.astype(np.float32))
        )
    return tf.convert_to_tensor(new_swarm, dtype=tf.float32)

but when I tried to run:

opt = pso(
    loss_grad(),
    layer_sizes,
    n_iter,
    pop_size,
    B,
    c1,
    c2,
    initialization_method="lecun",
    verbose=True,
    gd_alpha=1e-4,
)

I got this error:

\pso-pinn-main\src\swarm\utils.py", line 124, in multilayer_perceptron  *
        H = tf.nn.tanh(tf.add(tf.matmul(H, W), b))

    ValueError: Dimensions must be equal, but are 102 and 3 for '{{node MatMul}} = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false](sub_1, w)' with input shapes: [100,102], [3,50].

Any ideas on how to work around this? Thanks again!

[caio-davi]

It's hard to say without seeing all the script and error logs. But I'm assuming you are having trouble in the forwarding step, right? How did you declare layer_sizes?

[engsbk]

layer_sizes = [3] + 5 * [50] + [1]

I can write a more straightforward example for the wave equation and share the ipynb, if it can facilitate our discussion. If you think that is a good idea, I can upload it now.

[caio-davi]

The thing is that I'm very busy for the next 2/3 weeks, so I can't commit to helping you bc if it is something that takes too long, I can't afford the time to spare. I'm sorry. But if you can wait, I'll be more than happy to help, though! If you like, you can share the notebook, and I'm going to take a look at it as soon as I have a chance.

[caio-davi]

Your layer_sizes look good, btw. The issue seems to happen in the first layer, by the dimensions on the log. But we need to dig further to have a better understanding of it.