Images in cnf_wells_rtms_not_working.jl are not sharp after tranining

[chuymtz]

Hello Rafael,

I was able to run the cnf_wells_rtms_not_working.jl script. However, Im not entirely sure what could be the problem with the model after doing the training. The images do not look very crisp at all. Any ideas of what could be the problem?

Using the provided compass data set, I feed seismic + wells

but the generated image is still very noisy.

Any thoughts what could be wrong? I suspect this is not a problem with the unet. Maybe i needed to train longer (200>)?

Thanks!

[rafaelorozco]

Hey! This actually rings a bell, so I think that we can not totally get rid of noisiness in the samples. Look at the image I posted that is closer to what i showed at ml4seismic. For sure the posterior mean helps to get rid of the noise.

Im going to go through the script now and see if I can tweak it to make the results closer to this figure. The script I sent was the minimal code that had a normalizing flow. I can send some updates in a few hours

Note also that the variance between samples is only noise that is something that I have been wanting to fix. I have some ideas on that side as well.

[chuymtz]

Thanks Rafael, noted! Im going through it as well maybe im missing something. Your images look a lot shareper than mine though (i say mine but its your work; i mean the ones i made from your script)

[chuymtz]

The posterior samples you show there look great. Im hoping i can get something similar in this dataset first.

[rafaelorozco]

Hey I worked on it and I made a few advances here: https://github.com/rafaelorozco/GenerativeGeostats.jl/blob/cnf_not_working/scripts/cnf_wells_rtm_working.jl

I played around with the CNF architecture a version that gets rid of most of the noise and doesnt need the unet so I just took it out.

After 500 epochs on a small training dataset of 100 it gives results like these

Still A lot of things I want to try out but this is a good starting point. I will keep you updated!

[chuymtz]

Hey this morning I noticed this

Was the problem that you were passing a sample from the latent space to the forward method?

I'll check it out! Thanks!

[chuymtz]

Hello,

Could you explain why this line is necessary?

#dont have split scales so need 2 channels
m_train = tensor_cat(m_train,m_train)
grad_train = tensor_cat(grad_train,grad_train)

Thanks!

[rafaelorozco]

Hey! By design the CNFs currently require multiple channels since the invertible coupling layer is based off of splitting along channel dimension.

Wth split_scales=true this problem resolves itself since split_scales artificially increases channel dimension with invertible downsampling but if split_scales=false you dont get this so you need to manually increase channel size.

The new results I got with split-scales=false I think are better than any of the previous iterations so I think it is worth exploring more.

[chuymtz]

Did you mean to use the funtion z_shape_simple_forward here

mask = Y .> 0
res = (mask.*X_gen .- Y) 
total_grad = res
#NOTE: total_grad =  z_shape_simple_forward(G, res)

When taking the residual? I get an error because G.Z_dims is nothing. I thinnk this was the intention. thanks

[rafaelorozco]

Hmm no, z_shape_simple_forward is only needed when split_scales=true. Since split_scales=false in this version then we dont need it anywhere

[chuymtz]

Hello rafael,

Sorry I'm still not able to fix the script to run. Im a little confused about two things.

• How come G.Z_dims is empty. Is this something that should be taken care of at the level of G.Z_dims[i] = collect(size(Z)). I ask because this a line in the z_shape_simple_forward but not in z_shape_simple. Hence the error I get is

julia>     Z_save, ZX = split_states(ZX_test[:], G.Z_dims)
ERROR: MethodError: no method matching length(::Nothing)
Closest candidates are:
  length(::Union{Base.KeySet, Base.ValueIterator}) at abstractdict.jl:58
  length(::Union{JLD2.Group, JLD2.JLDFile}) at ~/.julia/packages/JLD2/VWinU/src/JLD2.jl:500
  length(::Union{IterTools.Properties, IterTools.PropertyValues}) at ~/.julia/packages/IterTools/cLYFo/src/IterTools.jl:1006
  ...
Stacktrace:
 [1] split_states(Y::CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Z_dims::Nothing; L_net::Int64)
   @ InvertibleNetworks ~/.julia/packages/InvertibleNetworks/ushIF/src/utils/dimensionality_operations.jl:481
 [2] split_states(Y::CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Z_dims::Nothing)
   @ InvertibleNetworks ~/.julia/packages/InvertibleNetworks/ushIF/src/utils/dimensionality_operations.jl:480
 [3] top-level scope
   @ ~/projects/GenerativeGeostats.jl/src/utils_cnf.jl:30

Since

julia> G.Z_dims == nothing
true

[rafaelorozco]

Hey! Just to be clear, if you are running cnf_wells_rtm_working.jl then you should not be using either z_shape_simple or z_shape_simple_forward anywhere. Those functions are only applicable when split_scales=true but split_scales should be false

[chuymtz]

You're right. I think its working now. My bad. One more question more conceptual.

Now i see what you meant in the formulation of the presentation.

But to be honest im not sure how I feel that i cannot ocndition on the wells themselves. in practice this is what i would want to do. I give the modelwell information + seismic and it generates realizations of the velocity. Is this challenging from implementing? Why is this choice made here?

[rafaelorozco]

Yeah that is a good point. We can definitely explore some scenarios where we condition on the well. What i would emphasize is that the line between training and testing datasets is completely blurred in this case. I argue that the training dataset IS the test dataset. So you would input the wells into the algorithm by using those during training. This is a bit of a weird point to explain though

[rafaelorozco]

In general that choice is made such that the network doesnt simply learn the identity to lower the mismatch but that is not a given since you can play games here to prevent that it learns the identity