How to train CRF on multiple images

[Luka-M]

Hello,

I'm testing this CRF implementation and was wondering is there an easy way to train CRF on multiple images. Do you just fill the crf object with new unary and pairwise potentials and repeat training with new objective function?

Luka

[lucasb-eyer]

Hi Luka,

I'm sorry, it looks like I completely missed your issue! I believe that I currently didn't wrap the learning part of the API at all, only the inference, so is it possible that you mean "run inference" as opposed to "train"?

[lucasb-eyer]

Closing due to inactivity and unclarity about actual issue.

[lishi0927]

Hello, I also have the same question. I have some training images and test images, So how to train the CRF model and get the parameters, do you have any piece of advice about learning?

[lucasb-eyer]

Funny you ask, I added a few more pointers about learning to the README just yesterday!

I myself don't have the time (and need) to implement learning, but I think it should be straightforward to add for someone motivated enough. Hope these pointers can give you a start, and should you manage to add learning, I'd be happy about a pull-request!

[lishi0927]

Okay, thank you for your reply.

[lishi0927]

Ah, and i have another question about learning. I have read the original learning cpp code and find that they use only one image optimize the CRF function and get all learning parameters. So if I want to train N images, I have to get N couples of learning parameters and choose the best learning parameters that get the minimize error to the other images, or do you have any piece of advice to get the learning parameters of the multiple images? Thank you.

[lucasb-eyer]

Disclaimer: I've not used densecrf with learning.

Since it's learning the parameters by gradient-based optimization (l-bfgs here), my guess is that you could do it similarly to neural networs: compute the gradients on say B random pictures, so you get B gradients. Average them, then do an optimizer step. Then pick B next pictures, compute average gradient, do an optimizer step. This way you should in theory optimize the parameters over all pictures.

I'm not sure how well this will work in practice with dense crfs, but if you try it, I'm curious (and others probably too) so please report back whether it worked or not. And of open a PR with any changes you needed :)

[lishi0927]

Okay, thanks for your advice and i will have a try.

[reynoldscem]

I think part of the issue with averaging gradients won't come from the fact we're trying to optimise a CRF, but rather that we're using L-BFGS - as it is using line searching & an approximate Hessian to calculate updates. Still, worth a try at least.

[lucasb-eyer]

I've seen people successfully use L-BFGS with neural nets, given they use large batches. So I personally would try to average the gradient over all my images before I run an L-BFGS step.

Averaging gradients over all images is the same as computing the gradient of the total loss if the total loss is the sum over each image's loss. So I feel like it should work this way.

But yeah, the caveat is that I haven't tried this, and as usual with nonlinear optimization, you never know until you try :wink: Hence I'm very curious to hear back from anyone trying this.

Also, re-opening since there's more information now, albeit not from OP.

[dhawan98]

hey! if any of you have coded or found the above code. can you point me towards it, or send me the code. I need to run the training process on a set of images and predict on some other image.

[manishh]

Useful discussion, anything added to the repo on learning?

[Mikeflames]

@dhawan98, @lucasb-eyer Did you find the code for the Set of images? If found, please direct me . Thanks