Image segmentation is one of the most important steps leading to the analysis of image data. The goal is dividing the image into parts that have homogeneous attributes, and have a strong correlation with objects or areas of the real world contained in the image. Region-based segmentation methods, e.g., Markov Random Fields (MRFs), are usually robust to noise and easy to capture contextual dependencies, but often generate rough boundaries and hard to incorporate shape and topology constraints. On the other hand, edge-based segmentation methods, e.g., deformable models, are usually easy to incorporate shape prior and object topology, but sensitive to noise (false edges and weak edges) and, sometimes, initializations.
 
In this project, we are combining deformable models and Markov random fields using a graphical model framework for better image segmentation. The integrated framework takes advantages of both models and generate better segmentation results in many cases.
 
The tightly coupled model:
droppedImage_2
The exact (yet intractable) inference
droppedImage_3
The variational inference (to decouple the original intractable model inference)
droppedImage_4
The extended MRF model (solved by the belief propagation algorithm)
droppedImage_5
The probabilistic deformable model
droppedImage_6
The optimal variational parameters
droppedImage_7
The whole segmentation algorithm is an EM algorithm solving the above equations iteratively:
droppedImage_8
 
 
Results
 
five
 
fiven
 
heart
 
More results in our paper:
 
Rui Huang, Vladimir Pavlovic, and Dimitris N. Metaxas, "A Graphical Model Framework for Coupling MRFs and Deformable Models" in Proceedings of CVPR, Vol. 2, 739-746, 2004.
 
A "more" tightly-coupled model (belief propagation inference can be performed in the whole model instead of using the variational inference):
 
Rui Huang, Vladimir Pavlovic, and Dimitris N. Metaxas, "Hybrid Framework for Image Segmentation Using Probabilistic Integration of Heterogeneous Constraints," in Proceedings of CVBIA (LNCS 3765), 82-92, 2005.
 
An extension to 3D segmentation:
 
Rui Huang, Vladimir Pavlovic, and Dimitris N. Metaxas, "A Tightly Coupled Region-Shape Framework for 3D Medical Image Segmentation" in Proceedings of ISBI, 2006