Sharp: Virtual
Reality Medical Simulators
(1995-2003)
Scientific leader: Christian Laugier
Contact: César Mendoza
INTRODUCTION
SHARP (now e-Motion) has been involved in the development and conception of new methodologies and ideas for the next generation medical simulators since the middle of the last decade. Nowadays, extensive research is being done on the application of computers and robots for surgery, planning and execution of surgical operations and in training of surgeons.
Our research has mainly been based on the dynamic simulation of the different agents that take part in a complete virtual reality medical simulator system. Virtual reality provides an environment where there is no risk to patients, they are less stressful and they are less expensive compared to traditional training. Furthermore, they give 3D information and let the surgeon trainee to practice with arbitrary anatomies and pathologies several times.
A wide variety of approaches have been investigated and we can group then in the following axis of research:
|
|
|
To get a deep insight of our research in this field (and other robotics related fields) get our year activity reports.
A key factor in our research has been the development of models to simulate human tissue. Generally speaking, human tissue can be considered as a deformable body of viscoelastic material. To enable dynamic simulation of these bodies, we have modified and patched three well known physical models onto the geometry of human tissues.
Mass-spring models. The general idea of these models consists of a mesh of point masses connected by elastic links and mapped onto the geometric representation of the virtual object, i.e. the masses are the vertices and the springs are the edges of the triangulation or tetrahedralization. Newton based dynamic is used to modify the point-masses positions and create deformations. These models are dependent on the geometrical connectivity between the point-masses and dampers and in the volume conservation. To deal with this our research team has proposed the use of angular springs. See for example:
Joukhadar, A. and Laugier, C. (1996). Dynamic simulation: Model, basic algorithms, and optimization. In Proc. of the Workshop on the Algorithmic Foundations of Robotics, Toulouse (FR).
Other main drawback of mass-springs is the setting of the parameters values of the model. Our investigations and our application requirements (echographical simulator) leaded us to handle parameter identification for these models. Based on the experimental data and the computational requirements, a two layer lumped element model is chosen. The two layer model is composed of a surface mesh of masses, linear springs and dampers, and a set of nonlinear springs orthogonal to the surface to model volumetric effects by giving normal support to the surface mesh.
d'Aulignac, D., Cavusoglu, M. C., and Laugier, C. (1999a).Modelling the
dynamics of a human thigh for a realistic echographic simulator with
force feedback. In Proc. of the Int. Conf. on Medical Image Computer-Assisted
Intervention, Cambridge
Finite element methods. This method is by far the most accurate models to represent human tissue. It transforms the continuum mechanics of deformations (physical equations, boundary conditions, …) into discrete problem so it can be solved using numerical analysis. The idea is to subdivide the object into a finite set of sub-elements or sub-volumes, making a geometric discretization of the object. Next, the physical properties of the object are interpolated to each sub-element using shape functions, so the continuum mechanics of the object is expressed in terms of a set of sub-element, which are associated according to their neighbourhoods to obtain a set of equations representing the physics of the object.
Our research in this type of methods has been focused not in the model itself but in the way to solve its governing equations (in a quasi-static taste and using implicit integration with some type of relaxations). See the section of numerical integration methods.
Long Element Methods and Volume Distribution Methods. Due to the complexity and the computational time taken to solve the governing equations of the finite element methods we have proposed two new models. Long Element Method and the Volume Distribution Model.
Long Element Method (LEM) views the objects as two-dimensional distributed elements filled with an uncompressible fluid. The advantage of this method is that the number of the elements is one order of magnitude less than in a discretization based on tetrahedral or cubic elements. In the static long element method, each element is assumed to be filled with fluid. But at the same time, each element is also assumed to obey Hooke's Law in the axial direction. Pascal's principle and the law of conservation of volume are used as boundary conditions to establish the state of equilibrium. For a deeper view on the model, see:
Costa, I. and Balaniuk, R. LEM - An approach for real time physically based soft tissue simulation. In Proceedings of IEEE International Conference on Robotics and Automation, 2001
K. Sundaraj, C. Laugier, and I. Costa Ferreira, An approach to LEM modelling: Construction, collisiondetection and dynamic simulation, Proc. of the IEEE-RSJ Int. Conf. on Intelligent Robots and Systems, Hawaii, HI (US), Oct. 2001.
Many aspects of this approach still remain to be explored.
The Volumetric Distribution Model (VDM) is a model suitable for soft tissue simulation. This model is surface based, hence its complexity is in general one order of magnitude lower than FEM. VDM is inspired of FEM, it uses bulk variables like pressure, volume and bulk modulus as model parameters. Pascal's principle and volume conservation are used as boundary conditions. A key feature of VDM is the absence of discretization of the interior. Any surface based polygonal mesh can be used as input data. An advantage in the medical field is that these types of triangular meshes are common. The required parameters for the VDM model like volume and area of the facets are easily extracted from the polygonal mesh. The intrinsic parameters in VDM for soft tissue depend on the organ being simulated. If for example, a liver is being modelled, it being 95% irrigated by blood requires only the density and bulk modulus B of blood. Such parameters can be obtained from the literature. To get a deep insight on this method, check
Sundaraj,
K., Phd Thesis,
Real-time dynamic simulation
and 3D intercation of biological tissue : application to medical simulators,
janvier 2004.
There exists several ways to solve the governing equations of the dynamics of a physical model. Some of our main ideas in this field are.
A. Joukhadar, C. Laugier, Adaptive time step for fast converging dynamic simulation system , in : Proc. of the IEEE-RSJ Int. Conf. on Intelligent Robots and Systems, 2, p. 418_424, Osaka (JP), Nov. 1996.
Surgery simulations try to make interactions between virtual organs and human operators as natural as in the real world. In a surgical procedure virtual simulation, we distinguish three main tasks:
Collision Handling. The first step to carry out realistic interactions is to detect where the objects collide (this axis of research is known as collision detection). However, locating the contact primitive between two objects may be computationally expensive, especially if the objects are composed of thousands of polygons. These interactions change the velocities of the objects involved, their kinematics and their potential energies. Our research in this area has been focused in obtaining fast computations for collision detection, correct object deformations due to collisions and the development of techniques to avoid objects to interpenetrate. See these articles as a small example of our works in this area:
A. Joukhadar, A. Scheuer, C. Laugier, Fast Contact Detection between Moving Deformable Polyhedra, Proc. of the IEEE-RSJ Int. Conf. on Intelligent Robots and Systems, 3, p. 1810_1815, Kyongju (KR), Oct. 1999.
A. Joukhadar, A. Deguet, C. Laugier, A Collision Model for Rigid and Deformable Bodies, Proc. of the IEEE Int. Conf. on Robotics and Automation, 2, p. 982_988, Leuven (BE), May 1998.
A. Deguet, A. Joukhadar, C. Laugier, A Collision Model for Deformable Bodies, Proc. of the IEEE-RSJ Int. Conf. on Intelligent Robots and Systems, Victoria, BC (CA), Oct. 1998.
A. Joukhadar, A. Deguet, C. Laugier, Towards Realistic Dynamic Simulation : Deformations and Collisions Models, Proc. of the Workshop on Dynamic Simulation : Methods and Applications, p. 21_31, Grenoble (FR), Sept. 1997. Workshop held in association with the IEEE-RSJ Int. Conf. on Intelligent Robots and Systems.
A. Joukhadar, A. Wabbi, C. Laugier, Fast Contact Localisation Between Deformable Polyhedra in Motion, Proc. of the IEEE Computer Animation Conf., p. 126_135, Geneva (CH), June 1996.
A. Joukhadar, C. Laugier, Dynamic Simulation: Model, Basic Algorithms, and optimization, Proc. of the Workshop on the Algorithmic Foundations of Robotics, Toulouse (FR), July 1996.
A. Deguet, A. Joukhadar, C. Laugier, Models and Algorithms for the Collision of Rigid and Deformable Bodies, Robotics. The Algorithmic Perspective, P. K. Agarwal, L. E. Kavraki, et M. T. Mason (Eds.), AKPeters, 1998, p. 327-338.
K. Sundaraj, C. Laugier, Fast contact localisation of moving deformable polyhedra, Proc. of the Int. Conf. on Control, Automation, Robotics and Vision, Singapore (SG), Dec. 2000.
Topology modifications: Surgery simulations include the execution of complexes tasks on the virtual organs such as tearing, cutting, and suturing. Considering the human organ as a virtual object made of thousands of polygons and volumetric elements, these manipulations cause a change in the way the polygons are joined. In other words, the original topology of the object is modified. See the following articles as a brief view of our works in this area:
F. Boux de Casson, C. Laugier, Simulating 2D Tearing Phenomena for Interactive Medical Surgery Simulators, Proc. of Computer Animation, Philadelphia, PA (US), May 2000.
Haptic feedback: An important requirement in medical simulators is to give to the operator the sensation of touching the human organs being simulated. Haptic systems give people the sensation of touching objects (in our cases, organs) in a virtual reality environment. These systems use a force reflecting mechanical device to apply a force to the user (surgeon trainees) to create the illusion of physical contact of the virtual object. Many research issues related to the stability and realism have been investigated within our research group. For example, instabilities may arise from the difference rate between the physical simulation (including the visual rendering) and the haptic simulation. To get a deep insight on this, have a look on the following articles:
R. Balaniuk, C. Laugier, Haptic interfaces in generic virtual reality systems, Proc. of the IEEE-RSJ Int. Workshop on Intelligent Robots and Systems, Takamatsu (JP), Nov. 2000.
Most our research has leaded to two complete systems:
We have developed a mass-spring model of a human thigh based on real data acquired. We have addressed the difficulties of determining the parameters of this model to fit the measurements and the computational demands. Implicit integration is used to update the model through time. We have also included accurate force-feedback using a Phantom haptic device. Our simulator can be used to train practitioners to detect thrombosis.
d'Aulignac, D., Cavusoglu, M. C., and Laugier, C. (1999a).Modelling the
dynamics of a human thigh for a realistic echographic simulator with
force feedback. In Proc. of the Int. Conf. on Medical Image Computer-Assisted
Intervention, Cambridge
  |
We have taken measurements from a real thigh with respect to an external forces produce by a PUMA articulated arm. From the reaction forces, we have created a non-linear mass-spring model.
|
|
|
 |
For every position, orientation of the probe (represented by the haptic device) an echographic image appears in the screen of the simulator. The images are taken from a real set of echographic images and they are interpolated to obtain the desired one. |
|
In the final system, the trainee looks at the artificially generated
echographic images (by interpolations) and interacts with a computer
simulated dynamical thigh model through a haptic interface. Our
investigations have also focused on solving the difference rate problem
between the haptic device and the physical simulation. See these
papers for further information about this:
R. Balaniuk, C. Laugier, Haptic interfaces in generic virtual reality systems, Proc. of the IEEE-RSJ Int. Workshop on Intelligent Robots and Systems, Takamatsu (JP), Nov. 2000.
|
We have also been involved in the development of a virtual reality laparoscopy simulator. Our works in the field were carried out in the framework of a national french project (CAESARE). We mainly work in modeling the dynamics of a human liver using mass-springs and explicit finite elements. Further developments included the execution of different tasks such as cutting and tearing. To enhance the realism of the simulation we have added force feedback using a Phantom haptic device.
 |
 |
 |
 |
F. Boux de Casson, C. Laugier, Modelling the dynamics of a human liver for a minimally invasive simulator, Proc. of the Int. Conf. on Medical Image Computer-Assisted Intervention, Cambridge (GB), Sept. 1999.
