Real-Time Visually and Haptically
Accurate Surgical Simulation
Karl D. Reinig, Charles G. Rush,
Helen L. Pelster,
Victor M. Spitzer, James A. Heath
The Center for Human Simulation,
University of Colorado Denver, Denver, Colorado 80262
Abstract
The Visible Human Database provides
a complete submillimeter photographic and radiological description of both a male
and a female cadaver. The Center for Human Simulation has developed real-time
algorithms that allow a user to wield a virtual scalpel and produce arbitrary
cuts into the reconstructed body. Real-time interaction is achieved by producing
the appropriate texture mapped surfaces directly from the Visible Human Database.
We are currently combining a 3 degree of freedom PHANToM haptic device (SensAble
Devices) with our surgical cutting algorithms to produce a system with the look
and feel of surgical cutting.
1. Introduction
The universal use of flight simulators
in the aircraft industry encourages the investigation of simulation techniques
in the surgical arena. Scientists and engineers have long postulated the requirements
for such a system while waiting for hardware and databases to catch up. Realistic
real-time visual and haptic interaction with anatomical structures is a major
component of any surgical simulator. This paper briefly presents application of
the Visible Human Database to simulate real-time visually and haptically realistic
surgical cutting. The method takes advantage of the existence of graphics hardware
capable of rapidly rendering texture mapped polygons. A texture mapped polygonal
representation of the surgical area is created apriori. As cutting proceeds, the
polygonal surface is altered in real time to append the new surfaces produced
by the cut. Texture maps for the initial and newly created surfaces are produced
directly from the Visible Human Database.
2. Visible Human Database
The Visible Human Database is a volumetric
color description of the anatomy of an entire male and an entire female cadaver.
The data was created at the University of Colorado Medical School at the Center
for Human Simulation (CHS) using a custom cryomacrotome. Data for the male was
gathered in one millimeter increments from head to toe (z axis). Each slice was
digitized at 1/3 millimeter resolution in x and y. The female was done the same
way with three times the resolution in the z direction giving true 1/3 millimeter
voxels. The National Library of Medicine funded the collection of the images as
part of the Visible Human Project. The data is in the public domain and is available
on the internet (/ftp://nlm.nih.gov/visible/).
After taking the time to segment
and classify each voxel in a subvolume of the male or female it is possible
to interact with the data in many ways. We use computer generated edges to assist
in the extraction of consistent and accurate structural boundaries. The uniformity
of the resulting structures allows us to render anatomical structures directly
from the data. Figure 1 shows an
example rendering of the muscles, bones, and peripheral veins of the right knee
of the male.
In addition, marching cubes [1]
techniques can be used to create polygonal surface models directly from the
segmented data. Because of the accuracy and consistency of our boundaries, we
can create the texture maps for the polygonal models directly from the data.
Using existing high-end graphics workstations, the texture mapped polygons may
be rendered in real-time with a very high degree of visual realism.
3. Method
The basic premise is that at any stage
of surgery the entire field of interest can be represented by a set of surfaces.
In terms of the nuts and bolts of computer graphics, updating this set of surfaces
is a matter of updating the associated set of vertices, connectivity arrays, and
texture maps. Since all changes in the field are incremental, it is possible (with
reasonably fast hardware) to make these changes in real time.
Our current implementation is completely
Open Inventor based and performs the following tasks:
- Looks for intersections of the
virtual scalpel blade with the current possible surfaces of contact
- Severs the polygons as the virtual
blade passes through them
- Creates new polygons representing
the depth of cut
- Creates texture maps for the new
polygons directly from the Visible Human Database
- Spreads the new surfaces apart
revealing the cut
We have recently added haptic feedback
through the use of a PHANToM (SensAble Devices) force reflective device. A simple
model allowing springiness and shear converts the position and recent history
of the virtual scalpel into the proper force feedback. Variation in force due
to interactions with different anatomical materials is currently being added to
the model. This will make bones impenetrable, tendons harder than muscle, etc.
4. Results
Creating texture mapped polygonal surfaces
"on the fly" gives visually realistic simulations of surgical cuts. Figure
2 shows a single frame from a real time cut made on an SGI Onyx workstation.
The surgeons who have wielded our virtual scalpel have been unanimously impressed
by the visual detail.
The addition of tactile feedback
greatly enhances the "believability" of an already visually convincing method.
It is immediately obvious to the user that he or she can place the scalpel and
create the desired incision with significantly greater accuracy and ease.
5. Discussion
To be truly useful a surgical simulator
needs many other attributes, even when limited to the simpler minimally invasive
techniques. We consider this work to be laying the foundation for future surgical
simulations. Seeing and feeling the results of incisions are the fundamental building
blocks of any useful surgical simulator. Near-term enhancements to the surgical
simulator should include:
1) Realistic models for blood and
other fluids.
2) Separate polygonal surfaces for different tissue types (muscle, tendon, fat,
etc.).
3) The ability to cut completely through a material allowing pieces of anatomy
to be removed.
4) Better deformation algorithms for the displacement of tissues when stretched
and cut.
In addition to the visual and haptic
features above, a dynamic physiological model should be included in the surgical
simulator. For example, the effects of severing blood vessels, nerves, etc.,
would be used to estimate the state (blood pressure, heart rate, etc.) of the
simulated patient.
There is nothing in our basic method
that precludes the addition of any of the desired capabilities. We have postulated
reasonable solutions to each. These solutions will not be realized in the near-term
unless this effort is funded. Even with funding, a comprehensive surgical simulator
is still years away.
In the meantime, the Center for
Human Simulation is putting this technology to immediate use in the development
of a simpler (and immediately useful) simulator for needle insertions. A working
prototype of our needle insertion simulator (NIS) was recently demonstrated
at the annual meeting o f the American Society of Anesthesiologists. The NIS
was used to teach celiac plexus blocks by giving the student both the feel and
visual feedback of a patient in a clinical setting.
6. Conclusion
We have developed the basis for a practical
surgical simulator. While there is still a great deal to be done before the educational
benefits of such a system are realized, we feel that the required technologies
(fast graphics processors, reliable haptic devices, robust software methods) already
exist. The production of a practical surgical simulator will require the melding
of many disciplines. The Visible Human Database can be the vehicle for linking
these disciplines.
References
[1] Lorensen, W. E. and Cline, H.
E., "Marching Cubes: A High Resolution 3D Surface Construction Algorithm," Computer
Graphics, vol. 21, no. 3, pp. 163-169, July 1987.
