Robots have been used for years by commercial
industry to accomplish repetitive manufacturing tasks. With
the use of advanced effectors (the equivalents of human limbs
and hands), the complete telepresence-capable robotic system
truly will become an enabler of logistics operations. Telepresence
will allow certain logistics functions formerly performed by
a human to be completed in a much more adaptive manner.
Some near-term telepresence efforts include allowing medical
providers to see, hear, and touch patients in real time with
the necessary visual, auditory, and tactile perception in order
to conduct or assist in remote surgery (telemedicine). In many
respects, telemedicine leads the way in the advancement of
telepresence capabilities. While the technology still falls
short of supporting telesurgery for Soldiers fighting in Iraq
and Afghanistan, the Army Medical Command's Telemedicine and
Advanced Technology Research Center successfully conducted
a robotic telesurgery over the Internet at a conference in
April 2005.
Effectively using technologies and advances from the telemedicine
community could extend the use of telepresence to mechanics
so that they could use robotic platforms to conduct high-risk
battlefield recovery and maintenance. Telepresence also
could be used to operate materials-handling equipment and
conduct
port discharge and depot repair operations. In those cases,
operators would have greater control over processes without
incurring additional safety or manpower burdens within
the battlespace.
Telemaintenance, as envisioned for the deep future, may allow
the Army’s most experienced maintenance technician
to participate from a CONUS-based depot in the repair of
equipment on the battlefield. Conceivably, he would work
as if truly present with a team of Soldiers on the ground
in the theater of operations. Likewise, those who designed
the system—the scientist or engineer team from an Army
research, development, and engineering center or contractor
support team—could participate in the maintenance activity.
Achieving Full Telepresence Immersion
A telepresence system is required to accomplish telepresence.
This system is composed of three essential subsystems (and
their related technologies): the home site, the communications
link, and the remote site. Telepresence technologies associated
with these three subsystems are similar to technologies found
in virtual reality except that, in telepresence, a user must
have feedback stimuli from, and the ability to exercise control
over, the remote site. The ultimate goal of a telepresence
system is to produce a transparent link from human to machine
that passes information so naturally between user and environment
that the user achieves a complete sense of immersion in the
remote-site environment. To accomplish this, sensory impressions
obtained from the remote site and delivered to the home site
must engage the
human senses fully with sufficient breadth, volume, and quality.
As an example, consider that you are at a home site viewing
a video of a remote-site environment on a theater-sized screen.
The video provides only a minor feature of the much larger
real-world environment present at the remote site. If a majority
of your total vision is subjected to the video image on a
large, curved screen and the video image depicts natural
human motions, then the visual element becomes perceptually
real world. Engaging the other senses similarly and in a
synchronous manner will enable you to progress toward the
feeling of being fully immersed in the remote, real-world
environment. The ideal situation occurs when high-quality,
high-resolution, and consistent information is presented
to all of your senses.
Telepresence requires a complete human-computer-machine interface
that incorporates audio, visual, haptic (touch), olfactic
(smell), and gustatic (taste) technologies
with home site elements perceptually identical to remote,
real-world elements. To date, the senses of sight and hearing
have been the focus of intense research and have formed the
core of virtual reality systems because these senses are
the most important senses by which we receive information
related to our surroundings. The contributions of smell,
touch, and taste are not as great, and current technologies
for reproducing smell and taste are difficult to implement.
One might argue that, compared with the other senses, taste
(and perhaps smell) play marginal roles in creating a full
immersion experience. Nevertheless, technology progression
eventually will enable all human senses to be engaged in
the telepresence experience.
Aside from the human-computer-machine sensory interface,
communication between the home and remote sites must be “real
time” so the user feels that he is indeed in the remote,
real-world environment. Home or remote site latency detracts
from the realism that telepresence systems seek to achieve.
Any communications link may be used by a telepresence system.
The specific type of link depends on distance, bandwidth
requirements, latency tolerance, availability of services
between sites, and so on. To achieve high fidelity immersion
in military applications, direct, dedicated umbilical links
between home and remote sites are desirable. Latency in communication
between the various sensory elements also will erode the
feeling of being fully immersed in the remote real-world
environment. Current and future advances in processing power
will reduce latency, which will provide users with more accurate
and readily available telepresence systems for myriad applications.
|
|
In the
future, telepresence may be used in materials-handling,
port-discharge, and depot-repair operations. Here,
a home-site user remotely operates a robot to unload
cargo at a remote site.
(Art by Eric Proctor of the Sensors
and Electron Devices Directorate
of the Army Research Laboratory.) |
|
Enabling Technologies for Telepresence Immersion
A number of enabling home-site interface technologies are
required to realize telepresence fully. In some cases, these
technologies are under study; in other cases, commercial
off-the-shelf technologies are already available.
Visual technologies. Humans mainly
interact with other humans through vision. Therefore, visual
sensory elements
must be
mature. The typical binocular field of view (FOV) of a
human is approximately 180 degrees horizontal (with approximately
120 degrees of binocular overlap) and 150 degrees vertical
(limited by facial features such as cheeks, nose, and forehead).
Current prototype telepresence systems use head-mounted
displays (HMDs) to provide visual information to the user.
HMDs are stereoscopic devices that can convert two-dimensional
video images of remote-site environments into three-dimensional
(3–D) visual images. Typical FOVs for HMDs are quite
narrow—approximately 100 degrees horizontal and 60
degrees vertical—although full-immersion HMDs with
FOVs of more than 180 degrees horizontal and 80 degrees vertical
do exist. Full-immersion HMDs have been demonstrated in military
environments, but they exhibited poor display resolution,
limited FOV, and visual, position, and tracking latencies.
Latency greatly affects the illusion of full immersion, and
visual, position, and tracking latencies create visually
induced motion sickness because the motions perceived in
the real, remote-world environment are not reflected in the
user’s body. Achieving near-zero visual latency is
quite challenging because of the complexity of the real-world
environment but may become possible with advances in quantum
computation and quantum communication. (See “Quantum
Computation and Communication” in the September–October
issue of Army Logistician.) An alternative 3–D vision
technology (holography) provides a way to create images without
using lenses. Although this technology is very promising,
moving holographic images are currently difficult to provide
and therefore probably will not be developed until the 2010–2020
timeframe. Additional vision technologies, such as 3–D
computer displays and virtual retinal displays, are under
study and have great potential.
Auditory technologies. In
addition to the visual sensory element, full telepresence
immersion requires authentic
auditory reproduction of sound. The original sound field
recorded at the remote site must be identical to the sound
field reproduced at the home site. Today, audio reproduction
is engineered with fidelity that exceeds the limits of human
perception. Hearing, however, is inherently a spatial perception.
The human auditory system detects sound waves with two ears
(binaural hearing) to determine information about the 3–D
location, distance, and size of sound sources. Therefore,
the ultimate goal is to reproduce the spatial properties
of sound as accurately as possible. Current prototype telepresence
systems achieve accurate spatial reproduction through the
use of high-quality stereo headphones. Multichannel audio
systems such as 5.1 (6-channel) and 10.2 (14-channel) surround-sound
systems improve spatial reproduction
by increasing the number of channels around a user. Future
telepresence systems will include headphones that incorporate
virtual sound or multichannel surround-sound headphones,
the capabilities of which exceed current 5.1 surround-sound
headphones. Additional auditory technologies under study
include sound transmission through the skull and HyperSonic
Sound (HSS) technology. (HSS is intense focusing and channeling
of sound over great distances without dispersing its quality.)
These are not yet mainstream technologies, but they are on
the horizon and are very promising.
Tactile technologies. The human sense of touch
is conveyed to the human brain through the haptic sensory
system. Haptic technologies seek to apply tactile sensations
to a human’s interaction with a computer using a haptic
device such as a data glove equipped with sensors to sense
the bending of the fingers and movements of the hand. The
goal is not only to allow the user to feed information into
a computer but to permit the user to receive information
through a haptic interface. Using a data glove in virtual
reality, for example, a user can pick up a virtual object
such as a cup. A computer then senses the movement of the
user’s hand and moves the virtual cup on a display.
This provides the user with the feel of the cup in his hand
through tactile sensations sent by
the computer.
Olfactory and gustatory technologies.
Technologies to reproduce the human senses of smell (olfactic)
and taste
(gustatic) have, for the most part, been ignored compared
with visual, audio, and haptic technologies. To appreciate
the complexity of reproducing these sensory elements, keep
in mind that humans have approximately 10 million sensory
neurons for smell and approximately 10,000 taste buds that
contain between 50 and 100 taste cells representing sweet,
sour, bitter, salty, and umami (the flavor that is characteristic
of glutamates such as monosodium glutamate). Current electronic
noses can recognize certain odors. These electronic noses,
which are composed of arrays of electronic chemical sensors
and pattern-recognition capabilities, are much simpler than
their biological counterparts. Therefore, users are limited
to a predetermined set of odors. Unlike electronic noses,
gustatic technologies are quite complex. Currently, users
experience taste through biological lipid and polymer membrane
sensors. Full telepresence immersion will require maturation
of olfactic and gustatic technologies.
Achieving Telepresence
In addition to advances in enabling technologies, telepresence
requires advances in remote-site technology (robots) and
communications link technology (to achieve near-zero communication
latency).
Robots. Sensory elements received from the remote site will
be obtained by effectors. An example of an effector is a
human-like robot. Robots controlled by users from remote
locations will carry out operations required in the field.
Robots must be able to perform myriad tasks with the versatility
of humans and, in some cases, with strength exceeding that
of humans.
Advances in robot technologies have increased over the years,
but robots remain very primitive. Research, for the most
part, is confined to universities. Robots have the ability
to see and hear but lack extensive haptic, olfactic, and
gustatic sensory elements and are primarily purpose-built
for specific tasks. Advances in telepresence will require
advanced robots that can perform multiple tasks and have
the ability to adapt.
Near-zero communication latency.Communication latency is
perhaps one of the biggest detractors from the feeling of “being
there here,” or fully
immersed in a remote environment. To demonstrate the effects
of
communication latency, consider the following example.
Assume that you
want to send a simple communications signal, such as a
pulse of light, around the globe. Without the use of any
communications
networks, it would take 0.13 milliseconds for the pulse
to make the trip. This time delay is referred to as a distance-induced
latency. Humans can detect time delays of approximately
16
milliseconds and greater.
Therefore, in this simple example, the communication latency
would be negligible. Typically, however, communication
between a source and a receiver involves
the transfer of a large amount of data. As a result, distance-induced
latencies must be coupled with latencies in processing
speed (transmission-induced latencies, or throughput).
Today, packet
switching is the dominant means of transmitting communications.
Packets of information are sent individually between nodes
of a network in a way that optimizes bandwidth and minimizes
latency.
Near-zero communication latency requires that, for the
user to achieve full telepresence immersion, communication
between a robot and the user occur in less than 16 milliseconds.
Unfortunately, current high-bandwidth
data transmissions (full motion video) have significant
latency. This latency is both distance-induced and
transmission-induced. This is sometimes evident, for example,
when a news reporter presents a video report from a remote
location halfway around the world. Latencies of a second
are quite common and very noticeable. Although the distance-induced
latency is very small, transmission-induced latencies built
into the equipment are significant.
Future processor technologies undoubtedly will be faster
than those today, and new communications paradigms will
be developed. Advances in quantum computation could benefit
true telepresence. In the near term, however, steps can
be taken to decrease bandwidth requirements and reduce
communication latency so that certain elements of telepresence
can be realized. For instance, greater levels of interaction
between unmanned systems can be facilitated by creating
environment or terrain models in advance of operations.
Representations of the physical environment can be mapped
ahead of mission execution to help reduce bandwidth requirements.
The remaining bandwidth then can be dedicated to representing
dynamic features that will allow new levels of human-machine
interface that otherwise would be infeasible. Enabling
technologies for decreasing latency, such as data compression,
increased digital modulation, and subdivided optical nodes,
also can be implemented.
Certainly, near-zero communication latency affects global
military logistics beyond its impact on telepresence. As
Dr. Theodore Bially, the Director of the Defense Advanced
Research Projects Agency Information Exploitation Office,
said in 2004, “The fog of war will plague us as long
as the information provided to any level of command is
incomplete, inconsistent, delayed in time, difficult to
manipulate or hard to visualize. To lift that fog we must
provide each of our warfighters with total, accurate and
up-to-the-minute battlefield situational information .
. . .” It is easy to see how telepresence could facilitate
this task.
What can we expect in the deep future (2030)? In the deep
future, sensory stimulation will completely bypass the
human sensory organs, and the perceptual neurons in the
brain will be stimulated directly. Sensory threshold filters
will prevent overload of the human perceptual neurons.
Full-immersion telepresence will be realized with completely
noninvasive sensory stimulation directly to the brain.
Sensory element information sent from a user to a robot
will be accomplished using noninvasive brain-computer interfaces.
Recent advances in brain-computer interfaces have demonstrated
that noninvasive readings of brain activity can be harnessed
to perform primitive robotic motions. In the future, users
will have the ability to control robots just by thinking.
Finally, configurable, remotely assembling components (robot
swarms) will have the ability to adapt to remote, real-world
environments, thereby enhancing telepresence capabilities
and applications.
Telepresence will become possible following the development
and improvement of robots, near-zero latency communication
capabilities, multisensor integration and fusion, and multirobot
systems. Telepresence may be feasible in limited applications
by 2015.
Research in various fields, such as computer graphics,
computer vision, human-computer interaction,
brain-computer-machine interaction, acoustics, networking,
and databases, all support the telepresence theme. Work
is underway in the areas of telepresence systems design
and the architecture of physical spaces, multimodal sensing
(including camera and computer vision, microphone arrays
and acoustics, haptic sensors, and active badges), multimodal
presentation and display systems, virtual and augmented
reality, ambient intelligence, network infrastructure,
and spatio-temporal databases. Current telepresence design
and studies in the United States, the Netherlands, India,
Japan, and the United Kingdom should mature significantly
in the next 10 to 15 years.
Some network and communications hurdles exist in the use
of telepresence for military applications. Telepresence
devices use high-bandwidth fiber optics, which currently
are not available to forward-deployed units. Until they
are, short-range wireless and existing fiber optic networks
could be used for communications in logistics applications
such as materials-handling and depot operations. Full battlefield
utility of telepresence may require quantum computation
and communication advances or other paradigm-shifts to
support and overcome current data ransmission and bandwidth
limitations. However, it is plausible to expect that prototype
telepresence systems for logistics applications could be
fielded in the near term.
ALOG
Dr. Keith Aliberti is a research physicist
in the Sensors and Electron Devices Directorate at the Army
Research Laboratory at Adelphi, Maryland. He serves as the
laboratory’s liaison officer to the Army Logistics
Innovation Agency at Fort Belvoir, Virginia. He has a B.S.
degree in physics from Rensselaer Polytechnic Institute and
M.S. and Ph.D. degrees from the State University of New York
at Albany.
Thomas L. Bruen is a logistics management specialist at the
Army Logistics Innovation Agency at Fort Belvoir, Virginia.
He has a bachelor’s degree in engineering from the
U.S. Military Academy and is a graduate of the Army Management
Staff College’s Sustaining Base Leadership Management
Program.
