In Search of a Good Title » Projects

Ph.D. Thesis: Information Abstraction Visualization for Human-Robot Interaction

Curtis M. Humphrey — Fri, 05 Jun 2009 22:08:52 +0000

Vanderbilt School of Engineering

Future emergency incident responses, including Chemical, Biological, Radiological, Nuclear, and Explosive (CBRNE), will incorporate robots. The ability to interact with robots and understand the resulting volumes of information requires a system of human-robot interfaces employing directable visualizations that provide information immediacy, relevancy, and sharing appropriate for each human’s responsibilities.
This dissertation conducted two modified Cognitive Tasks Analyses (CTA) on the CBRNE incident response. The Cognitive Information Flow Analysis (CIFA) was developed to combine CTA results and to analyze the path of information as it passes through and is transformed by the system at different human-robot interaction (HRI) user levels. These analyses (i.e., modified CTAs and CIFA) collectively informed the HRI design and development.

The primary contributions of this dissertation are the development and evaluation of two novel visualization techniques that present immediate, relevant, and shared information provided by the robots to the human users in the system of human-robot interfaces. The General Visualization Abstraction (GVA) algorithm, the first technique, is designed to provide information immediacy and relevancy by displaying the most useful information at any given moment determined by rewarding information that is either historically and currently relevant or novel and emerging. The Decision Information Abstracted to a Relevant Encapsulation (DIARE) concept, the second technique, supports decision-making by representing prior event information as a defined volume in the visualization’s information space and encapsulates the volume into an explicit and visual object that can be shared across time and users.

User evaluations were conducted for both visualization techniques. The GVA algorithm’s evaluation results indicate that it can reduce cognitive workload, increase situational awareness, and improve performance for two different HRI user levels. The DIARE concept results indicate that participants were able to rapidly ascertain what had happened previously with great accuracy and good memory recall. Together, these two visualization techniques can assist decision-makers using directable visualizations, such as those used in HRI, by offering an effective method of sharing and providing real-time, relevant information.

Related Papers:

Curtis M. Humphrey. Information Abstraction Visualization for Human-Robot Interaction. PhD thesis,Vanderbilt University, August 2009

General Visualization Abstraction Algorithm for Geographic Map-Based Directable Interfaces

Curtis M. Humphrey — Fri, 05 Jun 2009 20:12:53 +0000

A screenshot of a directable interface visualization powered by the GVA algorithm in action.

Emergency incident response is evolving from a response involving humans with equipment to a response system combining humans and thinking machines (e.g. robots). The robots and other sensors will generate and capture volumes of information that will be presented in geographic map-based directable visualizations. This project involved presented a novel approach to performing information abstraction (i.e., selection and grouping) and determining how each information item should be presented (i.e., its shape) in direct-able visualizations for the emergency incident response. This new approach employs the General Visualization Abstraction (GVA) algorithm to make salient and direct attention to the most relevant information items by determining an importance value for each information item based on the item’s relationship with two classes of information: historically and currently relevant information, and novel and emerging information.

The evaluation results were statistically supportive, indicating that, in most cases, the GVA algorithm improved performance, lowered workload, and increased situational awareness. The implication to interface visualizations from these results is that the general approach behind the GVA algorithm (i.e., the grouping, filtering, and displaying of an information item based on an evaluation of its historical and currently relevant and novel and emerging properties) is not only possible but advantageous in directable visualizations.

Related Papers:

Humphrey, C. M., & Adams, J. A. (2009). General Visualization Abstraction Algorithm for Geographic Map-Based Human-Robot Interfaces. In Proceedings of the 4th ACM/IEEE international conference on Human robot interaction (pp. 289-290). La Jolla, California, USA: ACM. doi: 10.1145/1514095.1514180.

Humphrey, C. M., & Adams, J. A. (2009). General Visualization Abstraction Algorithm for Geographic Map-Based Directable Interfaces: Results. Manuscript in preparation.

Addressing Information Sharing in Directable Interfaces via DIARE Concept

Curtis M. Humphrey — Fri, 05 Jun 2009 20:19:52 +0000

DIARE Concept Snapshot

Decision Information Abstracted to a Relevant Encapsulation (DIARE) is based on the idea that evidence for a particular decision can be represented as a defined volume in the visualization’s information space spanning six components [x, y, e, t, s, m]. This defined volume becomes an object, or DIARE object, and contains information relating to that particular decision (i.e. range ofm) in terms of a spatial area (i.e. range of x, y, and e), time range (t), and detail range (v). A DIARE object acts as a super information object that can be shared between user levels and can itself become an element in the visualization. For example, several DIARE objects can be created by the person supervising the robots during an area survey, and later, someone else can search an overlapping area for any DIARE objects that deal with unusual items, which can cause the visualization to display one or two DIARE objects as information items on the map.

The evaluation participants generally stated that the DIARE concept’s strengths were its ability to provide: a quick visual event history overview (i.e., the timeline of DIARE objects), a quick mechanism to recall and replay important events (i.e., the ability to click on a DIARE object to recall that moment), and several useful elements when displaying a DIARE object (e.g., the DIARE object’s snapshot, information tags). These findings support the hypothesis that the DIARE concepts will facilitate information sharing across both time and users.

View of DIARE Concept in the Emergency Responses Interface as sceen in the user evaluations.

Related Papers:

Humphrey, C. M., & Adams, J. A. (2009). Addressing Information Sharing in Directable Interfaces via DIARE Concept. Manuscript in preparation.

A Human Eye like Perspective for Remote Vision

Curtis M. Humphrey — Fri, 05 Jun 2009 20:23:26 +0000

This figure depicts both the combined Human Eye Like Perspective image (left) and the two seed images (right) for the sinusoidal transpose function.

Remote imagery is usually affected by the keyhole effect, or viewing the world through a “soda straw.” This project focused on reducing the keyhole effect by improving the viewing angle of the imagery using a novel method that produces results more akin to that provided by the human vision system. The human eye like perspective method combines two images viewing the same point in space from the same, or approximately the same, point in space using two different focal lengths into one coherent image. The method employed in this project uses two cameras or images; however, the resulting image can also be achieved using a single, albeit complex, lens on one camera. The two views (wide angle and telephoto angle) were merged together using a transformation function that results in a final image with a relatively undistorted focus or detailed center and a surrounding context or peripheral area, thereby simulating the perspective provided by the human eye. This project explored linear, sinusoidal, and Gaussian transform functions, each with different scale values.

The combined Human Eye Like Perspective image for the Gaussian transpose function.

The early results of this human eye like perspective for remote vision indicate that the initial work was a success, as the resulting images provided a view of the situation that is different from other increase view angle solutions and in a manner that does not appear to be as cognitively confusing. Future work will focus on different focal lengths of the context and detail images as well as performance through a variety of tasks using video, rather than still images. Future work will also include human subject image quality evaluations of the resulting images.

Related Papers:

Humphrey, C. M., Stephen R. Motter, Julie A. Adams, and Mark Gonyea. A Human Eye Like Perspective for Remote Vision. Manuscript submitted for publication.

Robotic Tasks for CBRNE Incident Response

Curtis M. Humphrey — Fri, 05 Jun 2009 20:28:11 +0000

Robot Tasks - Inspecting a bag for hazards

Robots can significantly impact the CBRNE Response activities by providing directed, specific, first-hand information prior to and during hot zone entry. Hot zone entry is not permitted until there is a preliminary hazard identification, responders don PPEs, decontamination facilities are established, etc. Rapid robot deployments will provide additional information and assist in directing the response. This project identified requirements for eight robotic CBRNE incident response tasks that were developed from a CTA, an IFA, and direct responder feedback.

The CTA captured unobservable cognitive processes, decisions, and judgments representing expert performance. The CTA also identified a hierarchy of HRI user levels and the CBRNE response areas most appropriate for robots. The IFA focused on the path of information through the CBRNE response system, including how the information is used and transformed. The IFA also identified important information flow bottlenecks. The identified tasks, when implemented with robots, must provide the same or similar information to that provided by human responders to the decision-making command hierarchy. The identified robotic tasks can significantly impact the response capabilities when the robots are designed and developed properly. We believe that it is necessary to formally analyze and understand how humans conduct their response tasks and apply the analysis results to robot design and development. This approach is a departure from traditional robotic development; however, the application of CTA to other domains has demonstrated dramatic improvements in the resulting technology.

Related Papers:

Humphrey, C. M., & Adams, J. A. (2009) Robotic Tasks for CBRNE Incident Response. Advanced Robotics

Visualization for Effectively Co-located GIS Map Items

Curtis M. Humphrey — Sat, 20 Jun 2009 21:21:56 +0000

One of my summer project (2009) is to supervisor someone for awhile on a projected I designed. This year’s project is entited “Visualization for Effectively Co-located GIS Map Items” (title did take a while to invent). The official project paragraph is below along with a mockup of the idea we are implimenting.

“When information is sent from robots or remote sensors to a computer interface that representing the information as items on a geographic information system (GIS) map, the items can often be effectively co-located. Items effectively co-located present a visual issue because the items visually overlap to the point that only one item is truly visible. The long-term objective of the Effectively Co-located GIS Map Items project is to develop graceful and useful methods to visually display these items. This particular project will focus on identifying when items are effectively co-located and then developing and implementing one or more methods to visually present these items.”

Visualization mockup depicting an "stack" of items and the corrisponding "unstacking" visualization.

Designing a System of User Interfaces for Controlling Robots for Emergency Incident Response

Curtis M. Humphrey — Fri, 05 Jun 2009 21:52:23 +0000

System of User Interfaces

The emergency response incidents (e.g., Chemical, Biological, Radiological, Nuclear, and Explosive (CBRNE) incidents or weapons of mass destruction) are evolving from a response involving humans (e.g. first responders, government officials, civilians) with equipment (e.g. protective suits, vehicles, sensors) to a response system combining humans and thinking machines (e.g. robots, information technology). The difference between equipment (e.g. fire engines, radios, maps) and thinking machines (e.g. robots, computerized decision support systems) is that machines incorporate some cognitive abilities. If the emergency response system is to take effective advantage of emerging technology, the response activity needs to be understood in a way that facilitates the incorporation of these thinking machines and the development of effective human machine interactions. The incorporation of new thinking machines into the emergency response system is resulting in a shift, abet slowly, to a new paradigm.

The long-term objective is to design and develop a system of robots and associated human-robot interfaces (HRI) appropriate technology for all responders throughout the human response command hierarchy by first understanding the human activities, rather than starting with the technology design and then evaluating its effectiveness after development.

Remote Operator Interface an Operator/Supervisor Interaction Role

Operations Chief Interface an Abstract Supervisor

Cognitive Information Flow Analysis (CIFA)

Curtis M. Humphrey — Fri, 05 Jun 2009 20:32:16 +0000

Cognitive Infomation Flow Analysis for Emergency Evaluation

The Cognitive Information Flow Analysis (CIFA) is a method to combine results from multiple cognitive task analyses while providing a focus on the necessary system information flow, which includes how information is produced, consumed, and transformed by the various system functions and users. This project explored the CIFA concept, conducted a case study that applies the CIFA method to existing Goal-Direct Task Analysis and modified Cognitive Work Analysis results, and investigated CIFA’s use for informing the design of a system of human-robot interfaces.

CIFA can serve as a guide for system design and development. CIFA also provides a number of advantages over combining cognitive task analysis results using relational tables. First, CIFA can express the interconnectivity of the various system subcomponents, including partial ordering and parallelism, by fundamentally focusing on the information flow. Second, CIFA can identify both existing and potential, information bottlenecks and highlight teamwork.

Related Papers:

Humphrey, C. M., & Adams, J. A. (2009). Cognitive Information Flow Analysis: Definition and Case Study. Manuscript submitted for publication.

Emergency Response System Human-Robot Interaction User Levels

Curtis M. Humphrey — Fri, 05 Jun 2009 20:37:36 +0000

Emergency Response System Human-Robot Interaction User Levels

This project formulated how the large numbers of individuals that can be involved in an emergency response event can be encapsulated into ten different Emergency Response System Human-Robot Interaction User Levels that represent the various manners in which responders will interact with future robotic systems. The ten user levels were informed by existing HRI roles, domain relevant documents, subject matter expert interviews, and GDTA and CWA results. This project also originated a new human-robot interaction role: the abstract supervisor role.

Related Papers:

Humphrey, C. M., & Adams, J. A. (2009). CBRNE User Levels: Introducing the Abstract Supervisor Human-Robot Interaction Role and Case Study. Manuscript in preparation.

Multiple Robot Interfaces Scalability and the Halo Concept

Curtis M. Humphrey — Fri, 05 Jun 2009 20:50:11 +0000

Robot Interface Scaliablity and Halo Concept

As multiple robot systems become more common, it is necessary to develop scalable human-robot interfaces that permit the inclusion of additional robots without reducing the overall system performance. This project focused on the development of a scalable interface for a single human-to-multiple robot system with the intention to promote situation awareness and the management of multiple robots by combining the two types of displays. This project also explored a relational “halo” display that augments a camera view to promote situational awareness (SA) and the management of multiple robots by providing information regarding the robots’ relative locations with respect to a selected robot.

A user evaluation was conducted to determine the scalability of the interface focusing on the effects of increasing the number of robots on workload, situation awareness, and robot usage. The evaluation results indicate that the interface scales better than hypothesized. The evaluation highlights a tradeoff between workload and SA, as the number of robots increased so did the workload and SA, a result that contradicts prior research results. The increased workload was significantly lower than hypothesized while the increase in SA was contradictory to the hypothesis. The increase in SA indicates that developing interfaces that support larger numbers of robots may result in increased SA; however further analysis is required to ensure that these results are not due to limitations associated with the 3D SART tool.

Related Papers:

Humphrey, C. M., Henk, C., Sewell, G., Williams, B. W., & Adams, J. A. (2007). Assessing the scalability of a multiple robot interface. In Proceedings of the ACM/IEEE international conference on Human-robot interaction (pp. 239-246). Arlington, Virginia, USA: ACM. doi: 10.1145/1228716.1228749.