Cognitive Informatics at MTA SZTAKI

This blog post is about some of the work that’s being done at the Cognitive Informatics Research Group at MTA SZTAKI, in Budapest, Hungary. Hashimoto Lab is in loose but constant collaboration with this lab.

If we were asked to summarize what were are trying to do at the Cognitive Informatics Research Group, we would probably say that we are working on enhancing the flexibility of telemanipulation. Telemanipulation is the act of remotely controlling an actuator (which is located in a distant or possibly hazardous environment that cannot easily be accessed by the operator at any rate). A conceptual diagram of traditional teleoperation can be seen on Figure 1. As we can see, the operator handles a local master device, which is usually a replica of the actuator and is connected with the real actuator (the slave device) through telecommunication channels. Traditional teleoperation introduces many control and stability issues, to which many solutions have been proposed and these solutions have contributed to a huge body of literature on the subject.

Figure 1

Our goal is to enhance the teleoperation experience by introducing new channels of communication between the operator and the master device. In more traditional environments, generally we use exact replicas of the slave devices, and the modes of interaction between the operator and the master devices are analogous to the modes of operation that the slave device would require. In order to enhance these modes of interaction, we create an environment that aims to put the brain in the loop, as we like to say. The conceptual diagram of the master side of the telemanipulation setup can be seen in figure 2. In the figure, non-conventional communication channels are modes of communication that use sensory modalities to convey information that are different from the sensory modalities normally used for the task. For example, we could imagine force feedback to be provided through the visual cues or through audio. Provided that the channels are designed well, and the coding techniques used to code the transferred information are light enough not to overburden the operator’s cognitive capacities, the operator can quickly learn how to use the channel efficiently.

Figure 2

This example highlights an important aspect of our telemanipulation setup. By decoupling the feedforward (control) path from the feedback channels, we alleviate stability problems that could be caused by using the same device for actuation and feedback. In addition, by introducing a virtual reality between the master and slave side, we provide a means for software-based actuation. The software can be distributed in nature, so that several parties can contribute to the telemanipulation tasks simultaneously. Problems caused by feedback delays can also be alleviated by such a system, if blocking mechanisms are incorporated within the virtual reality in order to keep the operator from manipulating the system when it is still in a transient state.

Within the framework presented here, our research can be broken into the following 3 main branches:

• Cognitive actuation

• Cognitive communication channels for feedback

• Eto-robotics

These research directions, while somewhat disparate, all contribute to the same goals of providing more flexible means for (remote) human-robot interaction.

1. Cognitive actuation

We are developing a 3D virtual environment like the one shown in figure 2. It is highly modular and has well-defined interfaces for actuation devices. Any device can be incorporated into the virtual reality environment, provided that it is programmed to implement these interfaces. Currently we can control remote transportation robots by using more expensive devices such as position tracking data suits, or cheaper ones such as Nintendo Wii controllers and iPhones. The virtual environment has a distributed architecture, people can log on from different locations, connect their actuation and/or feedback devices (provided these devices implement a generic interface), and control devices that have been connected from different locations. Other forms of actuation include path control based on automated matching of human hand movements to CAD models.

In order to facilitate the control of various devices connected in various locations through our virtual environment, we have developed appropriate access mechanisms and distributed computational processing mechanisms.

1. Cognitive communication channels for feedback

Currently we have three projects within the area running in parallel: force feedback through peripheral vision, tactile feedback using applied models of tactile nerve endings, and tactile feedback through sound.

Force feedback through peripheral vision provides a vision-based feedback method that is minimally invasive. After a short learning period, the user can learn to interpret the visual cues without paying much explicit attention to them.

The models of tactile nerve endings used in our tactile feedback project involve descriptions of time-variant characteristic functions of different kinds of tactile receptors, and also their relative distributions.

Finally, our study of sound-based tactile feedback deals with two aspects: the kinds of sound parameters that can be used to convey physical parameters of surfaces (we find that optimal sound parameters depend on the dimension of tactile perception we would like to convey), and the ways these sounds can be combined so that the end result is meaningful and at the same time does not overload the user’s cognitive capacities.

1. Eto-robotics

Eto-robotics comes from the words ethology (the science dealing with animal behavior) and robotics. The goal of this research is to use models of animal behavior to create synthetic forms of robot behavior that users can sympathize with and relate to. The idea for this research comes from the realization that while humanoid robotics is a very impressive and dynamically evolving field, more often than not it produces robots that the user quickly becomes frustrated with. This is due to the contradiction between the user’s expectations and what the robots can do. The user’s expectations are unreasonably high because the robot looks like a human and is therefore expected to act like a human. According to some, the more human-like a robot becomes, the greater the tension will be between the user’s expectations and reality, because the robot will almost be human, but not a human (see uncanny valley effect).

The main idea of our research in eto-robotics is to create behavioral forms for robots that will help them to be perceived as a different species altogether. This way, the user will approach the robot with reduced expections, and a more realistic view of what kinds of tasks the robot can accomplish and how the user should interact with it in order to perform these tasks with greater efficiency. In a way similar to how we have learned to deal with cats, dogs and other pets (as a result of millennia of interactive experience), perhaps we can learn to interact with robots more efficiently by perceiving them as an eccentric, but somewhat loveable species.

Remember you can follow our updates in our Twitter Account