Multimodal interfaces for robot teleoperation:

This research topic is focused on the design of novel multimodal robot teleoperation interfaces. While teleoperating a robot sharing the same physical area than the operator is typically straightforward, since the operator can follow the situation with his/her own eyes, when the teleoperation is remote, such as in the case of Search and Rescue and Space exploration, the physical distance poses several challenges, as the operator loses a considerable amount of situation awareness.

The design of our interfaces explore haptic modalities to improve situation awareness and reduce cognitive workload, and semi-autonomy to cope with communication latency and/or limited bandwidth. We participated in the AMADEE-20 analog mission, where we deployed a haptic teleoperation console, and we are currently engaged with the AMADEE-24 mission. We also collaborate with the European Space Agency (ESA) on the use of haptic interfaces targeting the teleoperation of planetary rovers from the International Space Station (ISS).

The research group involved both ISR-Lisbon and ITI, having background on both robotics and human-computer interaction, focused on the study of novel multimodal teleoperation interfaces, with emphasis on (but not limited to) space robotics.  The group comprises two faculty (Rodrigo Ventura and Jose Luis Silva), three PhD students, and many MSc students.

More information here: https://merop.isr.tecnico.ulisboa.pt/

Related Projects

  • ZoomDetails

    Space Robotics

    Space Robotics: Space is challenging environment, not only in what concerns the potential of orbital applications, but also from an exploration point of view. It is also an inhospitable environment for human presence, thus making a particularly interesting domain for robotic presence. Space Robotics activities encompasses two tracks: the orbital track, targeting robot vehicles in…
  • ZoomDetails

    Energy Management in Building Spaces

    CtxtBasedMod In previous research, where we proposed using a reinforcement learning (RL) approach to optimized the operation of the Heating Ventilation and Air Conditioning (HVAC) system. Context-based models where the result of our struggle to find suitable representations for the environment while organizing ideas that were being proposed in different research fields (hybrid systems, lumped…
  • ZoomDetails

    Cognitive Architectures

    Cognitive Architectures In the engineering of machine intelligence one cannot disregard natural inteligence that humans, together with other animals, display. Intelligence is here understood in the broad sense of behaving appropriately situated in an environment, as in the case of a fight-or-flight response for securing survival, as well as in the case of a mathematician…
  • ZoomDetails

    Cooperative Active Perception

    Cooperative Active Perception One of the recent research topics at the Intelligent Systems Lab is cooperative active perception. In our context, cooperative perception refers to the fusion of sensory information between fixed surveillance cameras and robots, with as goal maximizing the amount and quality of perceptual information available to the system. This information can be…
  • ZoomDetails

    Planning under Uncertainty

    For single agents, such planning problems are naturally framed in the partially observable Markov decision process (POMDP) paradigm. In a POMDP, uncertainty in acting and sensing is captured in probabilistic models, and allows an agent to plan on its belief state, which summarizes all the information the agent has received regarding its environment. For the…
  • ZoomDetails

    Discrete Event Systems Based Robotic Task Modeling

    Discrete Event Systems Based Robotic Task Modeling Most of the existing robotic task models are not based on formal approaches, are concerned only with a small number of behaviors and are typically tailored to the task at hand. We have proposed, back to 1998 [1] a systems-theory-based task modeling approach for general robotic tasks which…
  • ZoomDetails

    Formation State Estimation and Control

    Formation State Estimation and Control Multi-robot formation feasibility, formation control and formation state estimation have been subject of research at the ISLab since 2000. Formation State EstimationObstacle Compliant Formation ControlFormation FeasibilityFormation State Estimation An integrated approach to Guidance, Navigation and Control (GNC) of formation flying spacecraft was developed under the project. The design process considers…
  • ZoomDetails

    Field Robotics

    Field Robotics ISLab has been developing, together with the Mobile Robotics Lab, and under the Theme B of ISR - Associate Lab, field robotics activities and applications, motivated by their novel scientific and technical challenges (such as the cooperation among land and aerial robots, and among robots and humans, in unstructured, frequently outdoors, scenarios), relevance…
  • ZoomDetails

    Institutional Robotics

    Institutional Robotics Institutional Robotics is a new strategy to conceptualize multi-robot systems, which takes institutions as the main tool of social life of robots with bounded rationality and bounded autonomy. This institutional approach intends to get inspiration from philosophical and social sciences research on social and collective phenomena, and is mainly inspired by concepts from…