Publications

2022

1. Stagewise Newton Method for Dynamic Game Control with Imperfect State Observation

   Armand Jordana, Bilal Hammoud, Justin Carpentier, and 1 more author

   IEEE Control Systems Letters 2022

   Abs Link Code

   In this letter, we study dynamic game optimal control with imperfect state observations and introduce an iterative method to find a local Nash equilibrium. The algorithm consists of an iterative procedure combining a backward recursion similar to minimax differential dynamic programming and a forward recursion resembling a risk-sensitive Kalman smoother. A coupling equation renders the resulting control dependent on the estimation. In the end, the algorithm is equivalent to a Newton step but has linear complexity in the time horizon length. Furthermore, a merit function and a line search procedure are introduced to guarantee convergence of the iterative scheme. The resulting controller reasons about uncertainty by planning for the worst case disturbances. Lastly, the low computational cost of the proposed algorithm makes it a promising method to do output-feedback model predictive control on complex systems at high frequency. Numerical simulations on realistic robotic problems illustrate the risk-sensitive behavior of the resulting controller.

2. 

   iRiSC: Iterative risk sensitive control for nonlinear systems with imperfect observations

   Bilal Hammoud, Armand Jordana, and Ludovic Righetti

   In 2022 American Control Conference (ACC) 2022

   Abs Link Code

   This work addresses the problem of risk-sensitive control for nonlinear systems with imperfect state observations, extending results for the linear case. In particular, we derive an algorithm that can compute local solutions with computational complexity similar to the iterative linear quadratic regulator algorithm. The proposed algorithm introduces feasibility gaps to allow the initialization with non-feasible trajectories. Moreover, an approximation for the expectation of the general nonlinear cost is proposed to enable an iterative line search solution to the planning problem. The optimal estimator is also derived along with the controls minimizing the general stochastic nonlinear cost. Finally extensive simulations are carried out to show the increased robustness the proposed framework provides when compared to the risk neutral iLQG counter part. To the authors’ best knowledge, this is the first algorithm that computes risk aware optimal controls that are a function of both the process noise and measurement uncertainty.

3. 

   Exponential integration for efficient and accurate multibody simulation with stiff viscoelastic contacts

   Bilal Hammoud, Luca Olivieri, Ludovic Righetti, and 2 more authors

   Multibody System Dynamics 2022

   Abs Link Code

   The simulation of multibody systems with frictional contacts is a fundamental tool for many fields, such as robotics, computer graphics, and mechanics. Hard frictional contacts are particularly troublesome to simulate because they make differential equations stiff, calling for computationally demanding implicit integration schemes. We suggest to tackle this issue by using exponential integrators, a long-standing class of integration schemes (first introduced in the 1960s) that in recent years has enjoyed a resurgence of interest. This scheme can be applied to multibody systems subject to stiff viscoelastic contacts, leading to integration errors similar to implicit Euler, but at much lower computational costs (between 2 to 100 times faster). In our tests with quadruped and biped robots, our method demonstrated a stable behavior with large time steps (10 ms) and stiff contacts (105 N/m). Its excellent properties, especially for fast and coarse simulations, make it a valuable candidate for many applications in robotics, such as simulation, model predictive control, reinforcement learning, and controller design.

2021

1. 

   TriFinger: An Open-Source Robot for Learning Dexterity

   Manuel Wuthrich, Felix Widmaier, Felix Grimminger, and 8 more authors

   In Conference on Robot Learning 2021

   Abs Link

   Dexterous object manipulation remains an open problem in robotics, despite the rapid progress in machine learning during the past decade. We argue that a hindrance is the high cost of experimentation on real systems, in terms of both time and money. We address this problem by proposing an open-source robotic platform which can safely operate without human supervision. The hardware is inexpensive (about \SI5000[$]) yet highly dynamic, robust, and capable of complex interaction with external objects. The software operates at 1-kilohertz and performs safety checks to prevent the hardware from breaking. The easy-to-use front-end (in C++ and Python) is suitable for real-time control as well as deep reinforcement learning. In addition, the software framework is largely robot-agnostic and can hence be used independently of the hardware proposed herein. Finally, we illustrate the potential of the proposed platform through a number of experiments, including real-time optimal control, deep reinforcement learning from scratch, throwing, and writing.

2. 

   Leveraging Forward Model Prediction Error for Learning Control

   Sarah Bechtle, Bilal Hammoud, Akshara Rai, and 2 more authors

   In 2021 IEEE International Conference on Robotics and Automation (ICRA) 2021

   Abs Link

   Learning for model based control can be sample-efficient and generalize well, however successfully learning models and controllers that represent the problem at hand can be challenging for complex tasks. Using inaccurate models for learning can lead to sub-optimal solutions that are unlikely to perform well in practice. In this work, we present a learning approach which iterates between model learning and data collection and leverages forward model prediction error for learning control. We show how using the controller’s prediction as input to a forward model can create a differentiable connection between the controller and the model, allowing us to formulate a loss in the state space. This lets us include forward model prediction error during controller learning and we show that this creates a loss objective that significantly improves learning on different motor control tasks. We provide empirical and theoretical results that show the benefits of our method and present evaluations in simulation for learning control on a 7 DoF manipulator and an underactuated 12 DoF quadruped. We show that our approach successfully learns controllers for challenging motor control tasks involving contact switching.

3. 

   Impedance optimization for uncertain contact interactions through risk sensitive optimal control

   Bilal Hammoud, Majid Khadiv, and Ludovic Righetti

   IEEE Robotics and Automation Letters 2021

   Abs Link Code

   This letter addresses the problem of computing optimal impedance schedules for legged locomotion tasks involving complex contact interactions. We formulate the problem of impedance regulation as a trade-off between disturbance rejection and measurement uncertainty. We extend a stochastic optimal control algorithm known as Risk Sensitive Control to take into account measurement uncertainty and propose a formal way to include such uncertainty for unknown contact locations. The approach can efficiently generate optimal state and control trajectories along with local feedback control gains, i.e. impedance schedules. Extensive simulations demonstrate the capabilities of the approach in generating meaningful stiffness and damping modulation patterns before and after contact interaction. For example, contact forces are reduced during early contacts, damping increases to anticipate a high impact event and tracking is automatically traded-off for increased stability. In particular, we show a significant improvement in performance during jumping and trotting tasks with a simulated quadruped robot.

2020

1. 

   Crocoddyl: An efficient and versatile framework for multi-contact optimal control

   Carlos Mastalli, Rohan Budhiraja, Wolfgang Merkt, and 7 more authors

   In 2020 IEEE International Conference on Robotics and Automation (ICRA) 2020

   Abs Link Code

   We introduce Crocoddyl (Contact RObot COntrol by Differential DYnamic Library), an open-source framework tailored for efficient multi-contact optimal control. Crocoddyl efficiently computes the state trajectory and the control policy for a given predefined sequence of contacts. Its efficiency is due to the use of sparse analytical derivatives, exploitation of the problem structure, and data sharing. It employs differential geometry to properly describe the state of any geometrical system, e.g. floating-base systems. Additionally, we propose a novel optimal control algorithm called Feasibility-driven Differential Dynamic Programming (FDDP). Our method does not add extra decision variables which often increases the computation time per iteration due to factorization. FDDP shows a greater globalization strategy compared to classical Differential Dynamic Programming (DDP) algorithms. Concretely, we propose two modifications to the classical DDP algorithm. First, the backward pass accepts infeasible state-control trajectories. Second, the rollout keeps the gaps open during the early “exploratory” iterations (as expected in multiple-shooting methods with only equality constraints). We showcase the performance of our framework using different tasks. With our method, we can compute highly-dynamic maneuvers (e.g. jumping, front-flip) within few milliseconds.

2018

1. 

   Planar time-optimal paths for asymmetric vehicles in constant flows

   Bilal Hammoud, Salah Bazzi, Elie Shammas, and 1 more author

   The International Journal of Robotics Research 2018

   Abs Link

   This paper introduces a new kinematic model to describe the planar motion of an autonomous underwater vehicle moving in constant current flows. The vehicle is modeled as a rigid body moving at maximum attainable forward velocity with symmetric bounds on the control input for the steering rate. The model approximates the effect a flow will induce on the steering rate of the vehicle due to the asymmetric geometry of the vehicle. By imposing restrictions on the magnitude of the flow, the model is then used to characterize and construct the minimum-time paths that guide the vehicle from a given initial to a final configuration in the plane. Algorithms for the time-optimal path synthesis problem are also introduced, along with several simulations to validate the proposed method. Lastly, insights into how one would approach the energy-optimal problem are given, highlighting the fundamental differences in formulation and methods used to solve for the optimal paths.

2016

1. 

   Planar Time Optimal Paths for Non-Symmetric Vehicles in Constant Flows.

   Bilal Hammoud, and Elie A Shammas

   In Robotics: Science and Systems 2016

   Abs Link

   This paper introduces a new kinematic model to describe the planar motion of an Autonomous Underwater Vehicle (AUV) moving in constant current flows. The AUV is modeled as a rigid body moving at maximum attainable forward velocity with symmetric bounds on the control input for the turning rate. The model incorporates the effect a flow will induce on the turning rate of the AUV due to the non-symmetric geometry of the vehicle. The model is then used to characterize and construct the minimum time paths that take the AUV from a given initial configuration to a final configuration in the plane. Two algorithms for the time-optimal path synthesis problem are also introduced along with several simulations to validate the proposed method.