Dynamic cloth folding

I worked on this project during my Master’s at Aalto University, with David Blanco-Mulero, Gokhan Alcan, and Ville Kyrki. Our paper, Learning Visual Feedback Control for Dynamic Cloth Folding, was an IROS 2022 Best Paper Award finalist.

The problem

A dynamic fold uses the motion of the robot to move the free end of the fabric. Different materials respond differently, so a trajectory that works for one cloth may fail on another. We trained reinforcement learning policies in simulation, using visual feedback and randomized material properties, and evaluated their transfer to real fabrics.

Making that experiment work also required aligning the camera, controller, and simulator. These are three of the engineering decisions I documented after the paper.

The policy and training setup

We trained with Soft Actor-Critic, Hindsight Experience Replay, and demonstrations in the replay buffer. The actor takes a 100 × 100 grayscale image, the goal, and the previous action. A CNN encodes the image; a fully connected head predicts Cartesian displacement. An auxiliary head predicts cloth points during training.

The critics use simulated cloth positions and velocities, while the actor uses observations available on the real robot. This asymmetric setup lets training use privileged state without requiring it at deployment.

Measure latency before choosing the camera mode

The policy needed timely observations of the moving cloth. In preliminary simulation experiments, 10 Hz was the lowest feedback frequency that learned the task consistently. That gave us a working observation-latency budget of 100 ms.

Our RealSense D435 RGB measurements averaged about 120 ms. We switched to the high-speed infrared mode and measured the complete camera, policy, and ROS path with a modified latency tool.

The camera and policy ran in a separate ROS node so waiting for images would not block the robot’s 1 kHz control loop. Original sensor measurements →

Use the same controller in both environments

We chose Cartesian displacement actions because their effect on the gripper was easy to inspect and constrain. After comparing controllers in simulation, we used operational space control and implemented the control step in C++.

I wrapped that control logic with pybind11 so the Python training environment could call the same implementation. This reduced the risk of separately maintained controllers drifting apart. The real robot ran at 1 kHz and simulation at 100 Hz, so the simulated step included ten desired-position updates before calculating torque.

We replayed recorded demonstration trajectories in both environments and compared the resulting gripper paths. The simulated robot initially reacted more slowly. Investigating the model’s inertial properties and enabling MuJoCo’s inertiafromgeom brought the trajectories much closer together. The remaining tracking error was visible rather than hidden inside a policy-training result. Controller implementation and trajectory comparison →

Randomize plausible cloth dynamics

Unconstrained parameter sampling produced cloth that was excessively damped or jittery. We first ran a successful demonstration trajectory against sampled cloth models, ranked them by the final folding reward, and retained twenty parameter combinations. Training then sampled from that set.

We also varied textures, lighting, camera pose, field of view, and image blur. The aim was to expose the policy to variation while keeping the dynamics relevant to the task. Parameter selection and visual randomization →

What the experiments established

The policy transferred directly from simulation and folded different fabric types on the real robot. The paper evaluates the contributions of visual feedback and material randomization. The experiments used a defined starting setup and evaluated folding across different materials.