Recorded in my evaluation tool demo. Both views show the same observation.
My contribution
I contributed to the evaluation and server tooling for the ICRA 2024 Cloth Competition. My merged pull request connects interactive segmentation, depth-based coverage measurement, dataset handling, and communication with participating teams.
From a mask to a measurement
The evaluator uses Segment Anything with a bounding box and optional positive and negative points to identify the cloth. The mask selects pixels in the depth image. Camera intrinsics and depth then give each pixel an estimated physical area, which is summed to obtain coverage.
For a pixel at depth z, the calculation is equivalent to area = z² / (fx × fy), where fx and fy are focal lengths in pixels. This accounts for scale changes with distance instead of simply counting foreground pixels. The implementation also exposes a depth-outlier threshold.
Make evaluation inspectable
The tool saves the mask, segmentation prompts, outlier threshold, and coverage result alongside the observation. That makes it possible to inspect how a score was produced and adjust a mask when necessary.
The server workflow lets teams download the latest observation and upload a grasp pose tied to a team and sample ID. I added client helpers and examples alongside the server integration.
Scope
This was a contribution to shared competition infrastructure. Coverage is a geometric proxy for unfolding, and depends on the mask and depth quality. It is not a general measure of manipulation success.
I am a co-author of the resulting dataset and benchmark paper in the International Journal of Robotics Research.