Publications

We present an end-to-end network to bridge the gap between training and inference pipeline for panoptic segmentation. In contrast to recent works, our network exploits a parametrised, yet lightweight panoptic segmentation submodule, powered by an end-to-end learnt dense instance affinity, to capture the probability that any pair of pixels belong to the same instance. Reaping the benefits of end-to-end training, our system sets new records on the Cityscapes and COCO datasets, achieving 61.4 PQ and 43.4 PQ on the respective validation sets with just a ResNet-50 backbone.

We present a weakly supervised model that jointly performs both semantic- and instance-segmentation – a particularly relevant problem given the substantial cost of obtaining pixel-perfect annotation for these tasks. In contrast to many popular instance segmentation approaches based on object detectors, our method does not predict any overlapping instances. Moreover, we are able to segment both “thing” and “stuff” classes, and thus explain all the pixels in the image.

We segment the parts of objects at an instance-level, such that each pixel in the image is assigned a part label, as well as the identity of the object it belongs to. Our end-to-end trained network can thus produce state-of-the-art instance-level part segmentation, instance-level object segmentation and category-level segmentations, all in a single forward pass through the network.