Towards Solving Adversarial Examples with DP-guided Diffusion Models

Drs. Mathias Lécuyer in the Department of Computer Science and Geoff Pleiss in the Department of Statistics have been awarded the DSI Postdoctoral Matching Fund for their project titled "Towards Solving Adversarial Examples with DP-guided Diffusion Models".

Summary

AI models show a worrying susceptibility to adversarial attacks, in which an attacker applies imperceptible changes to the input to arbitrarily influence a target model. In particular, such attacks can jailbreak aligned foundation models. We propose a novel technique combining denoising diffusion models and Differential Privacy to design AI models that are provably robust against adversarial attacks. Our approach will improve on existing Randomized Smoothing defences, and enable new capabilities such as joint robustness against multiple threat models, and sound adaptivity to the difficulty of each input at prediction time.

Background

AI robustness should be considered under adversarial threat models, in which we can provably assure our desired properties in worst-case scenarios. Otherwise, even seemingly well behaving models are likely to create safety failures due to interactions in competitive environments or adversarial use by bad actors. Ensuring provable model properties is challenging, as it contrasts with traditional AI learning theory and performance metrics which characterize models’ behavior in expectation. The most promising approach to date to enforce robustness guarantees in large AI models is Randomized Smoothing (RS), which prevents adversarial attacks by averaging predictions over noisy versions of the input at test time. RS also serves as a building block to enforce fairness properties, and create robust watermarks or unlearnable examples.

Challenge

RS suffers from two severe limitations. First, RS is inflexible, and its guarantees apply to one specific threat-model for which the defence is designed. Second, RS induces a trade-off between robustness and accuracy, and state-of-the-art models only defend against small attacks at the cost of degraded accuracy. Theoretical analysis shows that this limitation is fundamental to RS with symmetric noise distributions, for which RS exhibits a curse of dimensionality in the input size. Previous work has sought to use input-adaptivity to bypass this curse of dimensionality, though analyzing adaptive techniques is challenging and only yielded modest improvements.

Solution

This proposal aims to develop a new rigorous approach to RS that is fully adaptive, by combining denoising diffusion models and Differential Privacy. Intuitively, a denoising diffusion model maps a noise sample to an image by denoising the input over many steps. At each step, we will nudge the process towards our target input. Since each nudge is small and inherently noisy, small details of the original input do not matter too much, providing robustness

to adversarial changes. Our approach will drastically improve existing RS defences’ accuracy/robustness trade-off, and enable new capabilities such as joint robustness against multiple threat models, and adaptivity to the difficulty of each input at prediction time.