Source Models Leak What They Shouldn't

Teaser Figure comparing Conventional MU and Proposed SCADA-UL

Comparison of Conventional MU and Our Proposed Method. Conventional class-wise machine unlearning focuses on forgetting a subset of classes from a trained model. We propose SCADA-UL: Unlearning Source-exclusive Classes in Domain Adaptation aims to remove knowledge of source-exclusive classes (classes absent in target domain) while adapting a model to a new domain. For instance, if a land-use categorization model is adapted to a new geography, sensitive classes such as airports must not be transferred to the target domain.

The Privacy Risk in Domain Adaptation

When models are adapted to new domains (like satellite imagery or medical scans), they may inadvertently retain and leak sensitive, source-domain specific information—even when those classes are not represented in the new target data. For example, a disease diagnosis model initially trained on a dataset containing both mental and physical health conditions might be deployed in a setting where patient privacy policies prohibit the use of mental health data. Retaining these classes poses severe privacy risks including unintended information leakage or re-identification.

Source Hospital

Target Hospital

Physical
Chest X-Ray

Physical
Transferred

Mental
fMRI Scan

Sensitive

Transfer

Prohibited
Mental Data

Adapt

Zero-Shot Leakage

Methodology

We adapt a source trained model to a target domain while forgetting specific classes. We create initial adversarial samples by maximizing their probability of belonging to the forget classes. In subsequent iterations, the model minimizes the MU loss using a novel rescaled labeling strategy alongside the SFDA loss.

To address SCADA-UL, we propose an approach that performs simultaneous unlearning and domain adaptation. However, these two goals are naturally misaligned: adaptation transfers source domain classes to the target, while unlearning seeks to avoid the transfer of specific source-exclusive classes.

Adversarial Forget Samples: Without access to the source data, we generate representative samples for the "forget" class using adversarial optimization directly in the input space.
Novel Rescaled Labeling: To induce unlearning with minimal conflict during adaptation, we expose the model to false information—redistributing the output probabilities of the forget sample proportionally among the retain classes.
Joint Optimization: An iterative min-max optimization is used, where the adversarial sample generation evolves alongside the adapting model.

Mathematical Formulation

1. Adversarial Forget Sample Generation: We generate representative samples $\hat{x}$ for the "forget" class $c_{\mathcal{F}}$ by maximizing their cross-entropy loss ($\mathcal{L}_{CE}$) probability:

$$\mathcal{L}_{ADV}(w^{\mathcal{T}}, \hat{x}) = \mathcal{L}_{CE}(w^{\mathcal{T}}, \hat{x}, c_{\mathcal{F}})$$

2. Rescaled Labeling Strategy: To induce unlearning with minimal conflict, we redistribute the original output probabilities $y$ proportionally among the retain classes $\mathcal{C}_{\mathcal{R}}$:

$$\hat{y}_i = \begin{cases} 0 & \text{if } i = c_{\mathcal{F}} \\ \frac{y_i}{\sum_{j \in \mathcal{C}_{\mathcal{R}}} y_j} & \text{if } i \in \mathcal{C}_{\mathcal{R}} \end{cases}$$

3. Adversarial Optimization: The optimization is adversarial because the model minimizes the unlearning loss to "forget" the sample, while the sample simultaneously updates its pixels to maximize its resemblance to the forget class. Specifically, the target model ($w^{\mathcal{T}}$) is updated to minimize the domain adaptation loss ($\mathcal{L}_{SFDA}$) and the unlearning loss ($\mathcal{L}_{MU}$), while the adversarial samples ($\hat{x}$) are continuously updated to minimize $\mathcal{L}_{ADV}$.

$$\min_{w^{\mathcal{T}}} \Big[ \mathcal{L}_{SFDA}(w^{\mathcal{T}}, x^{\mathcal{T}}) + \alpha \mathcal{L}_{MU}(w^{\mathcal{T}}, \hat{x}, \hat{y}) \Big]$$

simultaneously with

$$\min_{\hat{x}} \mathcal{L}_{ADV}(w^{\mathcal{T}}, \hat{x})$$

Key Results

SCADA-UL consistently reaches the ideal high-retain, high-forgetting region across datasets, closely matching the retrain oracle while outperforming prior methods.

We also extended our approach to two highly practical variants: C-SCADA-UL (Continual SCADA-UL) for scenarios where classes must be forgotten sequentially across multiple requests, and UC-SCADA-UL (Unknown Class) for instances where the specific source-exclusive classes to be forgotten are initially unknown.

Conclusion

Our comprehensive suite of experiments across multiple datasets (OfficeHome, Office31, DomainNet, and real-world medical/satellite datasets) demonstrates the superiority of our adversarial optimization framework over existing Data-Free MU and Partial Domain Adaptation (PDA) baselines.

Retraining-Level Unlearning

Consistently achieves forget accuracies and unlearn scores comparable to a gold-standard model retrained from scratch without the sensitive data.

Eliminates Zero-Shot Leakage

Successfully prevents the model from retaining zero-shot transfer capabilities for source-exclusive classes in the target domain.

Simultaneous Adaptation

Maintains high performance on "retain" classes while adapting robustly to new target domain distributions.

BibTeX

@misc{devalapally2026oslashsourcemodelsleak,
      title={$\oslash$ Source Models Leak What They Shouldn't $\nrightarrow$: Unlearning Zero-Shot Transfer in Domain Adaptation Through Adversarial Optimization}, 
      author={Arnav Devalapally and Poornima Jain and Kartik Srinivas and Vineeth N. Balasubramanian},
      year={2026},
      eprint={2604.08238},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2604.08238}, 
}