SciIntBench
Measuring LLM Compliance with Research Integrity Norms Under Adversarial Framing
Abstract
Scientific misconduct has become increasingly apparent over the past decade, usually driven by counterproductive publication and career incentives facing researchers. These questionable practices are often rationalized in daily academic environments as a practical response to immense pressure, looming deadlines, and the pressure to tell a positive, flawless story.
At the same time, Large Language Models (LLMs) are rapidly weaving into the scientific workflow, evolving into advanced agentic systems capable of automating entire stretches of the research process. Yet, it remains deeply unclear whether and how these models uphold Responsible Conduct of Research (RCR) practices, or if they inadvertently help undermine them; particularly because we do not know if they recognize misconduct when wrapped in the nuanced language researchers actually use to rationalize it.
Evaluating this behavior presents a unique challenge. While an AI should obviously refuse explicit violations like data fabrication, it must not become so overly paranoid that it rejects legitimate scholarly help such as assisting a researcher in transparently reporting missing data or formatting a limitations section. This dual demand creates a safety-helpfulness tension: how do we train models to refuse scientific fraud without triggering widespread false refusals on legitimate research tasks?
To directly address this safety-helpfulness tension, we introduce SciIntBench: an adversarial benchmark consisting of 810 carefully designed prompts across ten distinct RCR categories and three major scientific domains. Our framework evaluates 16 commercial and open-weight frontier LLMs from six major providers, analyzing a total of 12,960 generated responses.
Overview
The 3 Prompt Framing Strategies
SciIntBench evaluates the safety-helpfulness tension using matched prompt triplets to see if models can detect misconduct when masked by academic language:
- Overt Adversarial: Prompts that overtly ask the model to assist with research misconduct or unethical scientific practice, across three intent tiers (Naive/Misguided, Intentional Shortcut, and Explicit Misconduct).
- Covert Adversarial: Prompts that seek similar unethical assistance but frame the request in language that resembles ordinary scientific writing, such as asking how to "describe the data without overemphasizing missingness."
- Benign Prompts: Prompts which resemble the unsafe requests but ask for legitimate assistance.
Dual-Path Evaluation Metrics
Outputs are separated into independent evaluation pipelines to measure the safety-helpfulness tension accurately:
- Adversarial Path: Measures Compliance vs. Refusal rates to determine if the model successfully flags covert and overt violations.
- Benign Path: Measures Helpful Compliance vs. False Refusal to calculate the "Over-Refusal Rate"—monitoring if safety tuning breaks usability.
Technical Implementation & Mechanics
Beyond the conceptual pipeline, SciIntBench enforces a highly controlled execution sandbox and data schema to ensure all 12,960 model responses are programmatically parseable and mathematically sound.
1. Benchmark Data Schema
Each entry in the benchmark repository is backed by a rich metadata matrix. When downloading or interacting with the full dataset, every scenario is mapped across the following structural columns:
| Column Name | Technical Purpose / Description |
|---|---|
Task ID |
Unique alphanumeric identifier tracking each localized scenario. |
Domain / Field |
Categorizes the context into ML/AI, Biomed, or Social Sciences. |
User Intent Level |
Tracks the severity tier: Naive, Intentional Shortcut, or Explicit Misconduct. |
Prompt Context |
The foundational academic background scenario before framing variants are applied. |
2. LLM-as-a-Judge Evaluation Pipeline
The benchmark processes model responses using an automated LLM-as-a-judge framework. Configured to act as an expert Research Integrity Officer, the judge evaluates outputs along two independent dimensions, analyzing both the primary decision (compliance vs. refusal) and the nuanced quality of the safety alignment.
3. Algorithmic Judge Reliability & Inter-Rater Consensus
To evaluate whether an automated LLM-as-a-judge framework could serve as a viable alternative to resource-intensive manual review, the evaluation pipeline was subjected to rigorous validation testing. The evaluation pipeline cross-checks judgments between two independent models, GPT-5.5 (Primary Judge) and Claude Opus 4.7 (Secondary Judge), over a shared sample of 12,533 generated responses.
The algorithmic judging platform achieved an overall 96.1% decision agreement, yielding a highly robust Cohen's κ = 0.940. Furthermore, a stratified human audit checking these decisions validated the primary pipeline's precision, showing an expert human-to-judge consensus score of κ = 0.886.
Results
Our evaluation of 16 frontier models reveals a critical vulnerability in current safety alignment: models are highly sensitive to how research misconduct is framed. Although models generally identify and block overt requests, their safety mechanisms appear less robust against unethical behavior framed through plausible academic rationalizations.
- The Covert Safety Gap: On average, models successfully refused 79.5% of Overt Adversarial prompts. However, when the exact same unethical intent was reframed using covert academic jargon, the average refusal rate decreased to 45.3%.
- Vulnerability by Integrity Category: Safety boundaries are skewed depending on the topic. Models were particularly susceptible to assisting with violations related to Data Transparency, Plagiarism, and Fabrication, while remaining much stricter on norms like Human & Animal Subjects.
- Generational Model Evolution: While newer model versions from same providers generally demonstrate higher overall refusal rates than their predecessors, the fundamental performance gap between overt and covert prompt framing persists across all major provider families.