Methodology

§2How we calibrate

The calibration baseline is constructed from six pre-2022 sources with weighted contributions:

Each source has an estimated baseline contamination rate documented in the methodology archive. Aggregate weighted contamination is approximately 3.4%, with a range of 2.0-5.3% accounting for per-source uncertainty.

Four parallel calibration corpora are maintained. Rotation occurs quarterly through a deterministic seeded permutation. The active corpus identifier is disclosed in every Pulse publication footer and in the API response for every Cert.

Calibration baseline data is available for independent audit at deadmeter.com/transparency/datasets/calibration-baseline-v1.

§3Confidence intervals

Reported confidence intervals have two components.

Sampling uncertainty derives from bootstrap resampling at 10,000 iterations. This reflects variability when resampling input documents within the stream or text fragments within a single document.

Calibration baseline uncertainty derives from per-source contamination estimates documented in §2. This component is propagated through corpus weighting and GLM coefficients to produce a wider total interval.

The reported interval combines both components conservatively. Actual measurement uncertainty likely sits closer to the sampling-only interval, but we report the wider interval as honest disclosure of model assumptions.

Two-interval display is available. The narrow sampling-only interval appears prominently. The wider total interval is accessible via a ‘show calibration uncertainty’ control.

Confidence intervals do not bound error against motivated adversaries. This is documented in §4 and §10.

§4Adversarial testing

The methodology is validated against a public adversarial benchmark suite. Nine attacks across four stream sizes and four content domains produce 144 evaluation cells.

Attacks

Stream sizes: n=50, n=200, n=500, n=2000. Content domains: technical, casual, academic, creative. All apolitical.

Three-ring disclosure architecture

Quarterly rotation refreshes 25% of the holdout. Three of nine attacks rotate per quarter. High-impact attacks rotate more frequently than stable attacks.

Per-attack degradation factors are reported transparently. Methodology degrades gracefully under most attacks. The fundamental limit of recursive paraphrasing is acknowledged in §10.

Full benchmark results published at transparency.deadmeter.com/benchmarks.

§5Versioning

The methodology uses semantic versioning combined with cryptographic hashes.

Off-cycle minor increments may occur when a major adversarial discovery or significant accuracy regression triggers methodology update.

Every Cert and every Pulse publication includes the active methodology version and its SHA-256 hash. Historical methodology pages remain accessible at deadmeter.com/methodology/v1-0, v1-1, and so on.

§6Single-operator transparency

This work is produced by one person with one tester.

Methodology development is performed by Pavel Ishchin. Backend testing and infrastructure validation is performed by Daniil. Marketing and publication formatting is performed by Evelina. There is no formal institutional review board, no redundant validation pipeline, and no independent replication baked into the workflow.

This is disclosed openly because it affects how findings should be weighted. Single-operator measurements should be treated as one signal among several, not as definitive evidence. Corroborating evidence from independent sources should be weighted highly.

The methodology is published openly to allow independent replication. Reference implementation is available as a Docker container for 5+ year reproducibility. Calibration corpus access is documented.

§7Apolitical scope

Deadmeter does not measure or report on political content. No elections. No government information operations. No active conflicts. No partisan advocacy. No identity politics.

This is permanent commitment, not a temporary positioning. The apolitical scope is implemented through:

• ·Stream selection criteria that exclude political subreddits, political blog sources, and political forums
• ·A NER-based filter in BS Meter and Dead Internet Index that rejects inputs above a threshold density of named political entities
• ·Sponsor category restrictions that auto-reject political organizations and campaigns
• ·Coverage limitations on tools applicable to politically-charged content

The scope deliberately privileges defensibility over breadth. Three clean streams of technical discourse are stronger than four streams including a politically-charged one.

§8Limitations

1. 01

   Population-level inference only.

   All ecosystem scores describe statistical distributions across many documents within a monitored stream. They cannot label individual posts as AI-generated or human-authored at the document level with the same confidence.

2. 02

   Pre-2022 baseline contamination.

   Our reference corpus for human-only writing is drawn from 2018-2021 sources. Estimated AI-generated contamination of this baseline is 1-5%, meaning the absolute floor of all measurements is offset by an unknown small amount.

3. 03

   Adversarial uncertainty exceeds reported confidence intervals.

   CIs reflect statistical noise under typical conditions. They do not account for actively adversarial inputs (paraphrased AI text, mixed-authorship documents, stylometric obfuscation), where actual error can be substantially larger than the band suggests.

4. 04

   English text only (v1.0).

   The calibration corpus is English-language. Non-English documents are rejected at input validation. Multilingual support is out of scope for this release.

5. 05

   Apolitical methodology.

   No tool in the suite uses topic, sentiment, or political content as a feature. All measurements are structural-only. The system is intentionally blind to what a text is about.

6. 06

   Stream selection bias.

   The streams we monitor are chosen by our team based on coverage, accessibility, and platform diversity. They are not a random sample of the open internet. Generalization beyond the monitored set should be made cautiously.

7. 07

   Single-operator project.

   This work is produced and maintained by one person with one tester. We do not currently have institutional review, redundant validation, or independent replication. Treat findings accordingly and weigh corroborating evidence highly.

8. 08

   Detection ceiling on adversarial text.

   Following Sadasivan et al. (2023), no statistical detector can reliably distinguish AI-generated text from human-written text under sufficient paraphrasing. Our tools degrade gracefully under paraphrase attacks but cannot provide proof against motivated adversaries.

9. 09

   TPS_behavioral signals are observational proxies.

   The five behavioral signals (NGCS, CRJ, AAR, IPTD, EA) are calibrated on stream-level patterns. They have not been validated against ground-truth labeling at the individual account level and should not be used to infer intent or identity for any specific account.

10. 10

    Reservoir computing component is excluded from v1.0.

    The candidate fifth feature based on reservoir computing methodology is held back pending Phase 0 empirical validation. All scores reported in v1.0 use the four-feature configuration with the f3 by f4 interaction term only.

§9Multi-source closure

The methodology is anchored across multiple sources to avoid single-source failure modes.

Three primary anchors

• ·arXiv preprint (planned for publication within 30 days of launch)
• ·This methodology page (versioned, hash-anchored, immutable archive)
• ·Public adversarial benchmarks at transparency.deadmeter.com

When external pressure or methodology disputes occur, multiple independent verification paths exist. No single platform takedown or single regulatory action can erase the methodology record.

§10Sadasivan boundary

Sadasivan et al. (2023, arXiv:2303.11156) demonstrated a fundamental limitation: under sufficient recursive paraphrasing, any binary classifier for AI-generated text mathematically converges to random performance. This is not a flaw in any specific classifier. It is an information-theoretic property of the task.

For IMS_text (per-document Cert), the boundary applies directly. Death Certificate cannot provide an absolute verdict on authorship under recursive paraphrasing.

For IMS_ecosystem (population-level), the boundary applies indirectly. Aggregation over many documents bypasses the per-document boundary, but sufficiently sophisticated coordinated attacks can still degrade the aggregate signal substantially.

Additional impossibility results