Conformance suite

The conformance suite is a single, subject-agnostic, black-box test that certifies any connector implements the ingestion contract correctly. Point it at a live ingest endpoint plus a scoped connector token and its website id, and it asserts spec compliance identically for every subject: a hand-built connector, the SDK, or an AI-built Universal AI Plugin pipeline.

It is the gate that keeps every connector signature-compatible and behavior-compatible with the server.

What a subject is

The runner is pure black-box HTTP. A subject is any live ingest endpoint plus credentials:

Field	Meaning
name	A label, for example generic-api or wordpress.
baseUrl	Origin only, for example https://api.bainquet.online.
connectorToken	A scoped connector bearer connectorId.secret.
websiteId	The token-scoped website id.
otherWebsiteId	A real, verified other website id, used to prove cross-site rejection.
siteDomain	The X-Site-Domain; must equal each item url host.

runConformance(subject) runs every case and returns { certified, results }. 100 percent green means certified.

What it checks

The suite asserts the frozen ingestion behavior, case by case:

Case	Clause	Asserts
Envelope and accept	T-EC-3.4 / 3.1	A valid signed batch returns 200 with the standard envelope, a per-item accepted status, and meta.requestId.
Idempotency	T-EC-3.1	Replaying the same item with a new idempotency key returns skipped with reason unchanged_checksum (or idempotent_replay), no duplicate.
HMAC tampered signature	T-EC-3.2	A tampered X-Signature returns 401 auth.invalid_signature.
HMAC stale timestamp	T-EC-3.2	An X-Timestamp outside the plus-or-minus 300s window returns 401 auth.timestamp_skew.
HMAC replayed nonce	T-EC-3.2	Reusing a nonce returns 409 connector.replay_detected.
Token scoping	T-EC-3.3	A cross-website website_id in the body returns 403 auth.scope_violation.
Batch partial semantics	T-EC-3.7	With partial: true, a bad item is isolated as a per-item error while good items are still accepted.
Intra-batch dedupe	T-EC-3.7	A duplicate (id, checksum) within one batch yields one accepted and the twin skipped.
Delete and tombstone	T-EC-3.5	POST /v1/ingest/delete tombstones a known id, reports unknown ids in not_found[], and returns a republish_job_id.

All nine cases pass for a correctly implemented connector. The delete and tombstone case (T-EC-3.5) is included: POST /v1/ingest/delete tombstones a known id, reports unknown ids in not_found[], and returns a republish_job_id.

Run it

The suite ships in the API at apps/api/src/seed/conformance.ts. The script entrypoint boots a live API, registers an organization, creates and force-verifies a website, issues a connector via REST, then runs the suite against it:

tsx src/seed/conformance.ts   # or: pnpm conformance

Exit 0 means certified; non-zero means a case failed. The output prints PASS or FAIL per case with a short detail line and a final N/total cases passed summary.

Use it as a CI gate

runConformance(subject) is a pure function with no bootstrap and no process spawn, so you can import it and run it against any deployed endpoint, for example as a CI gate on a connector pull request:

import { runConformance } from "@bainquet/api/seed/conformance";

const { certified, results } = await runConformance({
  name: "my-connector",
  baseUrl: "https://api.bainquet.online",
  connectorToken: process.env.BAINQUET_CONNECTOR_TOKEN!,
  websiteId: "site_xyz",
  otherWebsiteId: "site_other",
  siteDomain: "example.com",
  allowedSourceTypes: ["product", "page"],
});

if (!certified) {
  console.error(results.filter((r) => !r.passed));
  process.exit(1);
}

A connector that passes the suite signs requests the server accepts, handles idempotency and partial batches correctly, and respects token scoping and the replay guard. See Ingestion and signing for the scheme each case exercises.