How our AI works, what it sees, and how we keep it fair

We ship two AI-assisted features that affect candidate evaluation, plus one structured-extraction utility:

The single most common question we get is “does the AI know my race / gender / school ranking?” The answer is no for protected characteristics. The honest, complete answer is below.

For interview copilot, the data scope is restricted to the audio of that interview only. Past interviews from other firms or other candidates do not influence the assistant’s suggestions.

Algorithmic bias does not announce itself. We treat every AI-touched output as a potential source and run the following checks on a recurring basis:

1. NYC LL144 four-fifths rule audit - before the prescreen tool is enabled for any NYC-scope role, an independent auditor evaluates the selection rates by demographic group on a rolling 12-month sample. If any group’s selection rate falls below 80% of the highest-rate group, the tool is gated until a model retrain or threshold change resolves the gap. Results are posted at /legal/nyc-aedt-notice.
2. Monthly demographic parity check - we sample prescreen outputs across a month and compare the distribution of scores against the application population, controlling for the law-school cluster the candidate is in. If the spread on any group exceeds the band we expect from noise alone, an alert fires and the tool is paused until an engineer + recruiting-ops lead sign off.
3. Hold-out validation on every model update - before any change reaches production, we evaluate it on a hold-out set that includes adversarial examples (different writing styles, non-traditional backgrounds, immigrant / non-native English applicants, applicants who chose to not disclose school name) to ensure the new model does not regress on those cohorts.
4. Drift dashboard - the operations team monitors a live dashboard of score distributions, complaint rates, and override rates (how often firm reviewers disagree with the AI). A spike in any of these is treated as a signal to investigate, not noise.
5. External validation - once a year, a third-party reviewer audits the prescreen tool methodology and validates that the published bias-audit results reflect what the system actually does in production.

No AI output on Big Law Bear directly causes an outcome for a candidate. Specifically:

• A prescreen score does not auto-reject anyone. It surfaces alongside the application for a human reviewer to consider, accept, or ignore.
• Interview copilot suggestions do not produce a candidate score, hire/no-hire flag, or ranking. They are conversational prompts the interviewer can use or skip.
• Resume-parsing extractions do not auto-disqualify a candidate based on missing fields. Every extracted field is editable.

Big Law Bear logs every override (recruiter disagreed with the AI) so we can study patterns. Persistent one-direction overrides on a particular cohort would be a signal of bias and trigger the parity check above.

• Opt out of AI prescreen by writing to us via /help/contact (category: AI fairness). We mark your applications as no-prescreen for the current cycle. Your application stays in the queue; only the AI notes are suppressed for the firm. A self-serve toggle in account settings is on the roadmap.
• Decline AI interview consent (Illinois and any candidate who chooses) means the assistant is disabled for your interviews. See /legal/ai-interview-consent.
• Request what we have on you - under GDPR / CCPA you can request a copy of your data, including any AI-generated notes, by emailing privacy@biglawbear.com. We respond within 30 days.
• Challenge an outcome - if you believe an AI-assisted decision was unfair, write to the same address. Big Law Bear will review the logged AI output, the human override decision, and respond with what we found.
• Adverse-action notice - if a firm using Big Law Bear declines you and AI was a contributing input, the firm is responsible for providing the notice required by the FCRA (where applicable) and any state law equivalents.

You are the “employer” under EEOC guidance, NYC LL144, IL AIVA, and equivalent state laws. Big Law Bear is the tool vendor. Your firm is responsible for the candidate-facing notices and any required disclosures. We provide:

• Pre-built AEDT candidate notice email that goes out automatically on application submission for NYC-scope roles, with links to this page and /legal/nyc-aedt-notice.
• Per-firm AIVA consent capture (voice memos, post-call transcription, copilot scopes). AEDT opt-out is tracked separately on the candidate notice record.
• Audit-trail rows on every prescreen run (provider, model, tokens, cost, partner action of agreed / overrode), and on every AI interview artifact (voice, transcript, copilot). Exportable on request.
• Bias-audit input form at /portal/settings/compliance where your firm uploads the URL of its annual third-party bias audit. A surfaced firm-side compliance dashboard (notice-served rate, model-run gating) is planned but not yet shipped.

If you think an AI feature on Big Law Bear is unfair, opaque, or simply wrong, we want to know.

File a report at /help/contact (category: AI fairness). It lands in our admin triage queue and is reviewed by the engineering and recruiting-operations leads jointly. We treat fairness reports as P0 incidents and respond within two business days. The reporter is not identified to the firm using the tool.