02 of 09Ethical × Innovation Purpose

Hard stops

Is there a line you will not cross for money?

Most ethical failure starts as a series of small trades, each one reasonable, where a harm gets priced, offset, and carried forward. A hard stop refuses to price. The list is written and dated before the opportunity arrives. A list written afterwards is an argument.

Reading for Purpose
Weapons of Math DestructionCathy O'Neil, 2016. How optimization at scale encodes harm nobody authored.
Ruined by DesignMike Monteiro, 2019. The builder's own responsibility for what the built thing does.

What the practice requires

  • The stop list exists in writing, dated and published.
  • It is authored before diligence on any specific venture. Amended on a cadence, never in deal week.
  • A trigger stops the work. It does not open a negotiation about mitigation.
  • The stop is recorded with its reason, so the next team sees the precedent.
  • We report the count of ventures stopped. A screen that has never stopped a venture has not been tested.

Why we refuse offsets

  • An offset assumes the harm and the benefit are in the same unit and land on the same people. They usually are not.
  • Permit one offset and the list becomes a price sheet. Then it becomes a budget line.

Where it goes wrong

  • Writing the list vague enough to survive any deal.
  • Putting stop authority inside the team whose pay depends on the deal closing.
  • Treating an internal finding of harm as a communications problem.

Measures

  • Stop list published, dated, version-controlled. Yes or no.
  • Ventures stopped by the list, per year, with reasons recorded.
  • Hours from trigger identified to work stopped. Target: same working day.
  • Offset transactions permitted. Target: zero, permanently.
  • Share of stop decisions made by someone with no economic interest in the venture.

Three cases

These cases come from the public record: settlements, filings, testimony and the companies' own reporting. Citations are being attached case by case. Good, bad and ugly describe one decision against this one question, not the company as a whole.

Purdue and the marketing of an epidemic
UGLY

Purdue and the marketing of an epidemic

Aggressive OxyContin marketing contributed to an epidemic linked to over 500,000 U.S. deaths. The information existed the whole time. There was no stop it could trigger.

The teen-research file
UGLY

The teen-research file

Internal research showed Instagram made some teen girls feel worse about themselves. The company chose not to publish it and did not change the product. The stop existed on paper with no authority behind it.

Palantir's ICE contract
UGLY

Palantir's ICE contract

FOIA documents showed the software was used in ICE workplace raids, contradicting the company's public description of the contract. A stop list that can be negotiated stops nothing.

How this one is marked

Green means we have the evidence and it carries a date. Yellow means someone owns the gap and there is a date to close it. Red stops the work until it clears. Early work sits at yellow for a long while, and we mark it that way.

Green: evidenced and dated
Yellow: a work order with an owner and a date
Red: absent or contradicted