Key Takeaways

  • Safety by Design means building child protection into digital products from the very first line of code — not bolting it on after harm has already occurred.
  • The eSafety Commissioner's framework — three principles: Service Provider Responsibility, User Empowerment and Autonomy, Transparency and Accountability — is the global reference standard, cited by the EU, OECD, and multiple national regulators.
  • The OECD's 2024 report "Towards Digital Safety by Design for Children" identifies eight key implementation components that provide practical operational guidance.
  • Safety by Design is no longer voluntary. The EU Digital Services Act, UK Online Safety Act, UK Children's Code, California AADC, and Australia's Online Safety Act all mandate elements of it — with penalties reaching 10% of global annual turnover.
  • The business case for Safety by Design is strong: proactive design is significantly cheaper than reactive content moderation at scale, reduces regulatory risk, and builds user trust.
  • Implementation is achievable for companies of all sizes — starting with a risk assessment, applying targeted design interventions, and progressively building toward full compliance.

The Paradigm Shift: From Reactive to Proactive

For most of the internet's history, child safety online was framed as a content moderation problem. Build the platform. Let users generate content. Then hire a trust and safety team to find and remove the harmful material. This reactive model was baked into the infrastructure of the modern internet — and it has failed, comprehensively.

The numbers reveal why. NCMEC's CyberTipline received 88.3 million reports of child sexual abuse material in 2022 — a 329% increase over five years. Thorn's Safer platform processed over 42 billion images and videos for hash-matching in the same year. The Internet Watch Foundation's analysts reviewed millions of URLs. And still, harm to children online continues to accelerate. The reactive model cannot keep pace with the scale of user-generated content on modern platforms, the sophistication of bad actors who exploit platform design, or the speed at which harmful content spreads before moderation can catch it.

329%

Increase in CSAM files reported to NCMEC over five years, despite massive investment in reactive content moderation. The reactive model cannot keep pace with harm at platform scale.

Safety by Design is the alternative — and it is gaining momentum as the foundational paradigm for child online protection worldwide. Rather than responding to harm after it occurs, Safety by Design requires anticipating the ways a product could be used to harm children, or expose children to harm, and building safeguards in from the earliest stages of product development.

"You cannot moderate your way to child safety at internet scale. You have to design your way there — from the architecture up."

The shift is not just conceptual — it is regulatory. The EU Digital Services Act, UK Online Safety Act, UK Age Appropriate Design Code, California AADC, and Australia's Online Safety Act all embed Safety by Design requirements in binding law. Non-compliance now carries penalties that make proactive investment significantly cheaper than reactive enforcement fines.

The eSafety Commissioner's Three-Principle Framework

Australia's eSafety Commissioner — the world's first standalone government online safety regulator — developed the Safety by Design framework that has become the global reference standard. Built through extensive consultation with industry, civil society, and affected communities, the framework is organized around three principles that together create a comprehensive approach to child safety.

1

Service Provider Responsibility

The platform is primarily responsible for user safety — not users themselves.

The most fundamental shift in the Safety by Design paradigm is the reassignment of responsibility. Historically, platforms deflected responsibility for harms by pointing to user behavior: "users chose to share this content," "users chose to interact with this person." Safety by Design rejects this framing. Platforms that design systems which make harm predictable are responsible for the foreseeable consequences of that design.

This principle means:

  • Proactive risk assessment: Platforms must assess, before launch and continuously thereafter, the foreseeable ways their product could be used to harm children. A messaging feature, a direct-message function, a recommendation algorithm — all must be stress-tested for abuse pathways.
  • Default safety: The default experience for any user who might be a child must be the safest available configuration. Default settings define the experience of the majority who never change them — and children are disproportionately represented among non-technical users who accept defaults.
  • Built-in protective features: Safety mechanisms — content filtering, age-appropriate defaults, stranger-danger protections — must be built into the product architecture, not offered as optional premium features or buried in settings menus.
  • Responsibility for third-party content: Platforms cannot wash their hands of liability for user-generated content that harms children simply because they did not create it. If the platform's design enabled or amplified the harm, responsibility attaches.

This principle is directly encoded in the UK Online Safety Act's "duty of care" framework, the EU DSA's systemic risk requirements, and Australia's own Online Safety Act — all of which impose primary responsibility on platforms, not on users.

2

User Empowerment and Autonomy

Safety features must genuinely work for the users they are meant to protect.

Building safety features into a product is meaningless if children cannot find, understand, or use them. This principle requires that safety mechanisms be designed for actual children — not for hypothetical adult users who read terms of service and navigate complex settings menus.

In practice, user empowerment requires:

  • Accessible controls: Privacy settings, content filters, block and mute functions, and reporting tools must be prominently placed, simply described, and easy to use — accessible to a 9-year-old encountering a problem for the first time, under stress.
  • Age-appropriate interfaces: The language, design, and information architecture of safety features must be appropriate for the actual age range of users. Safety features designed for adult users are not appropriate for children.
  • Dark pattern prohibition: Platforms must not use manipulative design patterns — countdown timers, pre-checked consent boxes, friction-heavy privacy settings, "nudge" designs that push users away from protective choices — to undermine children's ability to exercise their rights. The UK Children's Code and EU DSA both explicitly prohibit deceptive design in services used by children.
  • Evolving autonomy: A 16-year-old has very different autonomy interests and capabilities from a 7-year-old. Designs that treat all under-18s uniformly fail this principle. Age-differentiated interfaces and controls — adapted as children grow — are required.
  • Meaningful consent: Where children's consent is sought for data processing or other actions affecting their rights, that consent must be genuinely informed, freely given, and in language they can understand. Checkbox consent that no one reads does not meet this standard.

This principle is operationalized in the UK AADC's prohibition on "nudge techniques" and the EU DSA's ban on deceptive interfaces. California's AADC mandates "age-appropriate" design explicitly based on the needs of the actual age group of likely users.

3

Transparency and Accountability

Safety claims must be verifiable — platforms must demonstrate, not just assert, that they protect children.

The era of self-certification is over. Platforms that claim to be safe for children must be able to demonstrate that claim through transparent reporting, independent audit, and regulatory oversight. This principle closes the accountability gap that allowed platforms to make impressive child safety commitments while implementing minimal actual protections.

Transparency and accountability requires:

  • Public transparency reports: Annual reports disclosing meaningful metrics — volume of CSAM reports submitted to NCMEC/IWF, content removal rates, age assurance implementation, enforcement actions, complaint resolution times — not just aggregated numbers designed to present a favorable picture.
  • Child Rights Impact Assessments: Documented assessments, conducted before product launches and major feature changes, of the anticipated impact on children's rights. UNICEF's D-CRIA Toolbox provides the methodology. EU DSA guidelines reference it explicitly.
  • Regulatory access: Regulators must have access to the platform data and internal processes needed to assess compliance. The EU DSA requires this for very large platforms. UK Ofcom has powers of investigation under the OSA.
  • Independent audit: DSA-designated very large platforms must undergo annual independent audits of their risk assessment and mitigation measures. The era of self-policing by platforms whose business model depends on engagement is ending.
  • Accessible privacy policies: Privacy policies for services used by children must include child-friendly versions — not just the legally compliant full text that no adult reads either. The UK Children's Code requires this specifically.

The OECD's Eight Implementation Components (2024)

Building on the eSafety framework, the OECD's 2024 publication Towards Digital Safety by Design for Children identifies eight specific components that a comprehensive Safety by Design implementation should include. This provides a structured framework for companies translating principles into practice:

1
Risk Assessment

Systematic identification and assessment of risks to children's rights before and during product operation

2
Age-Appropriate Design

Product design tailored to the developmental stage and capabilities of the actual age range of child users

3
Privacy-Protective Defaults

Maximum privacy as the out-of-box experience — data minimization, no profiling, geolocation off

4
Accessible User Controls

Safety features that children can find and use — prominently placed, simply worded, effectively functional

5
Safe Recommendation Systems

Content curation algorithms that actively filter harmful content categories for child users

6
Child-Appropriate Data Use

Data collected from child users used only for service provision — not profiling, advertising, or sale

7
Child-Friendly Reporting

Reporting mechanisms designed for children — simple, visible, low-friction, with meaningful follow-up

8
Transparency & Accountability

Public reporting, independent audit, regulatory access, and documented impact assessments

How Leading Platforms Are Implementing Safety by Design

Safety by Design is no longer theoretical. Regulatory pressure and reputational risk have driven major platforms to implement — with varying degrees of completeness — SbD principles. These examples illustrate both the progress made and the gaps that remain.

Meta: Teen Accounts (2024–2025)

Following years of regulatory pressure and the revelations of the Facebook Files, Meta launched "Teen Accounts" in 2024 — progressively rolled out to Instagram and Facebook globally in 2024-2025. Key features include: automatic enrollment for users under 16, with parental approval required to change most settings; default content filters removing self-harm, eating disorder, and violent content; restrictions on direct messaging (users under 16 can only be messaged by accounts they follow); screen time notifications; and nightly time limits. Teen Accounts represents genuine progress on default privacy, content restriction, and messaging controls — but independent researchers at the 5Rights Foundation and Center for Humane Technology have noted that age detection relies on self-declaration, and that the features can be bypassed by users who claim to be older than they are. Age assurance remains the unresolved core challenge.

YouTube: Supervised Experience and Age-Appropriate Content

Google's YouTube has implemented a tiered approach to age-appropriate content. YouTube Kids provides a separate, heavily curated environment for younger children. For teens on the main platform, YouTube in 2023 introduced "take a break" reminders, bedtime reminders, default autoplay off, and supervised experience settings that allow parents to link their account to a teenager's for oversight. Content recommendation algorithms for signed-in users under 18 are adjusted to reduce exposure to content involving dangerous challenges, violent events, and body image. These are meaningful implementations of Safety by Design principles — though civil society has noted ongoing concerns about how effectively age data is used to differentiate experiences, and how recommendation suppression is validated.

Discord: Server Safety, DM Restrictions, and Family Center

Discord serves a large population of young people (particularly gamers) and has faced significant scrutiny for child safety failures. Since 2022, Discord has launched Family Center (allowing parents to see which servers their children are active in and who they are messaging); default safe messaging settings for users 13-17; AI-based detection of grooming patterns; integration with Thorn's Safer platform for CSAM detection; and restrictions on friend requests from users the young person doesn't share a server with. Discord's approach is notable for its recognition that completely restricting teen access would lose the very users it needs to protect — so it focuses on contextual controls within the existing product architecture.

Important caveat: All platform self-implementations fall short of the full SbD standard in some dimensions. Independent audit, regulatory enforcement, and civil society scrutiny remain essential complements to industry voluntary action. The regulatory frameworks described below exist precisely because voluntary measures are insufficient on their own.

Phase-by-Phase Implementation Checklist

For companies beginning their Safety by Design journey, implementation is most effective when organized by product development phase — ensuring safety considerations are embedded at each stage rather than applied retrospectively.

Phase 1: Discovery and Risk Assessment

  • Determine whether children are or could foreseeably be users of your product — including whether it is primarily aimed at adults but accessible to minors
  • Identify the age range of likely child users and their developmental characteristics — what can a typical 8-year-old understand? A 14-year-old?
  • Conduct a preliminary Child Rights Impact Assessment (CRIA) using UNICEF's D-CRIA methodology
  • Map all foreseeable pathways through which your product could facilitate harm to child users: content, contact, conduct, contract, and data risks
  • Map regulatory obligations in each jurisdiction where you operate — EU DSA, UK OSA/AADC, COPPA (US), applicable state laws
  • Assess your current Safety by Design maturity against WePROTECT's four-phase Maturity Model

Phase 2: Design Interventions

  • Set maximum privacy settings as defaults for child users — geolocation off, profile not searchable, data collection limited to service provision
  • Design reporting flows in plain, age-appropriate language with visual cues — test them with actual children in the relevant age range
  • Audit all design patterns for "dark patterns" — friction that discourages privacy choices, pre-checked consent, confusing language around data sharing
  • Design content recommendation systems with explicit child-safety filters — define which categories are suppressed for child users and how suppression is validated
  • Restrict messaging from unknown adults to child accounts by default — require the child to follow the account before accepting messages
  • Design age-differentiated interfaces — the experience for a 7-year-old should be meaningfully different from the experience for a 15-year-old
  • Remove or disable engagement-maximizing features (autoplay, infinite scroll, streak notifications) for users under 16, or make them opt-in with clear disclosures
  • Design parental controls that are both functional and respect children's evolving autonomy

Phase 3: Technical Implementation

  • Integrate PhotoDNA or equivalent perceptual hash-matching for user-uploaded image/video content — available free from Microsoft
  • Integrate Thorn's Safer platform for CSAM hash-matching — covers 29M+ known hash values and is available to qualifying companies
  • Implement NCMEC CyberTipline reporting — legally required under US law (18 USC 2258A) for electronic service providers with actual knowledge of CSAM
  • Implement IWF hash lists for UK and European operations
  • Build age assurance mechanisms appropriate to the risk level of your product — self-declaration for low-risk services, age estimation or verification for high-risk ones
  • Implement COPPA-compliant consent flows for US child users, GDPR-compliant flows for EU users, ensuring age-appropriate language throughout
  • Integrate AI-based grooming detection where your product facilitates communication between users

Phase 4: Pre-Launch Validation

  • Complete and document the full Child Rights Impact Assessment
  • Test all safety features with actual child users of the relevant age range — under appropriate safeguarding conditions
  • Publish a child-friendly privacy policy alongside the full legal text
  • Establish escalation protocols for content reported by or involving minors — with clear timelines and law enforcement referral pathways
  • Ensure CSAM reporting is integrated and tested end-to-end with NCMEC/IWF systems
  • Confirm regulatory compliance status in each operating jurisdiction

Phase 5: Ongoing Operations and Improvement

  • Publish an annual transparency report covering child safety metrics — CSAM reports, content removal, age assurance implementation, enforcement actions
  • Conduct annual risk assessment updates — especially as new features are launched or user demographics change
  • Participate in industry hash-sharing consortia (NCMEC, IWF) — expanding the collective detection capability
  • Engage with Ofcom, DSA, and other regulator consultation processes on children's safety codes
  • Conduct periodic third-party audits of safety feature effectiveness
  • Review and update the CRIA after any major product change affecting child users

The Business Case for Safety by Design

Beyond regulatory compliance and moral obligation, there is a compelling business case for Safety by Design investment — one that the most sophisticated companies are increasingly recognizing.

  • Proactive is cheaper than reactive: Building safety into a product during design costs a fraction of what it costs to retrofit safety after a product has scaled. The moderation costs that platforms like Meta, YouTube, and TikTok bear — hundreds of millions annually — are largely the consequence of failing to build safety in from the start.
  • Regulatory risk reduction: The maximum penalty under the UK Online Safety Act is 10% of global annual turnover. For a large platform, a single enforcement action could cost billions. The investment in proactive SbD implementation is trivial by comparison.
  • Reputational protection: The past decade has seen multiple major platforms suffer severe reputational damage from child safety failures — Facebook/Instagram's mental health revelations, YouTube's COPPA violation fine, TikTok's regulatory investigations globally. The reputational cost of failure vastly exceeds the cost of prevention.
  • User trust and retention: Parents are significant gatekeepers of children's platform adoption — particularly for younger age groups. Platforms with demonstrably strong child safety records are more likely to receive parental approval, driving acquisition and retention of younger user cohorts.
  • Competitive differentiation: In a market where regulatory compliance is becoming table stakes, demonstrable safety leadership is emerging as a genuine competitive differentiator — particularly for platforms seeking enterprise and education market access.

Common Mistakes to Avoid

Based on regulator enforcement actions and civil society audits, these are the most common Safety by Design failures that lead to regulatory action:

  • Age as an afterthought: Designing a product for a general adult audience, then applying child safety features as an overlay after the core architecture is fixed. The architecture must be designed with child use in mind from the beginning.
  • Self-declaration as age assurance: Asking users to enter their date of birth and accepting any answer. Self-declaration is not age assurance — it is an absence of age assurance. Regulators are moving rapidly toward requiring verifiable age estimation or verification for high-risk services.
  • Safety features buried in settings: Building technically functional safety controls that are accessible only through complex settings menus that most users never visit. Prominence and accessibility matter as much as functionality.
  • One-size-fits-all for "minors": Applying identical settings and controls to all users under 18. A product that treats a 7-year-old and a 17-year-old identically fails both groups.
  • Transparency theater: Publishing impressive-sounding child safety commitments without the transparency reporting, independent audit, or regulatory access that would allow those commitments to be verified.
  • Compliance as ceiling, not floor: Treating regulatory minimum requirements as the target rather than the floor. Regulations typically lag behind the state of the art. Companies that meet only the minimum will find themselves re-exposed as requirements advance.

Remember: Safety by Design is an ongoing practice, not a one-time certification. The best companies treat child safety as a product discipline — with dedicated product managers, regular reviews, child safety red-teaming, and executive accountability — not as a compliance checkbox.

Regulatory Landscape: Where SbD Is Now Mandated

Regulation Jurisdiction Key SbD Requirements Max Penalty Status
Digital Services Act EU Risk assessments; age-appropriate design; no profiling-based ads to minors; systemic risk mitigation 6% global revenue In force Feb 2024
Online Safety Act UK 40+ children's safety code measures; highly effective age assurance; content filtering; safe design 10% global turnover Codes in force Jul 2025
Children's Code (AADC) UK 15 binding standards incl. privacy by default, no nudge, no profiling, geolocation off £17.5M or 4% global revenue In force since 2021
Age-Appropriate Design Code California, US Default high-privacy settings; age estimation; data protection impact assessments $7,500/intentional violation Key provisions upheld Mar 2026
Online Safety Act Australia Safety by Design standards set by eSafety Commissioner; proactive harm prevention AUD 49.5M (companies) In force
COPPA US (Federal) Consent for data collection from under-13s; data minimization; deletion rights $51,744/violation/day In force since 1998

Further Reading and Key Resources