Subscribe to our newsletter!

Submit

Thank you for signing up for our newsletter!

We’re excited to have you with us and will keep you updated with the latest news, insights, and updates straight to your inbox.
Oops! Something went wrong while submitting the form.
Back to Blog
November 10, 2025
|
Read time {time} min

Online Safety and the Limits of AI Moderation: What Parents Can Learn from Roblox

Written by
Permission
Stay in the loop

Get the latest insights, product updates, and news from Permission — shaping the future of user-owned data and AI innovation.

Subscribe

Roblox isn’t just a game — it’s a digital playground with tens of millions of daily users, most of them children between 9 and 15 years old.

For many, it’s the first place they build, chat, and explore online. But as with every major platform serving young audiences, keeping that experience safe is a monumental challenge.

Recent lawsuits and law-enforcement reports highlight how complex that challenge has become. Roblox reported more than 13,000 cases of sextortion and child exploitation in 2023 alone — a staggering figure that reflects not negligence, but the sheer scale of what all digital ecosystems now face.

The Industry’s Safety Challenge

Most parents assume Roblox and similar platforms are constantly monitored. In reality, the scale is overwhelming: millions of messages, interactions, and virtual spaces every hour. Even the most advanced AI moderation systems can miss the subtleties of manipulation and coded communication that predators use.

Roblox has publicly committed to safety and continues to invest heavily in AI moderation and human review — efforts that deserve recognition. Yet as independent researcher Ben Simon (“Ruben Sim”) and others have noted, moderation at this scale is an arms race that demands new tools and deeper collaboration across the industry.

By comparison, TikTok employs more than 40,000 human moderators — over ten times Roblox’s reported staff — despite having roughly three times the daily active users. The contrast underscores a reality no platform escapes: AI moderation is essential, but insufficient on its own.

When Games Become Gateways

Children as young as six have encountered inappropriate content, virtual strip clubs, or predatory advances within user-generated spaces. What often begins as a friendly in-game chat can shift into private messages, promises of Robux (Roblox’s digital currency), or requests for photos and money.

And exploitation isn’t always sexual. Many predators use financial manipulation, convincing kids to share account credentials or make in-game purchases on their behalf.

For parents, Roblox’s family-friendly design can create a false sense of security. The lesson is not that Roblox is unsafe, but that no single moderation system can substitute for parental awareness and dialogue.

Even when interactions seem harmless, kids can give away more than they realize.

A name, a birthday, or a photo might seem trivial, but in the wrong hands it can open the door to identity theft.

The Hidden Threat: Child Identity Theft

Indeed, a lesser-known but equally serious risk is identity theft.

When children overshare personal details — their full name, birthdate, school, address, or even family information — online or with strangers, that data can be used to impersonate them.

Because minors rarely have active financial records, child identity theft often goes undetected for years, sometimes until they apply for a driver’s license, a student loan, or their first job. By then, the damage can be profound: financial loss, credit score damage, and emotional stress. Restoring a stolen identity can require years of effort, documentation, and legal action.

The best defense is prevention.

Teach children early why their personal information should never be shared publicly or in private chats — and remind them that real friends never need to know everything about you to play together online.

AI Moderation Needs Human Partnership

AI moderation remains reactive.

Algorithms flag suspicious language, but they can’t interpret tone, hesitation, or the subtle erosion of boundaries that signals grooming.

Predators evolve faster than filters, which means the answer isn’t more AI for the platform, but smarter AI for the family.

The Limits of Centralized AI

‍The truth is, today’s moderation AI isn’t really designed to protect people; it’s designed to protect platforms. Its job is to reduce liability, flag content, and preserve brand safety at scale. But in doing so, it often treats users as data points, not individuals.

This is the paradox of centralized AI safety: the bigger it gets, the less it understands.

It can process millions of messages a second, but not the intent behind them. It can delete an account in a millisecond, but can’t tell whether it’s protecting a child or punishing a joke.

That’s why the future of safety can’t live inside one corporate algorithm. It has to live with the individual — in personal AI agents that see context, respect consent, and act in the user’s best interest. Instead of a single moderation brain governing millions, every family deserves an AI partner that watches with understanding, not suspicion.

A system that exists to protect them, not the platform.

The Future of Child Safety: Collaboration, Not Competition

The Roblox story underscores an industry-wide truth: safety can’t be one-size-fits-all.
Every child’s online experience is different and protecting it requires both platform vigilance and parent empowerment.

At Permission, we believe the next generation of online safety will come from collaboration, not competition. Instead of replacing platform systems, our personal AI agents complement them — giving parents visibility and peace of mind while supporting the broader ecosystem of trust that companies like Roblox are working to build.

From one-size-fits-all moderation to one-AI-per-family insight — in harmony with the platforms kids already love.

Each family’s AI guardian can learn their child’s unique patterns, highlight potential risks across apps, and summarize activity in clear reports that parents control. That’s what we mean by ethical visibility — insight without invasion.

You can explore this philosophy further in our upcoming piece:
➡️ Monitoring Without Spying: How to Build Digital Trust With Your Child (link coming soon)

What Parents Can Do Now

Until personalized AI guardians are widespread, families can take practical steps today:

  • Talk early and often. Make online safety part of everyday conversation.

  • Ask, don’t accuse. Curiosity builds trust; interrogation breeds secrecy.

  • Play together. Experience games and chat environments firsthand.

  • Set boundaries collaboratively. Agree on rules, timing, and social norms.

  • Teach red flags. Encourage your child to tell you when something feels wrong — without fear of punishment.

A Shared Responsibility

The recent Roblox lawsuits remind all of us just how complicated parenting in the digital world can feel. It’s not just about rules or apps: it’s about guiding your kids through a space that changes faster than any of us could have imagined! 

And the truth is, everyone involved wants the same thing: a digital world where kids can explore safely, confidently, and with the freedom to just be kids.

At Permission, we’re committed to building an AI that understands what matters, respects your family’s values and boundaries, and puts consent at the center of every interaction.

Recent articles

Your ASK Wallet, Now Powered by Coinbase

Sep 22nd, 2026
|
{time} read time

We've partnered with Coinbase to bring best-in-class wallet technology to our community.

Today we're updating the technology that powers your ASK wallet. Here's what's changing, what it means for you, and what stays exactly the same.

What's changing

We've moved your Permission wallet to Coinbase's platform. What that means in plain language is this: starting today, you are in full control of your ASK. When you click to send, redeem, or manage your balance, the transaction happens from your account, signed by your login. We no longer hold your wallet keys. Coinbase does, on your behalf, under your authority.

We chose Coinbase's embedded wallet specifically because it brings institutional-grade security to our users without requiring you to manage anything yourself. The infrastructure is Coinbase's. The wallet is yours.

Beyond the custody change, our users now have a wallet that can go wherever they go: exportable, cross-chain ready, and backed by a platform that serves millions of people around the world. We're proud to bring that to our community.

What it means to control your own keys

As Permission has grown, we felt strongly that your funds should be held by a platform built specifically for that purpose, with the security standards and regulatory rigor that come with it. Moving to Coinbase's embedded wallet reflects that commitment.

With today's change, Coinbase secures your private key inside their systems, and only your Permission login can authorize transactions. When you click to send or redeem ASK, the transaction is authorized by you, through your login. We are no longer part of that process.

What this means practically: your wallet operates on its own, independent of Permission. Treat your Permission login like you would a bank password. It is now the key to your wallet. If you ever want to take your wallet entirely outside of Permission, Coinbase supports key export and that option is yours.

What Coinbase sees

Because Coinbase is now part of the infrastructure, your email address, account identifier, and wallet information are shared with them for the purpose of operating the wallet. For details on how Coinbase handles this data, you can review their embedded wallet documentation and their privacy information.

For how Permission handles your data, our Terms of Use and our Privacy Policy govern that relationship, as they always have.

What stays the same

Everything you experience in the app. Earning ASK, redeeming it, transferring it, viewing your balance, managing your family. None of that changes.

And, what does change, we're excited about: key export, cross-chain support, and institutional-grade security. These are capabilities that would have taken years to build in-house and that Coinbase has spent that time perfecting. We chose to partner with the best-in-class, and our product and users will be better for it.

If you have questions, support is always here.

The Permission Team 🤝

What Is Family Friendly AI™?

Sep 9th, 2026
|
{time} read time

Only 15% of people globally say they trust AI systems, and 72% of parents are concerned about AI’s impact on their children.

AI is quickly becoming part of everyday family life, but Big Tech wasn’t built with families in mind. Family Friendly AI is technology intentionally designed for families, giving parents visibility into their children’s digital lives, guidance when they need it, and tools to encourage positive behavior.

5 Things That Make AI Family Friendly

1. Your family owns its data.

‍Your family’s data is never sold. It belongs to your family, and you stay in control of it.

2. Parents know what’s happening online.

‍Family Friendly AI gives parents visibility into their children’s digital lives, helping them fully understand how their children use and interact with technology.

3. It motivates children with rewards and incentives.

‍Parents can set rewards and incentives to encourage positive behaviors and help their children build better habits around technology and beyond.

4. It gives parents coaching and inspiration.

‍Parenting in a digital world comes with challenges that screen-time limits alone can’t help with. Family Friendly AI gives parents personalized AI-insights and guidance to help them navigate what their children are doing online and decide what to do next.

5. It earns families' trust.

‍Technology for families should have a higher bar. The companies building it should stand behind it with an unconditional, no-questions-asked money-back guarantee.

It’s time for AI, crypto, and the technology shaping our children’s lives to meet the family-friendly standard.

Big Tobacco Had Its Reckoning. Now It’s Big Tech’s Turn.

Aug 12th, 2026
|
{time} read time

The floodgates are open.

Thousands of lawsuits are moving forward. States are writing new rules for kids online. And lawmakers are beginning to tell AI companies what they can and cannot do when children use their products.

And you don't have to look far to see it happening...

The courts: 3,000+ lawsuits get the green light

On August 10, the Ninth Circuit allowed more than 3,000 lawsuits against Meta, Google/YouTube, TikTok and Snap to move forward.

The cases allege that the companies deliberately designed features of their platforms to be addictive, particularly for young users.

The tech companies had argued that Section 230 of the Communications Decency Act protected them from the claims. The court rejected their attempt to use Section 230 to stop the litigation at this stage, finding that it provides a defense rather than immunity from being sued.

At their core, these cases are allegations about the platforms themselves: how they were designed, how they kept people engaged, and what responsibility the companies bear for the consequences.

The companies will still have the opportunity to defend themselves against those allegations.

But with more than 3,000 cases now getting the chance to be heard, this is getting harder to argue away.

New Jersey: families get a way to enforce the rules

One day later, New Jersey Governor Mikie Sherrill signed the New Jersey Kids Code Act into law.

The law establishes new design and privacy requirements for covered online services likely to be accessed by minors. Among other provisions, it requires high privacy settings by default, restricts certain push notifications, prohibits dark patterns for minors, limits how children's personal data can be used and retained, and places restrictions on targeted advertising.

But one provision in particular changes the accountability equation: a private right of action.

An individual under 18 who is injured by a violation can bring a claim under the law, with statutory damages of $5,000 per violation. Parents may also bring an action on a minor's behalf.

Which is legal language for something pretty simple: families don't have to wait around for a regulator to act. They can take companies to court themselves.

Colorado: AI safety starts becoming a legal requirement

Then there's Colorado.

Earlier this year, Governor Jared Polis signed Colorado HB 26-1263, establishing specific requirements for operators of conversational AI services.

And this one is worth paying attention to because the law doesn't simply tell AI companies to "keep kids safe." It starts defining what that actually means.

Operators must estimate users' ages. When dealing with minors, the law requires recurring disclosures that they are interacting with AI rather than a person and establishes protections around sexually explicit interactions.

It also addresses one of the most unsettling questions surrounding companion-style AI: emotional dependence.

The law requires safeguards designed to prevent conversational AI from producing statements that simulate emotional dependence. It also requires protocols for responding to suicidal ideation and self-harm, privacy and account-management tools for minors and parents or guardians, and reporting requirements intended to help regulators evaluate whether those safeguards are actually working.

The law takes effect January 1, 2027.

For companies building conversational AI, that's a meaningful shift. Child safety is moving beyond a set of voluntary guardrails companies write for themselves. In Colorado, some of those guardrails are becoming law.

It's no coincidence that this is all happening at once.

Big Tobacco didn't wake up one morning and discover the world had changed its mind. The reckoning came piece by piece, until lawsuits became regulation and an industry that had spent decades setting its own standards was finally forced to take responsibility for the harm its products caused.

We're watching that shift happen again.

For years, the responsibility for keeping kids safe online has fallen on parents.

Set the parental controls. Check the privacy settings. Watch the screen time. Know which apps they're using. Figure out who they're talking to. Keep up with every new platform, algorithm and now AI chatbot entering their lives.

All while the technology on the other side of the screen gets more sophisticated by the month.

Now courts and lawmakers are starting to ask the companies building that technology a much more uncomfortable question:

If children are using your products, what are you doing to keep them safe?

For families, that's the shift that matters most.

This isn't another round of false promises to "do better."

This is legislation. These are lawsuits. This is accountability beginning to have teeth.

Parents will always have the role of protecting their children online. We happen to believe they should have far more visibility and control over the technology entering their families' lives, not less.

But parents cannot be the entire safety system.

The law is making clear that the companies designing the products, writing the algorithms and building the AI our kids interact with have a responsibility, too.

And when they fail to meet it, they'll finally be held accountable.

ChatGPTs Births A Parenting Tool That Needs Some Image Repair

Aug 4th, 2026
|
{time} read time

Sam Altman keeps pitching AI as a co-parent. The reason parents aren't buying isn't nostalgia, it's the lawsuits.

Last Friday, Sam Altman had an idea he was excited about. Hook your family calendar up to ChatGPT, tell it what your kids are into, and every morning on the drive to school it'll produce a little podcast: one kid's soccer game that afternoon, another kid's birthday coming up, maybe some news. He called it a "cool use case."

What should’ve felt really innovative, landed like the opening scene of a bleak dystopian movie. Two kids in the back, one parent up front, and a smooth synthetic voice narrating, to everyone present, the lives of everyone present. "Later today, Maya has soccer." Maya, who has soccer, looks out the window. Nobody says anything, because the podcast is saying it for them.

The internet population caught what we caught. The reply that stuck came from Alex Hirsch, creator of Disney’s Animated series, Gravity Falls. It was seven poignant words: "What if you just talked to your children?" That was the entire rebuttal, and it traveled a great deal further than the thing it was rebutting. Altman's post drew somewhere around 9,600 likes. Hirsch's reply cleared 120,000. On the CEO's own platform, the crowd took a vote, and the crowd chose the small talk.

Now, we want to be fair here, because the easy thing is to dunk and move on. But we’re parents here at Permission and anyone who has done the 7:40 a.m. drive on four hours of sleep, refereeing a backseat dispute about who touched whom first, knows the exact fantasy of a button that handles the morning. That instinct isn't a character flaw. It's a Tuesday.

But this wasn't a one-off. Altman has been quietly auditioning AI for the co-parent role for a while now. On The Tonight Show in December 2025 he said he couldn't imagine having to "raise a newborn without ChatGPT" then added that people had managed the trick for a few hundred thousand years without it. Also, last year, in a podcast hosted by Andrew Mayne, Altman admitted that people might form “problematic parasocial relationships” to a chatbot. (You know, the one-sided kind that we usually reserve for celebrities we've never met.) He sees the hazards clearly. He's pitching the product anyway.

When visibility turns into vulnerability.

The reason parents flinched at the idea of carpooling with a chatbot for school drop off isn't that they're allergic to convenience. It's that the company making the offer is, right now, being sued by multiple families who say its chatbot played a role in their loved ones' spiraling delusions and, in the worst cases, their deaths. OpenAI says it is continually improving how its models handle sensitive conversations, and that work genuinely matters. But you can see the problem. "Let me into your calendar, your commute, and your kids personal details" is a big ask from anyone. It is a much bigger ask from a company currently explaining itself in court.

Trust isn't a feature you ship in the next update. It's something people hand you slowly, and take back all at once.

Here's where we should admit an interest. We build Permission on a belief that sounds boring until you sit with it: your data belongs to you. With Permission your kids’ browsing history doesn’t get shipped out to the open internet. Not to a model, not to a growth chart, not to whoever posts the next cool use case. And the closer AI creeps toward our kids (and it is creeping, because kids are already asking it everything) the more one question starts to outrank all the others:

Where is the line between parenting and outsourcing parenting?

Because "parenting tool" is doing a lot of quiet work in that phrase. A tool is a hammer. It lives in a drawer, it does one honest thing, and it does not ask to read your child's messages or move into the family calendar. When a company calls its chatbot a "parenting tool," it's worth asking, gently, which word they mean. The tool part, or the parenting part.

We happen to think AI can be genuinely, unglamorously useful to families. Not by doing the talking for you, but by handing you the context you'd otherwise miss instead of a thousand panicked notifications, and then getting out of the way so you can make the call. That's a real distinction, and it deserves its own piece.

So take this as Part One: the news, the flinch, and the reason the flinch is earned. In Part Two, we'll make the harder and more hopeful argument that you can let AI help you parent without completely handing over your family secrets. There is a version of this where the grown-ups stay in charge. We think it's the only version worth building.

For now, the seven best words anyone has offered on the whole affair still belong to Hirsch. So we'll give him the last one, too.

What if you just talked to your children?

‍