Just a few weeks ago, a court ruled in Amazon’s favor after they blocked Perplexity’s agentic browser from operating on their platform. The detail that matters: the user had explicitly authorized Comet to make purchases on their behalf. That permission is the heart of the issue. The court’s finding was that user consent doesn’t override platform terms of service when an AI agent is acting on your behalf.
Some Deeper Context: Perplexity launched an AI-powered browser named “Comet” that allowed users to input their Amazon credentials and have the AI act on their behalf. The AI acted as a proxy to search, browse, and make purchases. Amazon alleged Perplexity disguised Comet to look like a standard Chrome browser session and accessed password-protected parts of the website without platform approval. A federal court issued a preliminary injunction siding with Amazon, blocking Perplexity's agents from accessing Amazon's systems and ordering the destruction of scraped data. (This is a preliminary injunction, not a final ruling yet)
Most people read that story as a tech industry turf war. I read it as the first real signal of where AI governance is heading, and why the teams building agentic products right now have a structural problem they probably haven’t named yet.
The Gap Not Being Talked About Enough
When you authorize an AI agent to act on your behalf, you’re primarily giving it two things: your intent (what you want done) and your permission (you said yes). Implicitly you are even giving it some level of discretion and trust. What you’re almost never giving it is your full constraint layer.
That constraint layer is what actually governs whether an action is appropriate. It includes platform terms of service, your company’s code of conduct, your client’s contractual limits, your industry’s regulatory requirements, to some extent your values, and your own personal authorization thresholds for what the agent can do without asking first.
The gap between intent and constraints is where exposure lives. A well-intentioned user can give an agent full permission, with clear goals, and still get burned, because the agent had no model of the constraints, only the goal.
That’s the problem the Amazon ruling made visible. It just happened to surface in a commerce context first.
What agents are built to do vs. what enterprise actually needs
Most agentic tools are oriented toward completion. Give them a task, they find a path, they execute. That’s genuinely useful. But it’s not the same as trustworthy. The past few months have been centered on making these agents more capable, giving them more access and focusing on improving their task completion.
Today, these platforms might show the user a quick permission prompt to take an action for the user. It would show a simple “ Okay can I proceed?” And the user is able to click yes. If the user clicks “Yes” or “proceed” the AI takes full confidence that the user has thought through the risks.
Except in most cases I would argue they haven’t. Because not all users will know what they don’t know. These systems have been built by world class engineers and data scientists. These individuals are world class technical talent and they know what the prompts are asking. Will every non-technical user?
In my experience, I have had the privilege of working with a lot of top technical talent. Usually the most useful person in any technical conversation isn’t the one who says yes the fastest. It’s the one who says: “We can do that, but here’s what I’d think about first.” That’s what a good CTO does. A good architect. A good tech lead who’s seen how this kind of thing has gone wrong before. They hold your intent in one hand and the full picture of constraints in the other, and they tell you when those two things are in conflict before you’ve committed.
Agents haven’t met that standard yet. And for most enterprise buyers right now, sitting in the pilot phase and still working out their AI governance posture, that gap is the thing that slows or stops adoption.
The market read
The tools that win in enterprise won’t necessarily be the most capable. They’ll be the ones that risk and compliance teams can say yes to. Benchmarks on fastest, capable, cost, etc., are just the starting point now. Trust benchmarks as a category have not yet taken off the same way. It isn’t a standard, yet. It isn’t on a product spec sheet but I believe it will be going forward.
I don’t have a clean answer for how this gets solved, or who owns it: platform, enterprise, or something else. I’m not sure anyone does yet. What I’m watching is whether any of the teams building agentic products right now start treating constraint-awareness as a first-class design problem, rather than something to revisit after scale.
The differentiator won’t be “look what the agent can do.”
It’ll be “look what the agent refuses to do, and why.” The claims that AI is good enough now, we should get out of the way, are misguided. Human-in-the-loop, judgement and taste will continue to be required. Not because they arent capable, but because they need our constraints.
If you’re building or buying in this space: Is constraint-awareness something you’re actively designing for, or is it still on the roadmap?
And if you’re a user: What would actually make you feel protected when you hand an AI agent access to your accounts, your calendar, your email? What would trust look like? Can we trust these systems to make us aware when our judgement is needed?
Genuinely curious what people are seeing and feeling on both sides of this.
Capability is the baseline. Judgment help is the differentiator.
