Jev is one of the most exciting things that’s happened in AI, full stop. It’s so incredibly fast, cheap, and smart, and it’s so well-designed to work with code that it has boiled my curiosity into a new state of matter.

I think the reason is that it lowers the barrier for trying new ideas. Snap your fingers and it’s done - no need to open your wallet.

And, I’m very often frustrated by how slow Claude Code or Codex are. Waiting 10, 20, 30 minutes for a task to finish, especially when you look back and Codex has been doing browser QA on something with a user count of 1 … it’s rough.

Now that I’ve been listening to Diego’s talks the past few days, I realize how much of my frustration is truly from models trying to please & infer & be aligned with my alleged preferences.

Three absolutely sick things I’ve done with Jev so far:

User gut checks

Oftentimes you need intelligence to get some kind of intuition at scale.

Question for you: how familiar are you with REI? How about Canvas? Comcast? Supercell?

I recently ran into this question. More broadly, how familiar is the average American with any given company? In an ideal world you could snap your fingers and assemble a panel of users and get their opinions.

So that’s what I did … kind of. With 100,000+ Jev calls.

To do it, I borrowed 25 personas from Tencent’s 1 Billion Personas repo that represented our target audience. I enriched the personas with ages. Then, for each one, I asked Jev from 1 to 5 how familiar this person would likely be with each company in a long list of companies. Scores ranged from 1.37 (Canvas) to 4.76 (McDonald’s).

Of course once you can measure something, you can optimize for it. So I set Codex the task of assembling 500 companies with a 4.0 score or above. (It was actually very annoyed, since the task is quite difficult - could you name 500 companies the average American will be comfortably familiar with?)

What’s cool about Jev here is not just that it was allegedly trained to be 100% reliable (whatever that means), but also that you can keep its context window extremely pure because you can send one judgment at a time. It’s unbelievably easy to pollute traditional LLMs’ context window in a way that impacts their judgments. Just by batching different kinds of judgments together, you can increase error rates.

Building a writing eval

Jev was 50x faster and 30x cheaper than GPT 5.6 Terra at running an evaluation.

In fact, I only had the idea after Jev came out. Let’s say you’re trying get AI to write argumentative essays, holding them to a very high quality bar. For any given essay, it’s actually quite challenging to write a faithful eval that LLMs can execute. It’s also much easier to define wrong ways of doing something compared with right ways.

So, you want to write an eval, but it’s effortful. You could spend that effort improving the essay-writing instead, since your domain expertise is likely a better judge than whatever eval you design anyway.

But if you’re getting it for like sub-cent and sub-second costs, why not?

For our system, I built a new eval for a part of it, then ran it with both Jev & GPT 5.6 Terra on medium thinking. You can pick some kind of metric as a proxy for your own judgements of quality, such as your own quick & dirty 1-10 ratings, or time (assuming recent outputs are better than older ones).

This is where it gets interesting. Jev displayed considerably better holistic quality sense, where GPT best measured surface-level features.

Let’s go back to the essay writing example. Imagine two teachers who each give the following feedback:

• Jev: Good job, your essay clearly communicates the main idea. (holistic quality)
• GPT: Good job, you didn’t use the word “load-bearing” anywhere in here. (surface-level features)

Jev tends to be more accurate at giving the former type of feedback. It can map surface-level features as well, but if I had to pick one of these types of feedback to optimize a learning system around, I’d pick Jev.

Jev’s probability also contains richer information compared with GPT. GPT generally gives high confidence; its scores have lower variability. Jev uses the full range from 50-100, with much broader variability. Whereas GPT’s scores and judgments are random-feeling, in my experience, Jev’s tend to correspond well with the underlying material. It makes sense, since if I’m 80% confident that a certain essay thoroughly explains the main idea, then it’s probably better than one where I’m only 60% confident it does that.

Scoring research findings

I’m currently exploring how people are using AI in different areas of work. Take software engineering as an example. This is a very noisy task, since online research is plagued by vapid, overly SEO’d content designed to get clicks and views. Not to use a Claudism, but separating the signal from the noise is very challenging.

What if, instead of trying to identify the best diamond in the rough sources, we could take every potential use case and classify it a million different ways, then cluster them to derive an overall picture of individual fields and the buckets of tasks where people in that field are finding AI useful?

Okay, maybe not a million different ways. But, how about hundreds?

This method is cool because usually less is more, since you want high quality data points, but in this case more is more, because you can build up a massive dataset where similar, low signal use cases will just cluster together without distracting from high quality ones, and you can measure so many dimensions so quickly that you also define a vast negative space of what “no one is talking about” (again, I cringe at the Claudism…).

So, I’m currently running deep research on this topic, and for each potential use case, scoring it on hundreds of dimensions to determine how AI is being used and what kinds of problems it solves, as well as how reliable the source is.

I’ll leave you with one thing

I hope these use cases have been useful and inspiring to you. I cannot stop thinking about Jev and all the cool things I want to do with it, so I hope that excitement is infectious!

Plus, Jev is fun to say. I just can’t get this one thing out of my head… do you see it too?

Big reveal: it’s Jeb Bush’s 2016 campaign branding, Jeb! Now you’ll never unsee it!