What Today's AI Models Can and Cannot Do in Practice

Each new model generation arrives with the claim that the old problems are solved. Some of them genuinely are. Several are not, and the ones that are not tend to be exactly the failures that cost you something when you do not know about them.

This is a working map: what current frontier models do well, what improved, and what still breaks.

What genuinely got better

Instruction-following

The most underrated improvement. Older models would acknowledge a constraint and then ignore it three paragraphs later. Current ones hold multi-part instructions — format, length, tone, and prohibitions — much more reliably across a long response. This is why detailed prompts pay off more now than they used to.

Long context

Context windows are now large enough to hold entire books, codebases, or years of correspondence. What matters more than the size is that attention across that span improved: models are considerably better at finding and using a detail buried in the middle, which used to be a well-documented weak spot.

It is not perfect. Precision still degrades toward the middle of very long inputs, so for anything critical, put the important material near the start or the end rather than trusting a search across 300 pages.

Reasoning

Models that work through a problem before answering are meaningfully better at multi-step problems — math, logic, planning, and debugging. The gain is real and it is not universal: reasoning modes help on problems with structure and do little for tasks that were always about phrasing.

Reading images

Photographs, diagrams, screenshots, charts, and handwriting are all now practical inputs. Reading a chart and describing the trend works. Reading a dense table from a photo and getting every cell right does not.

Code

Generation, explanation, debugging, and translation between languages all improved substantially. Models are now genuinely useful on unfamiliar code, which is where most engineering time actually goes.

What still fails

Confident fabrication

The single most important limitation, and it has not been solved. Models produce plausible, well-structured, incorrect information with exactly the same confidence they produce correct information. It concentrates in specific places: citations, statistics, dates, quotations, legal and medical specifics, product details, and anything niche.

There is no tone cue. This is why verification cannot be replaced by judgment about how sure the answer sounds.

Arithmetic on long inputs

Reasoning improved; column-of-numbers arithmetic remains a place where errors slip through, and a wrong subtotal propagates silently through everything after it. Check the math.

Knowing what it does not know

Models are poorly calibrated about their own uncertainty. Asking “how confident are you?” produces an answer that sounds calibrated and largely is not.

Genuinely current information

Training data has a cutoff. A model without web access will answer questions about the last few months from a mix of stale knowledge and inference, and it will not flag that it is doing so. This is why a research tool that actually retrieves sources is a different capability, not a nicer interface.

Persisting outside the conversation

A model has no memory of you beyond what the product around it stores and supplies. Features that carry context across chats are product features, not model capabilities — worth understanding, because it tells you where the limits of “it remembers me” actually are.

Taste

Models converge on the median of what they were trained on. That makes them reliable and slightly boring. The line in a piece of writing that makes it worth reading is almost always the one a person insisted on.

The capability map, practically

Trust it with: first drafts, restructuring, explanation, summarizing material you supply, translation for comprehension, code you will review, brainstorming, and being argued with.

Trust it with verification: anything factual, anything numerical, anything cited, anything about a real person or company, anything with a date.

Do not trust it with: decisions with legal, medical, or financial consequence; claims that go out under your name unchecked; anything where being confidently wrong is expensive.

How to work with the failures rather than around them

Why more than one model helps

Different models can surface different assumptions or approaches. A disagreement is a reason to investigate further, while agreement still does not prove a factual claim.

ChatUp’s current catalog can include model choices from OpenAI, Anthropic, Google, xAI, and DeepSeek. Use another available model for critique, then verify facts with reliable sources. AI Research is the separate workflow for retrieving current sources.

Frequently asked questions

Are hallucinations solved?

No. They are less frequent and just as confident. Verification remains mandatory for anything factual.

Does a bigger context window mean it reads everything perfectly?

No. It can hold more, and precision still degrades in the middle of very long inputs. Put critical material at the edges.

Is a reasoning model always better?

For structured problems, usually. For writing, conversation, and phrasing, often not — and it is slower. Match the mode to the task.

How do I know which model to use?

Try the same prompt on two. The differences are real, partly a matter of taste, and take about a minute to discover for your own work.

Know the shape of the failures

Capability claims move fast; the failure modes are stable. Fabrication, arithmetic, calibration, and currency have been the same four problems for years. Knowing where they live is more useful than knowing which model topped which benchmark this quarter.