Lab demos say assistants help. Field data from customer support is clearer about how much, and for whom. Brynjolfsson, Li, and Raymond studied a generative AI conversational assistant rolled out to agents at scale and measured issues resolved per hour—not chat volume alone.
What the study measured
The NBER working paper Generative AI at Work (w31161) tracks the staggered introduction of a GPT-based assistant that suggests replies during live support chats. Agents could accept or ignore suggestions. Productivity is defined as issues successfully resolved per hour.
Across 5,179 agents, access to the tool raised that rate by about 14% on average. The gain came from shorter handle time, more chats handled per hour, and a small improvement in resolution—not from dumping unfinished conversations.
Who benefits most
Effects are uneven. Novice and lower-skilled workers saw roughly a 34% improvement. Experienced and highly skilled agents saw little change. The paper’s interpretation is that the model encodes practices of stronger agents and shortens the learning curve for newer ones.
That pattern matters for rollout design. A blanket “everyone gets the same uplift” expectation will misread the data. The tool is closer to a training and consistency layer than a multiplier for your best people.
Quality and secondary effects
The authors also report improvements in customer sentiment and employee retention, with suggestive evidence of worker learning. Productivity did not require trading away the interaction quality measures they track.
For operators, the practical lesson is to instrument resolution rate, handle time, and quality together. A speed-only dashboard can hide regressions in escalation or rework.
What this does not claim
This is one large software firm’s support organization and one assistant design. It is not a general proof that every chatbot will raise throughput 14%. Context, suggestion quality, and agent incentives all matter.
It is, however, one of the clearest public field estimates that generative assistance can move a real operations KPI when it sits inside an existing human workflow rather than replacing it.