Research
The Kenyan AI business-advice experiment argues for small reversible decisions
Treat generated advice as a hypothesis to evaluate, especially when it changes spending or daily operations.
Zocri · 2 min read · Sources checked
An average result can conceal different experiences
The Kenyan field experiment reported by UC Berkeley Haas did not establish a statistically significant average revenue or profit effect and found heterogeneous outcomes. That finding does not classify a new owner as a likely winner or loser. It does challenge the idea that access to a business-advice assistant reliably improves performance by itself.
For an owner, the useful response is to make advice testable. A suggestion to change opening hours, discount stock or add a service should be examined against the business's actual customers, costs and constraints. A confident explanation is not evidence that the action fits this particular situation.
Convert the advice into a hypothesis
Write what should change, why it might help and what observation would support the idea. If the suggestion is to follow up inactive customers, define the eligible group and the response that would matter. Use the business's permitted communication practices and avoid assuming that every contact should receive a new message.
Keep a do-nothing comparison in mind. Some proposals sound useful because they describe activity, even when the activity may not improve the outcome the owner cares about.
Start where reversal is affordable
Choose a bounded trial with limited cost and a clear ending. Avoid committing scarce stock, substantial spending or a permanent operating change before the basic assumption has been checked. Where the advice concerns a specialized or consequential decision, consult someone qualified in that area.
Record the actual change made. A trial cannot be interpreted clearly if several unrelated decisions shift at the same time.
Review results without forcing a success story
Compare outcomes with the starting expectation and note alternative explanations. A busy week may reflect an external event rather than the new action. A quiet week may be too little evidence to reject a promising idea. Keep the conclusion proportionate to the volume and quality of observations.
Also count the time spent implementing the advice, not only the cost of asking for it.
Keep the owner’s judgment visible
Retain a short decision record: suggestion, assumptions, action, result and next step. This makes it easier to learn across trials and to explain why an attractive recommendation was rejected.
Our framework is an interpretation of the experiment's uneven results, not a promise of improved business performance. AI can supply ideas to examine. The owner still needs a method for choosing, testing and stopping actions in the circumstances the business actually faces.