Distributed predictions at scale