We develop a general framework for aligning large language models (LLMs) with business and economic objectives using available data. Our approach adapts an LLM to enhance the quality of decisions that can be expressed in terms of text, e.g. for persuasive messaging, while overcoming safety issues that have hindered practical deployment of LLMs. The core idea is to fine-tune an LLM on data for which credible causal inferences can be made. Using results from past A/B tests, we train an LLM to convert worse-performing decisions into better-performing decisions. When deployed, the LLM generates improvements to decisions proposed by a human expert. This design makes it unlikely for AI assistance to harm performance, which mitigates risks and eases adoption. We confirm in a field experiment that our framework performs as intended, with LLM-assisted decisions not underperforming and usually outperforming a human expert. In 36 email marketing campaigns covering 283 million total impressions, we find that subject lines created with assistance from our tuned model attain click-through rates 33% higher than a human expert baseline. While the email marketing setting is simple, its precise measurement of treatment effects ensures that observed performance improvements are attributable to genuine improvements in decision quality, thus validating the effectiveness of our framework. Additionally, we show that our LLM-generated decisions are coherent, factually accurate, and robust in new contexts, which is a major contribution addressing the primary challenges to practical deployment of LLMs. Our approach is applicable to generic objectives beyond persuasive messaging and to other modalities like images, and its modest operational costs make it suitable for a broad range of scenarios. Overall, our results illustrate that significant gains are achievable by integrating AI technologies, historical data, and human expertise in decision-making processes and provide practical guidance for realizing these benefits.