Current developments in AI are hailed as magic tools that can help domains process large amounts of data. Increasingly, humanities researchers are investigating the applicability of AI models to their use cases, with mixed results. Problems arise when the questions asked of models are in the realm of long tail, concept drift, multi-linguality and data quality. In this talk, I will explain why the expertise of humanities and social science scholars is vital to the improvement of AI tools and why we should be (more) involved in the development of these tools.