Hackathon: AI Language Models in Bible Translation

In partnership with SIL International and Microsoft, tools.bible is launching an official Challenge to explore the innovative use of AI Language Models in Bible Translation. This Challenge will occur as part of the Microsoft Global Hackathon, Sept 11-15th, and will be fully remote.

For one week, invited members of the SIL and tools.bible communities will have the opportunity to work directly with Microsoft employees on projects exploring how AI Language Models, particularly OpenAI’s GPT-4 model, might enhance quality and accelerate the pace of Bible translation.

Below are two main projects Hackathon participants will be working on, but these may shift, change, or expand before or during the event. The tasks contained within these projects range in experience level from basic prompt engineering to more advanced tasks such as building AI agents and model evaluation.

If you are interested in joining, please fill out the application at the bottom of this page.

Deadline to apply: August 31st, 2023

Project 1: AI Question Answering for Quality Checking w/ AQuA and Scripture Forge

Producing high quality Bible translations is critically important in the mission to bring God’s word to every tribe, tongue, and nation.

Bible translators and quality consultants spend long hours carefully scrutinizing every verse as drafts undergo the rigorous quality checking process. Then the draft goes through “community checking” during which local communities contribute significant hours answering questions based on translation drafts to ensure important details have been translated effectively.

This process is one of the most time-consuming tasks in Bible translation and the number of translation consultants available for the work is extremely limited. While nothing can replace the Spirit-led discernment, prayer, and fellowship that quality checking requires, AI technology can serve and aid those teams in a variety of ways.

SIL International’s AQuA (Augmented Quality Assessment) and Scripture Forge tools serve as quality checking co-pilots designed to use AI to help translators, consultants, and communities in their work.

For example, Scripture Forge provides an interface in which translation teams can import Questions & Answers for community checkers to answer. Current open source Question & Answer datasets often do not contain questions for every single verse. Ideally, there should be questions not only for every verse, but multiple questions per verse. AI language models might help generate and evaluate this data ensuring community checking is effective and comprehensive.

Additionally, AQuA provides a handful of AI-driven assessments to score and visualize specific areas of scripture that may require extra or early review. If AI can improve the quality of new drafts before they begin the rigorous quality checking process, or identify potential revisions earlier in that quality checking process, overall efficiency could significantly improve. The AQuA team is currently exploring whether an AI-driven Question Answering model could be developed that would be useful in this way to translators and consultants.

Hackathon tasks:

  • Evaluate whether Question Answering models, including ChatGPT and other LLMs, could be used to create a new AQuA assessment, and which might be most useful.
  • Generate Q&A datasets by verse (or pericopes or even entire sections) that should be answerable given a new draft or its back translation.
  • Evaluate the effectiveness of the Q&A datasets for the given purpose.
  • Question-Answer pairs generated for SF community checking may be qualitatively different from those most useful for AQuA’s QA assessment.

Project 2: AI Agents & Translation for Low Resource Languages

Translating the Bible into low-resource languages (languages without much if any digital text data), is a challenging but crucial task. As powerful as AI language models have become, they still are predominantly trained on majority languages, limiting their applicability in these situations.

However, there may be creative ways to open up new possibilities for AI language models to contribute to this translation work. Hackathon participants will brainstorm, prototype, and test some of these approaches.

For example, hackathon participants could try simulating a Bible translation team who begins with minimal data, but different roles (translator, back-translator, linguist, consultant, etc.). Each AI agent is given a specific task, with prompts and data retrieval tailored for that task.

Alternatively, hackathon participants could explore using a swarm-based or hive-mind approach, where many AI bots are deployed on small, highly specific tasks across an entire translation, iteratively improving the results.

Furthermore, even the least resourced languages may benefit from new quality checking processes powered by AI insights, and always in reference to the Greek and Hebrew source texts.

Hackathon participants will have access to extensive linguistic data in various structured and unstructured formats, covering both the source texts (Greek and Hebrew) and target languages (i.e., partially complete translation projects courtesy of the eBible corpus).

Suggested approaches might include:

  • Creating one or more AI agents in an iterative simulation
  • Leveraging more conventional probabilistic techniques such as training statistical alignment models.
  • Solving out-of-vocabulary problems when an LLM is fed data from a new language that was not in the training data
  • Fine-tuning a sequence-to-sequence translation LLM on new data