Fork me on GitHub

Alex Rupérez

Published: 29 Jul 09:01
I starred huggingface/swift-coreml-transformers at GitHub.
Swift Core ML 3 implementations of GPT-2 and BERT for Question answering. Other Transformers coming soon!
View on GitHub


Swift Core ML implementations of Transformers: GPT-2, BERT, more coming soon!

This repository contains:

  • For BERT:
    • a pretrained Google BERT model fine-tuned for Question answering on the SQuAD dataset.
    • Swift implementations of the BERT tokenizer (BasicTokenizer and WordpieceTokenizer) and SQuAD dataset parsing utilities.
    • A demo question answering app.
  • For GPT-2:
    • a conversion script from Pytorch trained GPT-2 models (see our pytorch-transformers repo) to CoreML models.
    • The GPT-2 generation model itself, including decoding strategies (greedy and TopK are currently implemented) and GPT-2 Byte-pair encoder and decoder.
    • A neat demo app showcasing on-device text generation.

🦄 GPT-2

Unleash the full power of text generation with GPT-2 on device!!



The pretrained Core ML model was packaged by Apple and is linked from the main ML models page. It was demoed at WWDC 2019 as part of the Core ML 3 launch.

core ml 3

🦄 Demo Time 🔥


Apple demo at WWDC 2019

wwdc demo

full video here

BERT Architecture (wwdc slide)



We use git-lfs to store large model files and it is required to obtain some of the files the app needs to run. See how to install git-lfson the installation page