Fork me on GitHub

Alex Rupérez

Link: http://j.mp/2YCV0YJ
Published: 29 Jul 09:01
I starred huggingface/swift-coreml-transformers at GitHub.
Swift Core ML 3 implementations of GPT-2 and BERT for Question answering. Other Transformers coming soon!
View on GitHub


View README


Swift Core ML implementations of Transformers: GPT-2, BERT, more coming soon!

This repository contains:

  • For BERT:
    • a pretrained Google BERT model fine-tuned for Question answering on the SQuAD dataset.
    • Swift implementations of the BERT tokenizer (BasicTokenizer and WordpieceTokenizer) and SQuAD dataset parsing utilities.
    • A demo question answering app.
  • For GPT-2:
    • a conversion script from Pytorch trained GPT-2 models (see our pytorch-transformers repo) to CoreML models.
    • The GPT-2 generation model itself, including decoding strategies (greedy and TopK are currently implemented) and GPT-2 Byte-pair encoder and decoder.
    • A neat demo app showcasing on-device text generation.

🦄 GPT-2

Unleash the full power of text generation with GPT-2 on device!!

demo

🐸 BERT

The pretrained Core ML model was packaged by Apple and is linked from the main ML models page. It was demoed at WWDC 2019 as part of the Core ML 3 launch.

core ml 3

🦄 Demo Time 🔥

demo

Apple demo at WWDC 2019

wwdc demo

full video here

BERT Architecture (wwdc slide)

bert

Notes

We use git-lfs to store large model files and it is required to obtain some of the files the app needs to run. See how to install git-lfson the installation page