Botok

State-of-the-art tokenizers for Tibetan language.

This is the documentation of our repository botok.

Features

  • Support various dialects.

  • Fully customizable to world list and adjustments rules.

  • Allows adjusting word list and rules with Adjustments component of the Dialect Pack.

Contents

This is the content of botok.

Package Reference