A Formal Perspective on Byte-Pair Encoding

July 1, 2023