Nutz 1.b.53 Strings: Unicode, Width and Case Boundaries

June 3, 2016

It is easy to mistake an application-specific character classification for Unicode’s complete definition. Nutz 1.b.53 Strings offers Chinese-character, full-width/half-width and case helpers. Understanding them starts with whether an API accepts a char or a complete code point.

UnicodeBlock identifies blocks

isChineseCharacter uses UnicodeBlock and returns true for some punctuation and width forms too. It expresses the utility’s classification rather than accurately identifying the language of a passage.

A Java char is one UTF-16 code unit. Supplementary-plane characters require two chars. Even if EXTENSION_B appears in source, an API accepting one char cannot fully represent those characters.

String text = "𠀀";
System.out.println(text.length());                         // 2
System.out.println(text.codePointCount(0, text.length())); // 1

String length, code-point count and the number of visible glyphs also differ. With emoji or combining characters, counting charAt calls does not count displayed characters reliably.

Width conversion covers specific ranges

Full-width ASCII forms U+FF01 through U+FF5E differ from ASCII by 65248, with the U+3000 space handled separately. This offset does not apply to every Chinese punctuation mark; U+3002, the ideographic full stop, lies outside the range.

Whether a character occupies two display cells also depends on the terminal, font and width policy. An older utility’s wide-character categories cannot determine every UI layout.

Consider case rules before performance

ASCII case can use simple range transformations, but general Unicode casing has language-dependent and length-changing rules. Choose Character or String APIs according to requirements and specify Locale where stable language behavior matters.

Estimating StringBuilder capacity from a known length can reduce resizing, but two lines of code do not establish relative performance. Prioritize correctness and readability, measure real hot paths, and then consider micro-optimization.

References