Softanza
StartFrançais
Softanza
StartFrançais
Core data & language

String

Codepoint-correct superset of Python and Rust str; the clear gap is ICU's locale layer.

6 Strong · 4 Solid · 1 Partial · 1 Emerging

Measured against: ICU · Rust/Python str · Boost.Locale · Folders: base/string(opens in a new tab) · For programmers: this area's reference

one word in six scripts, counted by the engine
one word in six scripts, counted by the engine. rendered on 2026-10-01 at 04:00 by the site's render script tools/areas.ring, run inside the library at commit 0e72e2e2c · the source(opens in a new tab)

Rethought from first principles

What Softanza kept from what works, and what it rethought, as the library wrote it down in its design documents and narrations.

The Softanza wayA string is a sequence of characters a person reads, counted in Unicode code points, never in the bytes a machine stores.

Kept from the best practice

  • Case rules come from the Unicode standard through utf8proc, so case-insensitive search folds German ß to ss instead of using ASCII-only tricks. source(opens in a new tab)
  • Whitespace follows Unicode, which defines more than 25 space characters, so trimming also removes no-break and ideographic spaces. source(opens in a new tab)
  • Identity follows Unicode: two characters are equal only when their code points match, so look-alike Latin, Greek and Cyrillic letters stay distinct and detectable. source(opens in a new tab)

Rethought

  • Every position and length counts code points, not bytes, so text in Hebrew, Arabic or emoji is never cut through the middle of a character. source(opens in a new tab)
  • Positions start at 1, as people count, at every layer including the compiled engine, which removes a whole class of off-by-one errors. source(opens in a new tab)
  • The empty string is never counted, found, contained, replaced or removed, because a match on nothing can silently delete a real character. source(opens in a new tab)
  • Case sensitivity is one parameter of a single function, not a second twin function, which halves the number of functions to learn. source(opens in a new tab)

Softanza's string is one Zig engine (utf8proc + PCRE2 + SheenBidi/HarfBuzz) behind a 2,212-method Haro face that is codepoint-correct end to end and reaches from core slicing through similarity, formatting and text analytics: a genuine superset of Python/Rust str. What it deliberately is not is ICU: no locale collation, no CLDR, no legacy charset transcoding.

What a crafter does with it, and what stands out

  • A genuine superset of Python/Rust str on breadth and Unicode-correctness
  • Between/sections finder vocabulary wider than any str library exposes
  • A similarity and distance lane that ICU does not even enter
  • Formatting and text-analytics lanes general str libraries do not have

What is owed

  • Locale-aware collation + CLDR foundation (ICU's crown jewel)
  • Finish the narration audit: ~736 of 999 blocks unproven
  • Legacy charset transcoding (Shift-JIS, GBK, ISO-8859-*, Windows-125x)
  • Grapheme-cluster cursor (UAX#29) and line-break (UAX#14) primitives

The Softanza way, run

What Softanza does differently in this area, shown by code taken from its narrations and guards, and run for this page.

The empty string is never contained, not even in itself, so a match on nothing can never trigger an edit.

Softanza
? Q("").Contains("")
? Q("").Contains("text")
? Q("text").Contains("")
Output
0
0
0

run on 2026-10-01 21:17 inside the library at commit 0e72e2e2c; taken from base/doc/narrations/stzstring-emptiness-narration.md(opens in a new tab)

Length counts characters, not bytes: the accented word héllo has five characters although it takes six bytes.

Softanza
Then("NumberOfChars('héllo') is 5 (not byte count)", Q("héllo").NumberOfChars(), 5)
Output
  THEN  NumberOfChars('héllo') is 5 (not byte count)  [PASS]

run on 2026-10-01 21:17 inside the library at commit 0e72e2e2c; taken from base/test/string/00_string_narrated.ring(opens in a new tab)

Three letters that look identical are told apart by their code points and their scripts: Latin, Greek and Cyrillic.

Softanza
? AreEqual(["O", "Ο", "О"])
? Unicodes(["O", "Ο", "О"])
? Scripts(["O", "Ο", "О"])
Output
0
79
927
1054

latin
greek
cyrillic

run on 2026-10-01 21:17 inside the library at commit 0e72e2e2c; taken from base/doc/narrations/stz-unmasking-unicode-lookalities-narration.md(opens in a new tab)

The articles of this area (11)

The lanes, one by one

Lanes and their notes are quoted as they were read at the source, on Read from the source on 2026-09-16.

Unicode-correct core ops
Strong
Length, slice, index, insert, reverse all codepoint-correct via _EngineSlice/_EngineCount/str_mid_cp over utf8proc; guarded at multi-million codepoints_EngineSlice / _EngineCount / str_mid_cp (utf8proc)
Search, find, between, sections
Strong
Full finder family over str_find_* / str_between_*; between/sections vocabulary is a superset of str libraries; returns all matches past 65,536 (guarded)str_find_* / str_between_*
Replace, remove, edit
Strong
Codepoint-safe editing across replace.zig (70 KB): Replace[CS], ReplaceMany, positional, range and between edits; replaces Haro's byte-oriented substrengine/src/string/replace.zig
Segmentation and tokenization
Strong
word_break.zig (UAX#29) drives Words, sentence and line splitting, streaming tokenizers and n-grams; split is one-pass linear (guarded)engine/src/string/word_break.zig
Normalization
Strong
Full NF-form normalization through utf8proc (str_normalize), mark stripping (RemoveDiacritics), space collapsing and Simplifystr_normalize / str_strip_marks (utf8proc)
Similarity and distance
Strong
levenshtein, hamming, jaro, jarowinkler, jaccard, cosine, LCP/LCS, anagram, char-diff over codepoints in fuzzy.zig/compare.zig; a lane ICU does not enterengine/src/string/fuzzy.zig, compare.zig
Case, script, direction, language detection
Solid
Unicode-default case via utf8proc plus detectScript/detectLanguage/detectDirection; Unicode-correct but not locale-correct (Turkish i, Greek sigma)detectScript / detectLanguage / detectDirection
Formatting, padding, alignment, boxing
Solid
format.zig (75 KB): codepoint-width pad/justify/center, wrap, truncate, indent/dedent, mask, numberlines, boxing; box drawing emits ASCII not Unicode glyphsengine/src/string/format.zig
Pattern, regex, fuzzy
Solid
Real PCRE2 under regex.zig (match/count/find-all/replace/split, Unicode) plus a W-conditional DSL (28 *W methods); deep regex story rated on its own pageengine/src/string/regex.zig
Text analytics (NLP)
Solid
VADER sentiment, NER, POS, lemma (en/fr/ar), stem, WordNet, TextRank/TF-IDF, summarize, readability, stopwords; dictionary-grade and English-centred
Encoding, charset, crypto
Partial
encode.zig covers url/html, base64, hex, RLE, CSV; md5/sha256/blake3/hmac; no legacy charset transcoding (Shift-JIS, GBK, ISO-8859-*, Windows-125x)engine/src/string/encode.zig
Locale-tailored ops and collation
Emerging
Locale is Haro-side tables (stzLocale, stzCountry, stzLanguage, stzScript); engine locale.zig is ten C-locale stubs; no collation, no CLDR; L0 not startedlocale.zig; SOFTANZA_LOCALE_PLAN.md