pg_icu_parser

Overview

SQL/control version 1.0; upstream release 1.0.0. Local RPM/DEB builds support PG14-18 with a PG18 encoding-API compatibility patch and explicit ICU linkage. ts_headline is unsupported. PIGSTY PG14-18 packages verified across EL8/9/10, Debian 12/13 and Ubuntu 22.04/24.04/26.04 on x86_64/amd64 and aarch64/arm64; all 16 platforms imported into the local PIGSTY repository.

Version

Build

You can build the RPM / DEB packages for pg_icu_parser using pig build:

pig build pkg pg_icu_parser         # build RPM / DEB packages

Install

You can install pg_icu_parser directly. First, make sure the PGDG and PIGSTY repositories are added and enabled:

pig repo add pgsql -u          # Add repo and update cache

Install the extension using pig or apt/yum/dnf:

pig install pg_icu_parser;          # Install for current active PG version

pig ext install -y pg_icu_parser -v 18  # PG 18
pig ext install -y pg_icu_parser -v 17  # PG 17
pig ext install -y pg_icu_parser -v 16  # PG 16
pig ext install -y pg_icu_parser -v 15  # PG 15
pig ext install -y pg_icu_parser -v 14  # PG 14

dnf install -y pg_icu_parser_18       # PG 18
dnf install -y pg_icu_parser_17       # PG 17
dnf install -y pg_icu_parser_16       # PG 16
dnf install -y pg_icu_parser_15       # PG 15
dnf install -y pg_icu_parser_14       # PG 14

apt install -y postgresql-18-pg-icu-parser   # PG 18
apt install -y postgresql-17-pg-icu-parser   # PG 17
apt install -y postgresql-16-pg-icu-parser   # PG 16
apt install -y postgresql-15-pg-icu-parser   # PG 15
apt install -y postgresql-14-pg-icu-parser   # PG 14

Create Extension:

CREATE EXTENSION pg_icu_parser;

Usage

Sources:

• Official PGXN documentation

pg_icu_parser supplies the icu_parser full-text-search parser, which uses ICU Unicode word-boundary analysis. It is useful for languages such as Chinese, Japanese, Korean, and Hebrew where PostgreSQL’s whitespace- and pattern-oriented default parser may not produce suitable word boundaries.

Core Workflow

The server must be PostgreSQL 10 or later and must have been built with ICU support. Create the extension, then create a text-search configuration that selects its parser and maps the token types you need:

CREATE EXTENSION pg_icu_parser;

CREATE TEXT SEARCH CONFIGURATION sample (PARSER = icu_parser);
ALTER TEXT SEARCH CONFIGURATION sample
  ADD MAPPING FOR word WITH english_stem;
ALTER TEXT SEARCH CONFIGURATION sample
  ADD MAPPING FOR number, ideo, kana WITH simple;

SELECT to_tsvector('sample', '東京でPostgreSQLを使う');

Choose dictionaries for the actual language; english_stem above only demonstrates the mapping syntax.

Parser Surface

• word, number, kana, ideo, and blank are the token types emitted by icu_parser.
• pg_icu_parser.locale selects the ICU locale and defaults to en. Upstream notes that word-boundary rules are usually not locale-sensitive, so change it only after testing a concrete need.
• Any PostgreSQL text-search dictionary can be mapped to the emitted token types.

Trade-offs

• The parser requires ICU in the PostgreSQL build; creating the extension cannot add ICU to a server built without it.
• It emits fewer and coarser token types than the default parser and does not recognize special patterns such as URLs or email addresses as distinct categories.
• The reviewed release does not support ts_headline.
• Upstream reports that it is slower than the default parser, with a lower conversion overhead when the database encoding is UTF-8. Benchmark representative documents and query shapes on the target system.
• When parser mappings or locale behavior change, rebuild stored tsvector values and dependent indexes so old and new rows do not use different tokenization.