Advanced Search

Paid pluginsize Lidea, not a commitment

Indicative price, not an offer: Part of the suite for the two live shops; standalone price is an owner decision

Generated from docs/plan/paid/advanced-search/PLAN.md in the project's planning notes, which are not part of the public repository. Nothing described below is implemented — it is the written plan for it.

An idea, not a commitment. It has a written plan but no place in the build order and no date. What is committed.

Advanced Search is the paid @astrobaas/search module (commercial repo, packages/search, also bundled in @astrobaas/suite). It makes the free core search forgiving: Greeklish typed on a Latin keyboard, Greeklish however it is spelled, and typos — all answered from an index of the shop’s own vocabulary, so every correction is a word the shop actually sells. It runs in process, needs no external service, and never takes over ranking.

This plan replaces an earlier Elasticsearch design (backed up next to this file as PLAN.md.bak-2026-10-13). That design was never built; the lexicon module was, and it is what this plan describes and extends.

Status note: planned — partially built (1.1.0), see Gaps. The module exists and is tested; the plan as a whole is not done. The machine-read status line above stays at its canonical planned, not built. because scripts/regen-plan-index.py accepts only planned, not built., shipped or merged there and fails on anything else; it stays planned until the owner marks it shipped.

The problem

The free core search (C-146, src/lib/search/rank.ts) is ranked, weighted by field, accent- and case-insensitive, and has an operator synonym table. What it cannot do is guess. A Greek shop’s customers type:

• on a Latin keyboard: skeletos for «Σκελετός»;
• in whatever Greeklish they grew up with: skeletws, xeimerina, boufan, 8hkh;
• with a slip: γυαλιαα, skeltos.

Each of those finds nothing today. A dictionary-based spell-checker is the wrong fix: it suggests words the shop does not stock, and acting on the suggestion lands the shopper on an empty page.

What it does

All of it through one core seam, SEARCH_EXPAND (a TermExpander: “what else might this term be?”). The core consults it only for terms that matched nothing, memoised per query, and ranks anything reached through it below a literal match.

• Lexicon. Built per search from the catalogue’s short, high-signal fields — name, brand, SKU, GTIN, tags (descriptions deliberately excluded as prose noise). Words under 3 letters are not indexed; hard cap 50 000 words.
• Greeklish (exact). Every catalogue word is transliterated once with the core’s own transliterate; the shopper’s Latin term is then a lookup, not a reverse-mapping guess (i is ι, η, υ, ει or οι — the reverse direction cannot work).
• Greeklish spelling variants (1.1.0). A loose key both sides are reduced to — x/ch/ks, w/o, 8/th, b/mp, d/nt, i/h/y/ei/oi, u/ou, av/af/ay, doubled letters — compared exactly. Greek words only; Latin brand names keep their own spelling.
• Typo tolerance. Bounded Damerau-Levenshtein against the shop’s own words, scaled by length (≤4 letters exact, ≤7 one edit, longer two), transpositions cost one, closest-then-alphabetical so results are stable.
• Greeklish typos (1.1.0). The same distance search over loose keys, so a Latin-keyboard slip (skeltos) still reaches the Greek word. Needs both the Greeklish and the typo switch.
• Composes with the core’s synonyms (1.1.0). The hook’s initial value is the operator’s synonym expander from Settings → Reading. The module answers synonyms first, then its own sources, de-duplicated, never echoing the term. It returns the operator’s expander unchanged when it has nothing to add.
• Isolation. Each source runs in its own try/catch: if a lookup fails, the operator’s synonyms and the plain core search still answer. That is the lexicon design’s “fallback to native search” — there is no second engine to fall back from.
• Admin screen at /admin/plugin/search/ (admin role only, matching the settings endpoint it writes to): one switch per feature, default on, CSP-safe markup (no inline style or script).

What it deliberately does NOT do

Each boundary carries its reason. A boundary without a reason gets crossed by the next person who reads this.

• Does NOT rank. Field weights, phrase bonus and ordering stay in the core’s rank.ts. Reason: two scorers that disagree return plausible, different answers to one query, and nobody can see why.
• Does NOT own synonyms. They are free in core (C-146). Reason: they are operator data needing no index; and a second parser of the same key (what 1.0.0 had) silently changed what the operator’s rules meant.
• Does NOT use Elasticsearch, Meilisearch or any external service. Reason: the two live shops have hundreds of products, not millions; an in-process index of their own vocabulary is exact, free to run, and cannot be down. A shop that outgrows it is a different product decision, not a setting.
• Does NOT offer facets — waiting on owner decision 9 (“what search offers free”). Reason: whether brand/price/category filters are core or paid is an owner call, still open. Nothing is built for them until it is made.
• Does NOT index descriptions. Reason: prose words (“και”, “με”, “για”) are never searched for, and each is another target for a typo to land on wrongly.
• Does NOT cache the lexicon across requests. Reason: a renamed product would stay findable only under its old name with nothing saying why. Rebuild cost is one pass over short fields.
• Does NOT check a licence, phone home or expire. Reason: a shop’s search box must not depend on a licence server’s uptime; enforcement is commercial (updates and support).
• Does NOT stem or do language morphology. Reason: needs per-language rules; the vocabulary index already covers the forms the shop actually uses.

Data model

No tables. Three settings, all read from the ctx.settings map the core’s search call site passes into the hook (keys owned by src/settings.ts):

Only an explicit off turns a feature off: false, "false", "0", 0. search_synonyms belongs to the core.

The lexicon is in memory, per search: words, byLatin (transliteration → Greek words) and byLoose (loose key → Greek words).

API

No routes of its own. It plugs into the core’s existing search surfaces through SEARCH_EXPAND:

Nothing here adds a route, so nothing needs adding to openapi.json.ts. If a future gap adds one (e.g. G4’s index status), it must go there too.

Admin

/admin/plugin/search/ — “Search” in the sidebar, admin only. Three switches (Greeklish, Greeklish spelling variants, typo tolerance), each with help text and examples; a link to the core’s synonym field (Settings → Reading → Search synonyms) instead of a second editor for the same key. Saves through the core’s /api/settings/update; shows the endpoint’s own error message on failure.

The seam — why this is paid

Core owns: ranking (rank.ts), folding and transliteration (text-search.ts, lent to the module as host.text), the synonym table and its parser (expander.ts), the suggestion box, popular searches, and the SEARCH_EXPAND hook. Paid module owns: the vocabulary index and everything that needs it — Greeklish, spelling variants, typo tolerance.

The module imports nothing from the host. The host hands it definePlugin, PLUGIN_HOOKS, text: { fold, transliterate } and getSettings; a host without text is refused loudly at load, and the suite skips the module by name rather than failing the bundle.

Dependencies

• Core SEARCH_EXPAND hook with the operator’s synonym expander as initial value (core main since 2026-09-02).
• Core PluginHost.text (foldForSearch, transliterate) and getSettings.
• Plugin admin pages (adminPages, /plugin-admin.js).
• Nothing external. Facets would add a dependency on decision 9.

Gaps — what is left, in priority order

Closed in 1.1.0 (commercial branch feat/advanced-search-gaps): composition with the core’s synonyms (was a bug), per-source isolation, Greeklish spelling variants, Greeklish typos, switches that read text "false" as off, the admin screen without the duplicate synonym editor, a lazily built lexicon (nothing built unless a term misses; catalogue not read with every switch off), barcodes and SKUs kept out of typo correction, a one-time warning on a core too old to hand in the synonym table, tests for the wiring.

Left, as follow-ups:

• G1 — Greeklish in the suggestion box. /api/search/suggest does not put products in the hook context — and it cannot as written: it applies the hook (≈ line 57) BEFORE it loads products (≈ lines 65–67), and loads them only when the requested types include products or categories. The core change is a reorder (load first, then call the hook with products in ctx), plus a decision on whether a posts-only suggestion request should load the catalogue just to build a lexicon (recommended: no). Core seam, so not built from this repo.
• G2 — Greeklish in post/blog search. Same shape: the post routes hand no corpus. Needs the core to pass post titles/tags, or a lexicon source the host provides. Core seam.
• G3 — Did-you-mean. nearMisses already computes it; nothing surfaces it. Needs the core search response to carry a suggestion field. Core seam.
• G4 — Index status in admin. Word count, Greeklish keys, a “try a word” box showing what a term expands to. Needs catalogue access from an admin page; host.catalog exists only in an open core PR (#147) and must not be relied on until merged.
• G5 — Zero-result report. What shoppers searched and found nothing, and which of those this module rescued. Needs a module-owned store; host.createStore is in an open core PR (#142). The core’s popular-searches store counts only searches that found something.
• G6 — Lexicon reuse within a request / across identical catalogues. Today rebuilt per search. A cache keyed on a cheap catalogue fingerprint would cut the cost for large catalogues without the staleness risk; measure first.
• G7 — Operator-tunable field weights. Belongs in core rank.ts (relevance is core’s), so it is a core item, listed here only because merchants will ask.
• G8 — Facets. Waiting on owner decision 9. Not to be started before it.
• G9 — End-to-end through a real host. A test that boots a built core with the suite and drives a product search for skeletws through the hook.

Acceptance checks

Each of these must be able to fail. Before claiming this is done, break the code deliberately and watch each one go red.

• skeletos finds a product named «Σκελετός» (Greeklish exact).
• skeletws, xeimerina, boufan, doulapa, 8hkh each find their Greek product; with the variants switch off, skeletws does not (exact still does).
• skeltos finds «Σκελετός» with Greeklish and typos on; with either off, it does not.
• γυαλιαα finds «Γυαλιά»; a 4-letter typo finds nothing (short words exact).
• An operator synonym (μοντούρα, σκελετός) keeps working on product search with the module installed, and comes before the module’s own guesses.
• On post search (no catalogue) the module returns the operator’s expander itself, not null and not a replacement.
• With every switch off, the hook returns exactly what it was handed.
• A setting stored as the text "false" switches its feature off, on the screen and in search alike.
• A failing lookup inside the module does not cost the operator’s synonyms.
• A Latin brand name is never bent into another Latin word by the variants key.
• Every suggestion is a word in the catalogue.
• A pasted barcode or SKU one character off a stocked one finds nothing through this module.
• With every switch off the catalogue is not read; with switches on, no lexicon is built for a search whose terms all match, and at most one per search.
• The admin screen has no inline style or script, escapes the CSRF token, and is admin-only.
• The suite bundle loads search with the same version and hooks as the standalone package.
• G1 (when built): /api/search/suggest?q=skeletws offers «Σκελετός».

Risks

• Loose-key false friends. Merging χ/ξ, ι-sounds and doubles can make two different Greek words share a key. Bounded: candidates are always shop words, rank below literal matches, and the core keeps at most 12 per term; the variants switch turns it off.
• Cost on large catalogues. The lexicon is built at most once per search, only when some term missed, and the typo scan is linear in vocabulary size per missing term. Measured by review at ~44 ms for 500 and ~177 ms for 5 000 products per build. Fine at hundreds to low thousands of products; G6 if a shop is much larger.
• Host contract drift. The module depends on two things the core hands the hook. (1) The synonym expander as the starting value: every public core release (0.1.0 on) does; on an older core, or one that stopped doing it, the operator’s synonyms would stop applying — the module logs one warning when it sees a synonym table configured but no expander handed in. (2) products in the context on product search: a core that stopped passing them would silently turn the module into a pass-through, with no error. The suite test pins the composition; G9 would pin the host side.
• h read as η can land on a χ word. The loose key reads a bare h as η (kalh → καλή), which is the common Greeklish spelling. A shopper who writes h for χ (hronia) gets that only through the typo path, one edit away — and, rarely, a word whose η-reading exists in the catalogue is offered instead. Bounded the same way as other false friends: shop words only, ranked below literal matches.
• β typed as b. Not merged: b already stands for μπ, and merging β in would make vazo and bazo one key. Reached through the typo path.
• Test doubles. Unit tests use stand-ins for fold/transliterate; the expander test can re-run against the core’s real text-search.ts (ASTROBAAS_CORE_TEXT_SEARCH), which should be part of any release check.

Commercial context

The anchor is what the nearest equivalent charges on Shopify or Magento today. It is context for a pricing decision, not the decision.

---

Originally generated from the commerce plan; rewritten 2026-10-13 to describe the lexicon module that exists. See docs/COMMERCE-PLAN.md for the full catalogue and CONTRIBUTING.md for how to work on this repository.