Universal Collection Framework

Technical architecture for describing collections in LifeLoveMe so the platform can support Ramcharitmanas (RCM), Bhagavad Gita, Yoga Sutra, Bible, Quran, Kamasutra, chanting, Ayurveda/wellness, family tree, Bollywood songs, books, poems, essays, documents, audio, and video—without rewriting application code for each corpus. Apps read metadata and data files; collection-specific behavior is expressed through types, manifests, and schemas, not hard-coded branches for every title.

1. Purpose

Ramcharitmanas (RCM) is the first proof-of-concept collection: it validates search, indexing, audio, read-along, dictionary hooks, and Smart Search at production-like depth.

The Universal Collection Framework (UCF) is the contract by which future collections plug into the same platform: same mental model for identity, files, search, media, relationships, and deployment—different content and optional capabilities per CollectionType.

This document defines files, fields, identifiers, and boundaries. It does not mandate a single implementation language or UI framework; it mandates discoverable, versionable data that any LifeLoveMe client can load.

2. Core principle

Every collection should be described by metadata and data files, not by hard-coded app logic.

• Manifests declare what exists, how it is indexed, and which optional assets are present.
• Items carry universal fields; collection-specific nuance uses optional fields, tags, and typed relationships—not forked code paths for “RCM vs Gita.”
• Apps implement generic behaviors (list, hub, explore, play, search modes, graph) and collection profiles (which search modes are enabled, which media types appear, which Part semantics apply).

Hard-coding is allowed only at the edges (bootstrap URLs, feature flags, temporary shims) and should shrink over time as the framework absorbs patterns.

3. Collection types

CollectionType is a coarse classifier for routing, UI defaults, and validation rules.

A collection may combine types in profiles (e.g. sacred_text + media_collection); the manifest should declare the primary type and secondary capabilities.

4. Standard collection files

Each collection SHOULD ship a consistent directory of standard files. Names are conventional; paths are implementation-defined (e.g. /collections/{CollectionID}/).

Sharding: Large collections MAY split items.json into items-000.json, … with manifest entries listing shards. Search shards MAY mirror the same pattern.

Omission: Files that do not apply may be absent; the manifest MUST declare which optional files are included vs absent so clients do not probe blindly.

5. Universal item schema

Required fields

Raw vs Sort vs Display

• Raw — as ingested from editorial source (may include punctuation variants).
• Sort — normalized token used in LineRef construction and binary ordering (see §6).
• Display — pretty label for UI (may include Devanagari labels, book names, etc.).

Optional fields

6. Universal indexing

LineRef

LineRef identifies a primary textual line (or atomic item treated like a line):

LineRef = CollectionID.Part1Sort.Part2Sort.Part3Sort.Part4Sort

• Separator: dot . (URL-safe segment rules per deployment; encode when in query strings).
• Numeric parts: use 3-digit zero-padding in Sort segments when the segment is purely numeric (e.g. 4 → 004, 18 → 018).
• Alphanumeric parts: preserve suffix letters after digits where applicable, e.g. 4A → 004A, 208A stays `208A` (already three digit stem + letter; do not strip the letter). Rules are fixed per index_definition.json so parsers stay consistent.
• Part4Sort: may be a short safe code (metre, verse marker, Verse, Sutra, Ch, etc.); length cap is collection-defined but should stay identifier-safe (no spaces; prefer ASCII alphanumerics for maximum portability).

ItemRef

When a collection needs sub-line or multi-line packaged items (playlists, chanting cycles, UI cards):

ItemRef = LineRef.ItemNumberSort

ItemNumberSort is typically a padded ordinal under that line’s scope. If the collection has no sub-items, ItemRef may be omitted and LineRef is the sole canonical key.

Examples

Exact Part semantics are not global; they are declared in `index_definition.json` per collection so the same engine parses different Part layouts.

7. RCM mapping

Legacy Ramcharitmanas rows often carry RCM-specific columns. During migration and dual-read periods:

Migration policy

• Keep old RCM fields alongside universal fields until all consumers read LineRef / Parts.
• New collections must not depend on RCMID, LineID, KandNumber, etc.; they use CollectionID + Parts + LineRef/ItemRef only.
• Apps should prefer LineRef for linking, highlighting, and analytics; LineID remains for timeline continuity and tooling during transition.

8. Media schema

Each row in media_manifest.json (or equivalent) SHOULD support:

DeployTier

9. Timing schema

Rows in timing.json (or embedded segments) align media to text:

10. Relationship schema

Edges in relationships.json:

Examples by domain

• Family tree: ParentOf, SpouseOf, SiblingOf, ChildOf, PartnerOf
• RCM / narrative text: SpeakerOf, ListenerOf, Mentions, BattleWith, Blesses
• Songs: SungBy, WrittenBy, ComposedBy, FromMovie, FromAlbum

Relationship types SHOULD be registered in index_definition.json or collection_manifest.json so UIs can offer filters without code forks per type string.

11. Search framework

Search is implemented as modes over the same item corpus (and optional indices):

Default token rule (conventional)

• No wildcard: tokens match as whole words (word boundaries defined per language in index_definition.json).
• `*` wildcard: allows partial word / substring behavior where enabled (e.g. ram*).

Exact tokenization (Devanagari, ZWJ, chandrabindu) is locale-specific and must be specified per collection in the index definition—not hard-coded globally in app logic.

12. Ask LifeLoveMe routing layer

Ask LifeLoveMe is a global intent routing layer (box on home, plus APIs). It classifies utterances into intents and routes to the right surface:

Routing SHOULD be data-driven (intent patterns, aliases) with collection hooks (e.g. “Gita 2.18” → GITA index profile). Fallback: honest “not in library yet” plus suggested collections.

13. App architecture

Logical apps (may be routes in one SPA or separate deployables):

Clients load `collection_manifest.json` first, then lazy-load shards (items, search_index, media_manifest, etc.) according to flags.

14. Netlify deployment rules

• `deploy/web_public` remains lean: HTML/JS/CSS, small JSON, beta-tier media only.
• `deploy/web_public_large` (or equivalent) holds full local / archive builds: large media, full timings, bulk JSON.
• Preserve original filenames in manifests and on disk where possible.
• Do not copy full media archives into Netlify-sized deploys by default.
• Use `IncludeInDeploy` and `DeployTier` on every media row to drive build scripts.
• Copy only selected beta assets to lean deploy; everything else references external or local_only until policy changes.

Build pipelines SHOULD fail loudly if a lean deploy exceeds size budgets or if forbidden tiers are included.

15. Migration strategy

Staged rollout to avoid breaking the working RCM beta:

1. Keep RCM working — no regressions on search, play, read-along.

2. Add universal fields beside legacy RCM fields (dual-write / dual-read as needed).

3. Set `CollectionID=RCM` in manifest and items; treat RCM as first-class UCF collection.

4. Create generic collection library UI fed only by manifests (RCM entry first).

5. Create collection explorer wrapper that configures Explorer from manifest + index_definition (RCM becomes one profile).

6. Add small Gita sample — stress-test Parts and meaning search without full corpus.

7. Add chanting sample — short loop, media + timing smoke test.

8. Add family tree prototype — small relationships.json, Graph surface, privacy model.

9. Gradually retire RCM-only assumptions in code (replace with manifest-driven configuration).

16. Development guardrails

• Do not break the working RCM beta; ship incremental changes behind flags when risky.
• Do not edit `deploy/web_public` directly as a source of truth; promote builds from `web/apps` (or documented source tree) through an automated or repeatable copy step.
• Validate (lint, JSON schema, sample queries, timing spot-checks) before deploy.
• Maintain source vs deploy folder discipline (canonical code and data in web/ / data/; deploy is output).
• Add checkpoints (tags, branches, or dated bundles) for major schema or search-behavior changes.

Document status: architecture spec. Revise when the first non-RCM collection ships or when LineRef rules are extended for new Part layouts.