1
0
Files
breadcrumb-the-shire/docs/README-search.md
2026-02-04 23:31:53 +01:00

1.1 KiB

Search + Upload TODO (Internal)

Goal: Unified search across DB data and file contents (PDF, DOCX), starting with MariaDB FULLTEXT and private file storage.

MVP Steps

  1. Create documents table (uuid, original_name, mime, size, storage_path, content_text, created).
  2. Add FULLTEXT index on content_text (and later title/metadata fields if needed).
  3. Build FileUploadService:
    • Validate size/type (PDF/DOCX).
    • Store outside /web (e.g. storage/uploads/{uuid}/file.ext).
    • Extract text:
      • PDF: pdftotext (poppler-utils).
      • DOCX: unzip + parse word/document.xml (fallback: LibreOffice headless).
    • Save content_text + metadata.
  4. Add upload action + form (admin area).
  5. Add search endpoint:
    • Search DB entities.
    • Search documents.content_text.
    • Merge results into one list.

Future Enhancements

  • Async extraction (queue/worker) for large files.
  • Dedicated search engine (Meilisearch/Typesense) when scale/relevance needs grow.
  • Result ranking with field weighting (title > body).
  • Optional language detection / stemming.