[04] AI, Used Practically

AI is a tool, not a magic wand. My approach emphasizes restraint, using deterministic code where possible and LLMs only where necessary.

Principles

  • 01. Determinstic First: Check rules before calling an LLM. If a regex works, use it.
  • 02. Grounded Generation: Never let an LLM hallucinate. Always supply retrieval context (RAG).
  • 03. Cost Awareness: Use smaller models (Flash/3.5-Turbo) for structured tasks; save GPT-4 for reasoning.

Implementation Examples

Safe-Plate: Hybrid Classification

Why AI? Identifying obscure chemical additives that don't match exact strings in a DB.

Constraint: Cost & Latency.

Solution: We check a local JSON dictionary first (0ms). Only if the ingredient is unknown do we call Gemini Flash API (~400ms).

CollegeInfo: PDF Extraction

Why AI? Unstructured tables in syllabus PDFs.

Restraint: We don't summarize the whole PDF. We chunk it and retrieve only the relevant section to answer the student's question, reducing token usage by 90%.