picture_as_pdf
1. PDFLoaderStage
Uses PyMuPDF (fitz) to load the raw PDF file, count pages, and analyze metadata to determine if it is text-based or scanned (image-only).
Production-Grade Parser
Exam Parser Pipeline
Convert dense competitive exam PDFs into perfectly structured JSON data. Powered by a modular sequential processing architecture, featuring visual-aware LLM segmentation, self-correcting Pydantic validations, and automatic diagram extraction.
account_tree
9-Stage Pipeline Architecture
image
2. ImageExtractorStage
Extracts or renders every page of the PDF into a high-quality 200 DPI PNG image. Visual context is crucial for LLMs to interpret dense 2-column layouts, mathematical symbols, and diagrams.
document_scanner
3. OCRStage
Extracts raw text natively via PyMuPDF. If the page is scanned (0 characters extracted), it falls back to Tesseract OCR to build the text layer.
splitscreen
4. QuestionSegmenterStage
Applies regex pattern matching to locate question numbers and split the page text into individual, manageable question chunks. Safely falls back to page-level LLM parsing if chunking fails.
psychology
5. LLMParsingStage
The core AI stage. Sends the question chunk text and the high-res page image to the LLM (Gemini, DeepSeek, Qwen) to parse unstructured content into structured JSON.
fact_check
6. JSONValidationStage
Validates the raw LLM output against the strict Pydantic Question schema. If errors occur, it captures the error trace and prompts the LLM to self-correct (up to 3 retries).
crop
7. DiagramCropperStage
Takes normalized [ymin, xmin, ymax, xmax] coordinates from the LLM, scales them to the high-res image, and crops diagrams. Uses a Union Merge algorithm to combine overlapping crops.
quiz
8. AnswerResolverStage
Determines correct answers and explanations. Integrates three routes: reading an official answer key, querying Tavily Web Search, or requesting LLM solver execution.
file_download
9. ExportStage
Assembles all validated questions and sections into the final Paper schema. Exports a complete JSON file and extracts metadata (title, year) visually from the cover page if needed.
Architecture Highlights
LLM Factory Pattern
Enforces an abstract contract (BaseLLM) defining `.generate` and `.generate_with_image`. The pipeline manager remains provider-agnostic, allowing seamless provider switching via environment variables.
Resilient Validation Loop
If an LLM's response fails Pydantic schema validation, the system feeds the broken output and Python stack trace error back to the model, enabling quick self-correction of commas or formatting issues.
Visual context for equations and tables
By supplying high-resolution PNGs along with OCR chunks, the LLM maintains visual awareness of subscripts, mathematical notation, chemical diagrams, and multi-column document structure.
Deterministic Regex Chunking
Before LLM calls, the segmenter splits pages into individual question chunks. This mitigates LLM omissions on long, dense competitive exam papers by limiting the task size per invocation.
Technology Stack
PDF & OCR Processing
LLMs & Integration
Schemas & Core
Testing & Tooling
Key Features
- Strict Pipeline/Stages Architecture isolates complex tasks and guarantees separation of concerns
- Deterministic preprocessing and post-processing limits AI dependencies to only one single stage
- Pydantic-based validation with automatic self-correcting error-trace retry loops (up to 3 times)
- Visual-native parsing using high-resolution 200 DPI page images to prevent skipping dense text
- Normalized 0-1000 bounding-box coordination scale abstracts pixel dimensions for reliable diagram cropping
- Union Merge crop algorithm groups multiple bounding boxes of related figures or tables cleanly
- Dynamic cover-page LLM visual metadata extraction fallback for ambiguous filenames
- Swappable LLM provider factory allows switching between Gemini, DeepSeek, Qwen, and Ollama in 2 lines of config