Viglet Dumont
Viglet Dumont is an open-source data extraction platform. It connects your content (websites, databases, file systems, AEM, WordPress, and paged JSON APIs) to Viglet Turing and other search engines through a reliable, asynchronous processing pipeline.
Recursively crawl websites with URL filtering, authentication, locale detection and incremental change detection.
Run any SQL query against Oracle, PostgreSQL, MariaDB or MySQL and index each row as a searchable document.
Walk directory trees and extract text from PDFs, Word, Excel, PowerPoint and images (OCR) via Apache Tika.
Index on-premise, AMS and Cloud Service instances (author or publish tier) with delta tracking, locale mapping and a schema per content type.
Pull posts, pages and custom content types from WordPress installations into any search engine.
Index a job board, product catalog or partner feed from a JSON manifest: paging, daily and monthly call budgets, field mapping and expiry for sources a budget never lets you read in full, with no Java code.
Deliver content to Turing (default), Apache Solr or Elasticsearch via pluggable indexing adapters.
Point the wizard at a URL: Dumont infers the source's shape and field manifest, you correct the draft, and a dry run shows exactly what would be indexed (coverage, skipped records and a diff) before anything is written.
Model a source by its schema, not as a link spider: declared fields sent to the index ahead of the documents, values grounded rather than guessed, and coverage tracked so a layout change cannot silently drop a field.
A scheduled audit compares source, ledger and index, requeues what never landed, and shows where each field came from. A run cut short finishes without de-indexing, and a document to be deleted waits in a report you approve.
Every status a document passes through, filterable by source, run or document, so you can replay one run end to end. A log that cannot be read says so: a store that is down answers 503 rather than looking like a connector that indexed nothing.