crawler-intra-mart/crawler
Do Duy eb46e739e7 Add intra-mart documentation crawler
Playwright-based crawler that renders pages from api.intra-mart.jp and
document.intra-mart.jp, extracts main content, and converts to Markdown
via Pandoc. SQLite-backed queue drives a resumable sequential pipeline
across crawler, extractor, converter, and storage modules.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 16:29:54 +07:00
..
__init__.py Add intra-mart documentation crawler 2026-05-20 16:29:54 +07:00
crawler.py Add intra-mart documentation crawler 2026-05-20 16:29:54 +07:00
http_client.py Add intra-mart documentation crawler 2026-05-20 16:29:54 +07:00
navigation_parser.py Add intra-mart documentation crawler 2026-05-20 16:29:54 +07:00
playwright_client.py Add intra-mart documentation crawler 2026-05-20 16:29:54 +07:00
sitemap_parser.py Add intra-mart documentation crawler 2026-05-20 16:29:54 +07:00
sphinx_discovery.py Add intra-mart documentation crawler 2026-05-20 16:29:54 +07:00
url_manager.py Add intra-mart documentation crawler 2026-05-20 16:29:54 +07:00