eb46e739e7
Playwright-based crawler that renders pages from api.intra-mart.jp and document.intra-mart.jp, extracts main content, and converts to Markdown via Pandoc. SQLite-backed queue drives a resumable sequential pipeline across crawler, extractor, converter, and storage modules. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
12 lines
100 B
Plaintext
12 lines
100 B
Plaintext
playwright
|
|
beautifulsoup4
|
|
lxml
|
|
markdownify
|
|
pypandoc
|
|
loguru
|
|
pyyaml
|
|
httpx
|
|
aiofiles
|
|
tenacity
|
|
typer
|
|
rich |