Sokjan Crawler
About
Sokjan is a focused web crawler used for private research, search experimentation, and language-model tooling. It runs from a small self-hosted lab environment and is not a mass-indexing or advertising crawler.
If you see requests from this bot, they are part of limited-scope experiments: indexing public pages, testing search relevance, and feeding a private assistant with fresher data.
For block requests configure robots.txt as described.
Crawl & Respect Policy
Sokjan obeys robots.txt, honours standard crawl-delay
directives where present, and is tuned for low request rates.
The crawler is target-scoped and can be fully excluded from your site using standard robots rules:
Disallow: /
To allow Sokjan but keep it gentle, you can instead use, for example:
Crawl-delay: 10