$29 · one-time · v1.0.0
SplitCheck Toolkit
The browser checker is free and always will be. The toolkit is for when the data cannot go in a browser, or when you want this to run on every pull request.
What you get
- The `splitcheck` Python package and CLIPure Python, no required dependencies, 3.9+.
- All three detection levelsExact, normalized, and MinHash/LSH near-duplicate — same constants as the free checker.
- No size ceilingRuns on datasets that will not fit in a browser tab, on your own hardware.
- CI-ready exit codes0 clean, 1 contaminated above your --fail-over rate, 2 usage/parse error.
- Markdown and JSON reportsThe JSON report lists every matched pair, uncapped.
- Cleaned training file--clean writes your train file minus every matched record, as JSONL.
- Parquet and Hugging Face supportOptional extras: `pip install 'splitcheck[parquet]'`.
- A GitHub Action workflowCopy-paste a job that fails the PR when contamination appears.
What it is not
- Semantic/embedding-based similarity. Shingling finds lexical overlap; genuine paraphrases with no shared 5-gram are out of scope.
- A hosted service. Nothing phones home; there is no server component to the toolkit.
- Support beyond email. It is one person answering, usually within a couple of days.
Being straight with you about the value
MinHash deduplication is not a secret. There are free libraries that do the same maths, and if you are comfortable wiring one up, you should — it will cost you an afternoon instead of $29. What you are paying for here is that the afternoon is already spent: a CLI with sensible defaults, field auto-detection, reports you can hand to a reviewer, an exit code CI understands, and tests that prove it behaves. If that trade is not worth $29 to you, the free checker is genuinely free and has no upsell wall.