Awesome Benchmarks & research Jev Projects

Open 0.6B Jev replica with parallel decisions, complete probability distributions, training pipeline, weights, dataset, and live demos.

Train a small model that chooses among a changing list of text options, one probability per option in a single pass. Includes Doom, chess, and Wikispeedia demos.

When enabled for Codex runs, Jev reviews diagnostic tests and incident-resolution submissions from the supplied evidence.

Build calibrated AI Functions from human feedback using Jev and GEPA.

Tasksource recasts hundreds of classification datasets into Jev-compatible typed questions, answers, and procedural training examples.

An educational Jev-like visual inference experiment on Apple Silicon: shared context, direct candidate scoring, and local visual demos.

Benchmark comparing Jev Choice answers with LLMs on four-option questions about Convex.

One-pass typed decisions with calibrated probabilities (System One style model), fine-tuned from Qwen3.5-2B.

Jev model introduction and actual measurement: use Choice / Score / Noul to convert natural language into typed judgments and probabilities for classification, scoring and routing; supports comparison of comment marking, speed and results with models such as DeepSeek, including CSV/Excel import, original speed playback and offline reporting..

Personal-assistant agent built on Vercel's eve with 100 mocked tools, measuring how many steps it takes when Jev picks the tool versus the LLM.

A small open decision model: state + typed questions -> calibrated probabilities. A Jev / System One re-creation on Qwen3.5.

WebMCP benchmark comparing browser-agent interfaces, with Jev included as one of the evaluated configurations.

JevBench v1 - a benchmark for Jev-class typed decision models: smart, cheap, fast, reliable, open.

One-pass option scoring with a local Gemma 3 4B on Apple silicon via MLX, inspired by jevlike, with a Doom demo.

An open-source JEV-powered workbench for streaming data selection, quality evaluation, automatic LoRA training and held-out model evaluation.

Jev-style parallel constrained decisions for any MLX model on Apple Silicon. Typed, schema-valid JSON in one forward pass.

This is a LLM Gateway that mimics typesafe ai structured output. Like an imposter Jev.

Calibrate Jev questions against your own labels: tune criteria on labelled examples, confirm on a held-out set, get a verdict per question. Unofficial.

Non-autoregressive decision engine on ModernBERT (151M) with calibrated uncertainty (RLCD), TypeSafe AI Jev benchmark audit, and in-browser WebGPU playground.

Mini-Jev: what a Jev-style typed-decision interface looks like on a frozen Qwen3-4B — read the option letter's logits instead of generating JSON. Preregistered experiment, results, teaching bench.

Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文为主的双语 repo。.

A reproduction of Jev that turns any Qwen model into a fast decision model, serving the same /v1/systemone schema (Choice, Score, Noul) with no training and no generated answer text.

Unofficial study: Jev-style parallel typed decisions on stock 1.5B-8B models on an Apple Silicon laptop. Benchmarks, research notes, and a Hugging Face Space demo.

Jev vs Gemini 3.8 Flash: labelling 1,000 app reviews, 4.1× faster and 7× cheaper.

Open alternative to Jev: typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with benchmarks.

Measures what your Jev classifier's confidence is really worth, and sets the human hand-off line from what a mistake costs.

Local bilingual probability decisions from context, questions, and candidate answers. Independent research preview inspired by TypeSafe Jev.

Reproducible benchmark for measuring Jev reranking quality, latency, and cost in RAG.

Batched single-token choice inference for open language models, compatible with TypeSafe.

Open replica of TypeSafe's Jev: typed calibrated decisions in one forward pass, on Gemma 4 E2B / Gemma 3 270M (Modal).

A stronger one-pass scorer over a variable list of text options: hashed n-gram encoder, rival-aware attention, gated head, temperature scaling, benchmarked against jevlike.

Type-safe one-decision-per-token decoding engine for autoregressive LLMs, inspired by Jev.

Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.

Using Jev as an evaluator.

看看 Jev 能做什么：用中英文讲清热门应用、工作原理和各自优缺点。Explore Jev apps with plain-language examples, explanations, and comparisons.

Browser-agent comparison that benchmarks a one-look open-model policy against the Jev demo.

An agent skill to discover TypeSafe Jev opportunities, design typed questions, and learn from recent community experiments.

Openvons (open-Jev): 有限選択肢に確率で答える判断層 — テキスト / 画像 / 日本語音声コマンド.

An experimental JEV-powered framework for forecasting short-term stock price direction from structured market data.

Stop guessing confidence thresholds: calibrate, threshold, and drift-check typed decision models (TypeSafe Jev) against an LLM teacher.

JEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.

Reproducible paper-trading arena that gives Eikos and Jev identical market states and typed questions, then compares equity and probability forecasts.

Inspired by TypeSafe Ai, Ask a local LLM typed questions, get calibrated probabilities instead of text. Structured output without generation or parsing. MLX / Apple Silicon.

Reproduction kit for a 38-benchmark typed-decision index with frozen public-source suites, resumable runners, official scorers, and Jev-compatible HTTP engines.

LegalForecast-MTD benchmark alpha and official evaluation workflows.

Reproducible early-access evaluation of Jev on Korean understanding and medical text, with runtime and cost evidence.

A word-level language model whose output layer is Jev: n-gram drafter, Noul chunk verification, bits-per-token eval.

Daf-jev: composable Python toolkit for TypeSafe's Jev (System One) decision API — question builders, confidence gates, evaluator, calibration, CLI, MCP server, agent skill.

A show-and-tell capability study for Jev, TypeSafe's System One decision model.

Reproducible evaluation of Jev calibration on synthetic support tickets and public classification benchmarks.

Does a TypeSafe Jev rerank beat embedding search? Graded relevance eval (9,831 pairs, 164 zh/en queries) over the Agent Skills Hub catalog, with the judge-circularity bias measured.

A chatbot from typed Jev decisions: hierarchical speculative decoding over System One probabilities.

High-throughput synthetic & pretraining dataset sifter powered by TypeSafe AI Jev (api.typesafe.ai). Stream, filter, and score Parquet & JSONL datasets at 1,500+ rows/sec using System One typed decisions (Choice, Score, Noul).

Decision harness for TypeSafe Jev — confidence gates, shadow mode, recipes, and evals. Claude CLI 48.9s → Jev 1.3s on the same row-filter job.

Open, local reimplementation of a Jev-style decision layer for job postings.

Reproducible calibration and selective-risk benchmarks for Jev/TypeSafe decisions in DSPy workflows

Independent Jev 1.13.0 behavior study: report, controlled prompt experiments, raw results, and offline verification.

Open-source Jev-style System One decision model. Gemma 3 270M with a scoring head — fast, calibrated decisions in a single forward pass. No text generation. Inspired by TypeSafe.ai's Jev.

Can a decision model beat dedicated rerankers? TypeSafe Jev vs Cohere Rerank 4 vs ZeroEntropy zerank-2 vs a chat-model baseline: 14 datasets, every raw API response, bootstrap ranges on every gap.

A chatbot built on a model that cannot generate text (TypeSafe AI's Jev, driven autoregressively).

Calibration and confidence-based routing measured on Banking77: 80.2% accuracy at $0.103 per 500 decisions.

Eight minimal working examples of TypeSafe's Jev (a System One model) applied to mechanical and electrical engineering: CAD/CAE/CAM routing, FEM result triage, DFM screening, BOM alignment, hallucination-proof extraction. Zero dependencies.

Blind security benchmarks for Jev, TypeSafe's System One model: prompt injection and vulnerable code detection, built on jev-go.

Reproducible Jev Ultrafast research-browser eval harness + field note (QC’d cases, suite runner, report generator). Not investment advice.

Backtest Jev (TypeSafe) as a BUY/SELL/HOLD trader on NQ L10 order-book data.

Reproducible experiments on Jev probability calibration, uncertainty, and forecast preservation.

Discriminative Monte Carlo Tree Search using TypeSafe Jev System One Primitives and Gemini.

Not every coding task needs your best model. Experimental Jev-powered model routing for Claude Code — V3 prototype runs today, V4 routes at the task boundary.

Jev (TypeSafe) exploratory thread: claim audit, live demos, and runnable code.

Benchmarks and a playground for TypeSafe's Jev (System One) model: chess, and who-is-the-player-talking-to for speech-to-text game NPCs.

Jev (TypeSafe) vs Claude Haiku 4.5 on 2 000 phishing emails: accuracy, calibration, latency, cost. Reproducible benchmark.

Reproducible Jev research manuscript, evidence, and paper package.

A Jev-inspired decision interface for existing LLMs. Explicit choices, scores, calibration, and review thresholds.

I tortured Jev into being a RISC-V CPU.

An observable raw-character chat experiment powered entirely by TypeSafe Jev Choice.

Jev-shaped typed-decision models with measured latency, accuracy, calibration, and training cost.

Jev (TypeSafe System One) × ASReview SYNERGY abstract screening demo — Choice/Noul vs gold labels.

Drop-in /v1/systemone shadow proxy that forwards requests to Jev and local decision models, recording agreement, calibration, latency, cost, and policy-level differences.

Jev-style calibrated decision model (Choice/Score/Noul) on Qwen3.5-0.8B.

AI benchmark on Japan's 2026 Common Test: Jev vs luna-none vs luna-low (static dashboard).

Benchmarking Jev (Typesafe.ai) against a strong LLM on the Who&When Pro agent-failure-attribution benchmark (text subset).

Benchmarking TypeSafe's Jev decision model as a cost-efficient LLM router on RouterArena.

Experiments with openjev, an open Jev-style option-logit runner, on local models.

Typed-decision benchmark from PadFlow (land development SaaS): schemas, anonymized labeled rows, and a runner for confidence-calibrated models like TypeSafe Jev.

An evaluation of typesafe AI chess. As it turns out, the AI isn't doing really well even though chess is not a particularly open-ended game. Still, it's only a prototype and this probably wasn't optimzied for games.

Zero-shot spam filtering with TypeSafe Jev Noul questions, compared with TF-IDF baselines.

A small second eval for shadcn-ui/lint that uses TypeSafe's Jev to judge the linter's own output.

Rust port of TypeSafe system-one-adapter (LLM-backed system_one evaluations).

Position paper: the Hidden-Markov and fuzzy primitives missing from TypeSafe AI's Jev and System-One decision models. Two lemmas, one principle (Deferred Crispification), one architecture (BSF-S1).

Does Jev predict stock returns from news? It reads the news well; there is no tradeable alpha. Three arms separate reading from recall.

Can Jev pick the winner of a real headline A/B test? 64.5% across 10,984 Upworthy randomized experiments, 74.7% when the difference was decisive.

发明 RLHF 的人，这次做了个不会说话的模型：Jev 独立研究报告。52 页 PDF + 50 条中文实测复现包 + 143 条可回溯数据表.

Benchmarks Jev against other evaluation models in games with explicit states, legal actions, and measurable outcomes.

Typesafe.ai model jev finance benchmark.

Independent benchmark data for TypeSafe's Jev (System One model) vs LLMs: accuracy, calibration, cost. Boards + per-decision logs, CC-BY-4.0

Independent API-only calibration audit of TypeSafe AI's Jev decision model

Research workspace containing reproducible TypeSafe Jev evaluation and safety-judgment experiments.

Jev (TypeSafe) vs. Gemini 3.8 Flash vs. GPT-5.6 Luna na anotação estruturada de sentenças do TJSP: qualidade, tempo e custo.

Reproducible review-classification comparison measuring Jev and Luna accuracy, latency, and cost.

TypeScript experiments, evaluations, and latency benchmarks for TypeSafe's Jev model.

Benchmark of Jev as a malicious agent-skill detector on MalSkillBench with verify-and-escalate evaluation.

Ongoing Japanese research deck on Jev and System One models, maintained as Markdown slides.

Evaluating TypeSafe's Jev as a fast monitor and action gate for agent sabotage in SHADE-Arena, compared with Gemini 2.5 Flash/Pro.

Research toolkit applying conformal risk control and prediction-powered inference to certify Jev routing thresholds.

On-device iPhone visual decision tool using MLX and Qwen3-VL direct option logits.

TypeSafe's Jev vs gpt-5.4-mini and gpt-5.6-luna on four public classification sets: cases, per-item answers, scoring, charts

Reproducible evaluation of Jev Choice decisions on 182 valid ENEM 2025 questions, with raw results, accuracy, calibration, and latency analysis.

Reproducible guardrail benchmark that compares Jev and other decision systems using typed Noul checks and a fixed 0.5 blocking threshold, with public scoring contracts and author-run results.

Jev (TypeSafe) 性能評価プロジェクト — 日本郵便 KEN_ALL をマスタに、AI SDK 経由の Jev が住所のあいまい一致にどこまで使えるかを検証.

Application of TypeSafe Jev (noul judgment primitive) on the collusion.wiki corpus: agent vs human page authorship, head-to-head vs local Qwen3.8-Flash-Next.

Measures how well TypeSafe's RLCD-Jev model spots real secret credentials in file snippets.

Using Jev to test how well it predicts financial markets(just like most llms as of september 2026, it doesnt do that good).

Evaluating TypeSafe's System One primitives (Choice/Score/Noul) — where a typed oracle beats an LLM call.

A small reproducible MuJoCo pilot comparing Jev, Claude Haiku, and reactive rules for pick-and-place.

Charts: TypeSafe Jev evaluated on Thai standardized exams vs 110 other models.

Adversarial Jev evaluation suite with preregistered predictions, request logs, and reproducible experiment artifacts.

Does ORDER BY over a Jev probability put rows in a defensible order? Independent ranking, calibration and invariant measurements of TypeSafe AI's Jev: passes six pre-registered gates on 360 labeled rows, fails four of six on graded product relevance.