CDM (CRYSTAL Depth Metric)

CDM allows the under to see how deep in the basin the LLM fell: we developed… CDM v2 — a drop-in 68-line metric that finally tells you when a transformer is actually reasoning vs regurgitating. Four signals (entropy collapse, convergence ratio, attention Gini, basin-escape probability). Works on every model from DialoGPT to Llama-405B. Zero install issues.

Colab (runs free in 3 mins):

The CRYSTAL Manual Your Personal “Thought Depth” Meter for Any Local LLM (Readable by engineers, weekend tinkerers, and curious humans alike)

pip install -r requirements.txt

Works on:

Google Colab (free T4/A100) Local RTX 4090 / 3090 Mac M1/M2/M3/M4 Any rented RunPod / Vast.ai box

1. What is CDM and Why Should You Care? Right now, when you run a local Llama, Mistral, or Grok-1 model, you have exactly two ways to guess if it is actually thinking or just vomiting cached text:

Perplexity / entropy → “Is this fluent?” Your own eyes → “Does this answer feel smart?” Both are terrible.

CDM (CRYSTAL Depth Metric) is the first drop-in tool that tells you, in one number, how deeply the model had to dig into its layers to produce the current token.

CDM v2 in Plain English (the version you actually want) Four things have to happen at the same layer and stay true for several layers:

Entropy collapses hard (≥2.3 bits drop per layer) Hidden states stop wandering (convergence ratio ≤0.12) Attention becomes laser-focused on the few tokens that matter (Gini ↑) The decision survives real noise (≥88 % same top token when you jiggle the hidden state) If all four → deep CRYSTAL → real thinking. If any fail → shallow basin → cached answer.

Zero-to-CDM in Under 10 Minutes (DIY Edition) Requirements: any decent GPU (RTX 3090 and up, or Mac M2 Ultra, or rented A100)

Bottom Line – Why You Will Never Run a Local LLM Without CDM Again Perplexity told you the model is fluent. Benchmarks told you it memorized the test set. CDM tells you it actually thought.

Install it once and you will immediately see which of your prompts, models, and quantizations are real and which are just expensive parrots.