1/
🧵diving deeper into our new work on zkGPT: Proving LLM inference fast with Zero-Knowledge Proofs.
Why? Service providers might deploy a smaller/cheaper model than promised. ZK lets them prove correctness without revealing model parameters.
đź“„
2/
The Problem:
- LLMs = powerful but costly.
- Providers could cheat by running smaller models.
- Users can’t verify which model was used.
ZK Proofs solve this, but current zkML systems choke on real LLMs:
- No support for transformer architectures.
- Huge proving times (minutes→hours).
3/
Previous work:
- ZKML (Eurosys’24): General ML verification framework. Good for small models, but too slow for LLMs.
- Hao et al. (USENIX Security’24): Early zkLLM attempt, still pretty slow (thousands of seconds).
- Both suffer from massive nonlinear layer overhead + poor parallelization.
4/
Our contributions:
1. Efficient proofs for linear & nonlinear layers tailored to LLMs (e.g., GPT-2).
2. Constraint fusion → reduce overhead in nonlinear layers (like GeLU).
3. Circuit squeeze → boosts parallelism in proof generation.
4. Full-stack implementation optimized for transformer blocks.
5/
Results:
- Proves GPT-2 inference in <25 seconds.
- 279Ă— faster than Hao et al. (USENIX'24).
- 185Ă— faster than ZKML (Eurosys'24).
- Orders-of-magnitude less overhead than naive zk-transformer implementations.
6/
Why it matters:
- Enables practical zkLLM deployment — you can now verify an LLM’s output in seconds.
- Keeps model weights secret.
- Opens doors for privacy-preserving AI services with cryptographic auditability.
7/
Key insight:
Don’t just naively compile an LLM into a circuit.
Exploit structure:
- Linear ops (MatMul, LayerNorm) → custom efficient constraints.
- Nonlinear ops (GELU) → fused constraints to slash complexity.
- Parallel-friendly layout to max out modern prover hardware.
24.99K
554
The content on this page is provided by third parties. Unless otherwise stated, OKX is not the author of the cited article(s) and does not claim any copyright in the materials. The content is provided for informational purposes only and does not represent the views of OKX. It is not intended to be an endorsement of any kind and should not be considered investment advice or a solicitation to buy or sell digital assets. To the extent generative AI is utilized to provide summaries or other information, such AI generated content may be inaccurate or inconsistent. Please read the linked article for more details and information. OKX is not responsible for content hosted on third party sites. Digital asset holdings, including stablecoins and NFTs, involve a high degree of risk and can fluctuate greatly. You should carefully consider whether trading or holding digital assets is suitable for you in light of your financial condition.