ztxz16/fastllm

Why this repo matters

• •Zero-dependency C++ library enables high-performance LLM inference on consumer-grade GPUs without complex backend setups.
• •Supports tensor parallelism for dense models and hybrid inference modes specifically optimized for Mixture-of-Experts architectures.
• •Achieves full-precision DeepSeek inference on dual EPYC servers with single GPU at twenty tokens per second.