跳转到主要内容
LLM 推理

共 34 篇相关文章

相关标签：量化PrefillvLLMKV CacheAttention

Hyperloom Specialist 与配置调优：一个 Agentic 调参系统的源码拆解

ai-systems / llm-inference

llm-inferencegpu-optimizationagentichyperparameter-tuning +4

vLLM Async Scheduling：三态配置、投机解码与状态提交

ai-systems / llm-inference

llm-inferencevllmdecodekv-cache +1

Kimi K3：架构、训练与推理系统研究

ai-systems / llm-inference

llm-inferenceattentionmoedistributed-training +1

推理并行：DP、TP、PP、EP 与 CP 怎么选

ai-systems / llm-inference

llm-inferenceparallelismtensor-parallelpipeline-parallel +2

推理 Kernel / Runtime 优化：少搬、少启、少等

ai-systems / llm-inference

llm-inferencegpu-kernelflashattentionflashdecode +2

KV Cache Hit Ratio 修正模型：从直觉到统一公式

ai-systems / llm-inference

llm-inferencekv-cachesimulatorprefill +1

MoE 推理：Expert Parallelism（EP）、显存与调度

ai-systems / llm-inference

moeexpert-parallelisminference

模拟器建模指南：显存与吞吐公式

ai-systems / llm-inference

simulatormemory-modelinginference

Token Flow 与 Hidden State：从 Attention 到 LM Head

ai-systems / llm-inference

llm-inferencetransformerhidden-stateattention +2

LLM 推理系统全栈地图

ai-systems / llm-inference

llm-inferencesynthesissystemsoptimization

KV Cache：推理性能的命根子

ai-systems / llm-inference

LLMInferenceKV CachePagedAttention +2

量化：从 Scale 到 W4A8 的完整坐标

ai-systems / llm-inference

llm-inferencequantizationprefilldecode +1

从 AVX/AMX 到 Tensor Core：CPU 工程师理解 LLM 量化

ai-systems / llm-inference

llm-inferencequantizationcpugpu

FP4/FP8 量化：值域、Scale 与运行时合同

ai-systems / llm-inference

llm-inferencequantizationgpu

GLM-5.2 量化执行图：W4A8、BMM 与 KV8

ai-systems / llm-inference

llm-inferencequantizationfp4fp8 +2

量化方法与评测：从 PTQ/QAT 到可复现实验

ai-systems / llm-inference

llm-inferencequantizationprofilingresearch

Attention 架构演化：从多头注意力（MHA）到 GQA、MLA

ai-systems / llm-inference

llm-inferenceattentionkv-cache

从 Profiling 到 Simulation：推理性能分析的证据链方法

ai-systems / profiling

profilingsimulationllm-inferenceperformance-analysis +1

Prefill Trace：Worker 供给、DSA/MLA 与 Chunked Prefill

ai-systems / llm-inference

llm-inferenceprefilltracemla +2

GDN 与 Chunked Prefill：为什么 prepare_chunk_indices 会出现在 trace 里

ai-systems / llm-inference

llm-inferencegdnqwen3nextchunked-prefill +3

DeepSeek MLA：低秩 KV Cache 与推理效率

ai-systems / llm-inference

llm-inferenceattentionkv-cachedeepseek +1

Chunked Prefill 深入分析：调度、Chunk Size 与 Attention 形状

ai-systems / llm-inference

llm-inferencechunked-prefillschedulingprefill +3

DeepSeek-V3 Technical Report：中英对照解读

ai-systems / llm-inference

LLMDeepSeekMoEFP8 +2

DSpark 与 MTP：DeepSeek 投机解码调研

ai-systems / llm-inference

llm-inferencespeculative-decodingdeepseekdspark +1

Causal Attention：为什么 KV hit 后 Attention 按 1 - h² 缩放

ai-systems / llm-inference

llm-inferenceattentionkv-cachesimulator +1

Agentic Infra：LLM 推理性能优化与 GPU 利用率提升

ai-systems / llm-inference

llm-inferencegpu-optimizationprofilingawp +4

Agentic AWP：规模化 Profiling 驱动的 GPU 效率 Breakdown 与能力体系

ai-systems / profiling

awpgpu-profilingbreakdowngpu-efficiency +3

01. 什么是 AI 推理

ai-systems / reasoning

AILLMReasoning

02. Reasoning Model、Agent 与长任务

ai-systems / reasoning

AILLMReasoningAgent

03. RAG、Memory、Fine-tuning 与 Distillation

ai-systems / reasoning

AILLMRAGFine-Tuning +1

CUDA Agent

ai-systems / gpu-computing

GPUCUDARLLLM +1

Compute-bound vs Memory-bound：推理的两大瓶颈

ai-systems / llm-inference

LLMInferencePerformanceGPU +3

批处理与调度：推理服务的灵魂

ai-systems / llm-inference

LLMInferenceBatchingScheduling +3

投机解码：突破 decode 一次只出一个 token 的限制

ai-systems / llm-inference

LLMInferenceSpeculative DecodingEAGLE +2