You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
86 lines
6.0 KiB
86 lines
6.0 KiB
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="840" viewBox="0 0 1200 840" role="img" aria-labelledby="title desc">
|
|
<title id="title">FreeToken Decode 阶段的命中、缓存填充与 CPU 执行</title>
|
|
<desc id="desc">12 个去重路由专家中 8 个命中 GPU 缓存，4 个未命中。根据 4060 的测量带宽，一个未命中专家进入 GPU，三个由 CPU 计算，结果精确相加。</desc>
|
|
<defs>
|
|
<marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto">
|
|
<path d="M0,0 L0,6 L9,3 z" fill="#53606b"/>
|
|
</marker>
|
|
<style>
|
|
.t{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI","PingFang SC","Microsoft YaHei",sans-serif;fill:#17212b}
|
|
.muted{font-size:16px;fill:#5d6873}
|
|
.head{font-size:22px;font-weight:700}
|
|
.chip{font-size:15px;font-weight:700;text-anchor:middle}
|
|
.line{stroke:#53606b;stroke-width:3;fill:none;marker-end:url(#arrow)}
|
|
.dash{stroke:#89939c;stroke-width:2;stroke-dasharray:7 7;fill:none}
|
|
</style>
|
|
</defs>
|
|
<rect width="1200" height="840" fill="#f6f8fa"/>
|
|
<text class="t" x="50" y="55" font-size="30" font-weight="700">Decode：缓存命中直接算，4 个 miss 再由 q* 分流</text>
|
|
<text class="t muted" x="50" y="85">示例沿用论文 Figure 2：当前层去重后有 12 个路由专家，其中 8 个命中、4 个未命中</text>
|
|
|
|
<rect x="45" y="120" width="1110" height="108" rx="8" fill="#ffffff" stroke="#c7cdd4" stroke-width="2"/>
|
|
<text class="t head" x="80" y="157">Router 输出</text>
|
|
<g transform="translate(245 142)">
|
|
<g fill="#e7f5ef" stroke="#1f8a70">
|
|
<rect x="0" y="0" width="58" height="48" rx="6"/><rect x="68" y="0" width="58" height="48" rx="6"/>
|
|
<rect x="136" y="0" width="58" height="48" rx="6"/><rect x="204" y="0" width="58" height="48" rx="6"/>
|
|
<rect x="272" y="0" width="58" height="48" rx="6"/><rect x="340" y="0" width="58" height="48" rx="6"/>
|
|
<rect x="408" y="0" width="58" height="48" rx="6"/><rect x="476" y="0" width="58" height="48" rx="6"/>
|
|
</g>
|
|
<g fill="#fdebe8" stroke="#e85d4a">
|
|
<rect x="564" y="0" width="58" height="48" rx="6"/><rect x="632" y="0" width="58" height="48" rx="6"/>
|
|
<rect x="700" y="0" width="58" height="48" rx="6"/><rect x="768" y="0" width="58" height="48" rx="6"/>
|
|
</g>
|
|
<g class="t chip">
|
|
<text x="29" y="30">E3</text><text x="97" y="30">E8</text><text x="165" y="30">E17</text><text x="233" y="30">E42</text>
|
|
<text x="301" y="30">E58</text><text x="369" y="30">E91</text><text x="437" y="30">E103</text><text x="505" y="30">E144</text>
|
|
<text x="593" y="30">E155</text><text x="661" y="30">E188</text><text x="729" y="30">E201</text><text x="797" y="30">E233</text>
|
|
</g>
|
|
</g>
|
|
|
|
<path class="line" d="M600 233 V275"/>
|
|
<rect x="70" y="285" width="480" height="152" rx="8" fill="#ffffff" stroke="#1f8a70" stroke-width="3"/>
|
|
<text class="t head" x="105" y="326">GPU LRU Cache：8 个 hit</text>
|
|
<text class="t muted" x="105" y="355">逻辑 expert id 直接改写成物理 slot id</text>
|
|
<g transform="translate(105 375)" fill="#1f8a70">
|
|
<rect x="0" y="0" width="42" height="34" rx="4"/><rect x="52" y="0" width="42" height="34" rx="4"/>
|
|
<rect x="104" y="0" width="42" height="34" rx="4"/><rect x="156" y="0" width="42" height="34" rx="4"/>
|
|
<rect x="208" y="0" width="42" height="34" rx="4"/><rect x="260" y="0" width="42" height="34" rx="4"/>
|
|
<rect x="312" y="0" width="42" height="34" rx="4"/><rect x="364" y="0" width="42" height="34" rx="4"/>
|
|
</g>
|
|
|
|
<rect x="650" y="285" width="480" height="152" rx="8" fill="#ffffff" stroke="#e85d4a" stroke-width="3"/>
|
|
<text class="t head" x="685" y="326">Host Expert Pool：4 个 miss</text>
|
|
<text class="t muted" x="685" y="355">完整权重都在 RAM，接下来决定在哪里执行</text>
|
|
<g transform="translate(685 375)">
|
|
<rect x="0" y="0" width="72" height="34" rx="4" fill="#e85d4a"/><rect x="86" y="0" width="72" height="34" rx="4" fill="#e85d4a"/>
|
|
<rect x="172" y="0" width="72" height="34" rx="4" fill="#e85d4a"/><rect x="258" y="0" width="72" height="34" rx="4" fill="#e85d4a"/>
|
|
<g class="t chip" fill="#ffffff">
|
|
<text x="36" y="23">E155</text><text x="122" y="23">E188</text><text x="208" y="23">E201</text><text x="294" y="23">E233</text>
|
|
</g>
|
|
</g>
|
|
|
|
<path class="line" d="M890 442 V482"/>
|
|
<rect x="705" y="492" width="370" height="78" rx="8" fill="#fff0d6" stroke="#d59224" stroke-width="2"/>
|
|
<text class="t head" x="890" y="523" text-anchor="middle">4060 实测带宽决定 q*</text>
|
|
<text class="t" x="890" y="551" font-size="18" text-anchor="middle">q* ≈ 4 × 11.8 / 47.5 ≈ 1</text>
|
|
|
|
<path class="line" d="M805 575 C735 600 655 600 565 620"/>
|
|
<path class="line" d="M975 575 C1035 600 1040 610 1040 620"/>
|
|
<rect x="320" y="625" width="370" height="105" rx="8" fill="#e8f1fb" stroke="#2979b9" stroke-width="3"/>
|
|
<text class="t head" x="505" y="660" text-anchor="middle">GPU 分支</text>
|
|
<text class="t" x="505" y="690" font-size="18" text-anchor="middle">8 个 hit + 1 个新 fill → grouped GEMM</text>
|
|
<text class="t muted" x="505" y="716" text-anchor="middle">新专家留在 slot，下一 token 可能命中</text>
|
|
|
|
<rect x="825" y="625" width="300" height="105" rx="8" fill="#e7f5ef" stroke="#1f8a70" stroke-width="3"/>
|
|
<text class="t head" x="975" y="660" text-anchor="middle">CPU 分支</text>
|
|
<text class="t" x="975" y="690" font-size="18" text-anchor="middle">其余 3 个 miss 直接计算</text>
|
|
<text class="t muted" x="975" y="716" text-anchor="middle">与 GPU 搬运和计算并发</text>
|
|
|
|
<path class="line" d="M505 735 C505 770 625 780 690 780"/>
|
|
<path class="line" d="M975 735 C975 770 855 780 790 780"/>
|
|
<rect x="700" y="756" width="80" height="50" rx="8" fill="#7559b8"/>
|
|
<text class="t" x="740" y="790" font-size="26" font-weight="700" fill="#ffffff" text-anchor="middle">+</text>
|
|
<rect x="800" y="756" width="325" height="50" rx="8" fill="#eee9fb" stroke="#7559b8" stroke-width="2"/>
|
|
<text class="t" x="962" y="788" font-size="18" font-weight="650" text-anchor="middle">精确合并 CPU / GPU 部分和</text>
|
|
</svg>
|