https://moyutianzun.cn/en/categories/vllm/Recent content in Vllm on moyutianzun's blogHugoenmoyutianzunFri, 26 Sep 2025 09:10:35 +0800https://moyutianzun.cn/en/blog/vllm-v1-yuan-ma-jie-xi------dan-ji-ba-qia/Fri, 26 Sep 2025 09:10:35 +0800https://moyutianzun.cn/en/blog/vllm-v1-yuan-ma-jie-xi------dan-ji-ba-qia/<p style="">单机八卡，我们按照PP + TP的方式来进行方案说明，使用的是vllm框架，主要命令和函数如下：</p><pre><code>python single_node_multi_gpu_demo.py --mode pipeline_parallel --tensor-parallel 4 --pipeline-parallel 2 --model facebook/opt-13b <p>def pipeline_parallel_inference(self, model_name: str, tensor_parallel_size: int, pipeline_parallel_size: int): &ldquo;&ldquo;&ldquo;流水线并行推理 - 将模型层分布到多个GPU上&rdquo;&rdquo;&rdquo; print(f&quot;🚀 启动流水线并行推理 - 模型: {model_name}&quot;) print(f&quot; 张量并行: {tensor_parallel_size}, 流水线并行: {pipeline_parallel_size}&quot;)</p>https://moyutianzun.cn/en/blog/vllm-v1-yuan-ma-jie-xi------core/Tue, 23 Sep 2025 03:56:10 +0800https://moyutianzun.cn/en/blog/vllm-v1-yuan-ma-jie-xi------core/<p style="">一个client建立之后就会建立一个core engine，这些配置会通过QMZ IPC发送给core engine。</p><p style=""></p><h1 style="" id="core-engine-architecture">Core engine Architecture</h1><p style=""></p><p style=""></p><p style=""></p><p style=""></p><p style=""></p><p style=""></p><p style=""></p><p style=""></p><p style=""></p><h2 style="" id="worker-and-executor">Worker and Executor</h2><p style=""></p><h2 style="" id="multiprocexecutor">MultiprocExecutor</h2><p style="">在MultiprocExecutor类中，可以清晰的找到三部曲：</p><p style=""></p><p style="">1、创建RPC消息队列</p><pre><code class="language-python"># Initialize worker and set up message queues for SchedulerOutputs # and ModelRunnerOutputs max_chunk_bytes = envs.VLLM_MQ_MAX_CHUNK_BYTES_MB * 1024 * 1024 self.rpc_broadcast_mq = MessageQueue(self.world_size, self.world_size, max_chunk_bytes=max_chunk_bytes) scheduler_output_handle = self.rpc_broadcast_mq.export_handle()</code></pre><p style=""></p>