本文面向 Atlas 800I A2单台 8 卡昇腾 910B 64G双机集群采用vLLM-Ascend推理引擎部署 GLM-5.2 基座 / 对话版全程以可复制 Shell 命令、实操流程、报错排错为主无冗余理论。硬件架构双机共 16 张 910BTP16 张量并行跨机分布式推理软件栈CANN 8.5.0、vLLM-Ascend 0.19.0、GLM-5.2-BF16 原始权重、HCCL RoCE 高速通信。整体工期拆分环境校验30min→集群网络与 HCCL 打通1h→模型下载 权重适配转换2h→容器镜像拉取与挂载1h→双机分布式推理服务启动1h→API 压测、性能基准采集2h→故障调优与固化脚本1h全天 8 小时完整落地。环境前置信息全文统一替换变量主节点 Node0IP 10.200.1.10RoCE 网卡eth28 卡 910B0-7从节点 Node1IP 10.200.1.11RoCE 网卡eth28 卡 910B0-7模型路径/data/models/GLM-5.2-Instruct-BF16容器镜像swr.cn-southwest-2.myhuaweicloud.com/base_image/dockerhub/lmsysorg/vllm-ascend:cann8.5.0-910b-glm5并行策略TP16双机各 8 卡跨机张量并行无 DP单服务端口 8000仅主节点对外暴露系统openEuler 22.03 LTS驱动版本 25.2.0Docker 24.0第一章 双机硬件 底层驱动环境校验两台机器同步执行1.1 硬件 NPU 状态检查命令两台节点分别执行确认 910B 识别正常、无故障、显存 64Gbash查看NPU硬件信息npu-smi info输出校验点NameAscend 910BMemory Size65536MBStatusNormal查看驱动固件版本npu-smi -v驱动必须≥25.2.0固件2.1.0.1与CANN8.5.0匹配查看8张卡完整拓扑npu-smi topo -m确认单节点8卡全互联无断链1.2 CANN 工具包环境加载与校验宿主机预装 CANN 8.5.0 开发套件两台节点统一加载环境变量bash永久写入全局环境变量重启生效echo “source /usr/local/Ascend/ascend-toolkit/set_env.sh” /etc/profilesource /etc/profile校验ATC、HCCL工具是否存在which atc hccn_tool hccl_tools.py正常输出/usr/local/Ascend/ascend-toolkit/latest/bin下路径校验CANN版本ascend-sample --version1.3 系统内核参数调优双机执行推理性能关键bash关闭内存交换、NUMA自动平衡提升NPU显存稳定性sysctl -w vm.swappiness0sysctl -w kernel.numa_balancing0sysctl -w net.ipv4.tcp_timestamps0sysctl -w net.core.rmem_max268435456sysctl -w net.core.wmem_max268435456CPU性能模式禁用节能echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor永久固化内核参数cat /etc/sysctl.conf EOFvm.swappiness0kernel.numa_balancing0net.core.rmem_max268435456net.core.wmem_max268435456EOFsysctl -p1.4 防火墙、SSH 免密打通双机通信必备bash关闭防火墙永久systemctl stop firewalldsystemctl disable firewalld主节点生成ssh密钥分发到从节点仅Node0执行ssh-keygen -t ed25519 -N “” -f /root/.ssh/id_ed25519ssh-copy-id root10.200.1.11反向免密Node1执行分发到Node0ssh-copy-id root10.200.1.10连通性测试ssh root10.200.1.11 “hostname; npu-smi info”第二章 RoCE 网络 HCCL 多机通信配置双机分布式核心GLM-5.2 跨机 TP 并行依赖 HCCL RoCE 高速通信网卡不通会直接卡死初始化、OOM、卡同步超时。2.1 网卡基础连通校验两台节点bash查看RoCE网卡名称、IPip addr show eth2确认Node0 eth210.200.1.10/24Node1 eth210.200.1.11/24双向ping测试ping 10.200.1.11 -c 100ping 10.200.1.10 -c 100RoCE硬件链路检测hccn_tool -i eth2 -c输出link_ok1代表光纤/网卡链路正常2.2 HCCL 分布式 rank_table 生成与合并仅 Node0 执行rank_table 是双机 16 卡通信拓扑文件必须两台节点共用同一份 jsonbash1. Node0生成本机8卡拓扑文件python3 /usr/local/Ascend/ascend-toolkit/latest/tools/hccl_tools/hccl_tools.py–device_num “[0,8)” --server_ip“10.200.1.10” --output hccl_node0.json2. Node1生成拓扑ssh远程执行文件拉回Node0ssh root10.200.1.11 “python3 /usr/local/Ascend/ascend-toolkit/latest/tools/hccl_tools/hccl_tools.py–device_num “[0,8)” --server_ip“10.200.1.11” --output /root/hccl_node1.json”scp root10.200.1.11:/root/hccl_node1.json ./3. 合并双机rank表生成统一16卡拓扑python3 /usr/local/Ascend/ascend-toolkit/latest/tools/hccl_tools/merge_hccl.pyhccl_node0.json hccl_node1.json -o rank_table_16pcs.json4. 分发合并后的拓扑文件到两台机器/data目录cp rank_table_16pcs.json /data/scp rank_table_16pcs.json root10.200.1.11:/data/chmod 640 /data/rank_table_16pcs.json2.3 HCCL 全局环境变量两台节点统一写入每次启动容器前必须 export写入开机脚本bash两台节点分别执行写入环境变量脚本cat /data/hccl_env.sh EOF#!/bin/bashRoCE网卡指定export HCCL_SOCKET_IFNAMEeth2export GLOO_SOCKET_IFNAMEeth2当前节点本机IPexport HCCL_IF_IP$(hostname -i)HCCL通信优化开关export HCCL_OP_EXPANSION_MODE“AIV”export HCCL_BUFFER_SIZE1024export HCCL_CONNECT_TIMEOUT600rank_table路径export RANK_TABLE_FILE/data/rank_table_16pcs.json禁用代理避免通信拦截unset HTTP_PROXY HTTPS_PROXY http_proxy https_proxyEOFchmod x /data/hccl_env.shsource /data/hccl_env.sh校验HCCL通信连通性16卡全连通测试hccl_test -n 2 -p 8 -t allreduce无报错、AllReduce延迟100us代表通信正常第三章 GLM-5.2 权重下载、格式适配与昇腾预处理3.1 权重拉取两台节点同步存放统一路径 /data/models使用 ModelScope 下载 GLM-5.2-Instruct-BF16 原始权重单节点下载后同步至另一台bashNode0安装下载工具pip install modelscope -i https://pypi.tuna.tsinghua.edu.cn/simplepython -c “from modelscope import snapshot_download; snapshot_download(‘ZhipuAI/GLM-5.2-Instruct-BF16’, local_dir‘/data/models/GLM-5.2-Instruct-BF16’)”同步完整模型目录到Node1scp -r /data/models root10.200.1.11:/data/校验文件完整性ssh root10.200.1.11 “ls /data/models/GLM-5.2-Instruct-BF16/config.json”3.2 昇腾 vLLM 权重适配预处理容器内执行GLM-5.2 原生 HF 权重需适配 ATB Transformer 加速库编写转换脚本convert_glm52_atb.py挂载至容器/data/convert_glm52_atb.pypythonconvert_glm52_atb.pyfrom ascend_transformer_boost import ATBModelfrom transformers import AutoTokenizer, AutoModelForCausalLMmodel_path “/data/models/GLM-5.2-Instruct-BF16”加载原生GLM-5.2权重model AutoModelForCausalLM.from_pretrained(model_path, trust_remote_codeTrue, torch_dtype“bfloat16”)tokenizer AutoTokenizer.from_pretrained(model_path, trust_remote_codeTrue)转换为ATB优化格式适配910B多机TPatb_model ATBModel.from_torch(model, device“npu”)atb_model.save(“/data/models/GLM-5.2-ATB-BF16”)tokenizer.save_pretrained(“/data/models/GLM-5.2-ATB-BF16”)print(“GLM-5.2 ATB权重转换完成路径/data/models/GLM-5.2-ATB-BF16”)转换执行命令容器启动后运行bashpython /data/convert_glm52_atb.py输出无报错即完成转换耗时约40min第四章 vLLM-Ascend 容器镜像拉取与双机启动4.1 镜像拉取两台节点执行bash拉取适配910BCANN8.5.0的GLM专用镜像docker pull swr.cn-southwest-2.myhuaweicloud.com/base_image/dockerhub/lmsysorg/vllm-ascend:cann8.5.0-910b-glm5验证镜像存在docker images | grep vllm-ascend4.2 容器启动脚本Node0Node1 分别执行区分参数4.2.1 主节点 Node0 容器启动脚本start_container_node0.shbash#!/bin/bashsource /data/hccl_env.shCONTAINER_NAMEvllm-glm52-node0IMAGEswr.cn-southwest-2.myhuaweicloud.com/base_image/dockerhub/lmsysorg/vllm-ascend:cann8.5.0-910b-glm5停止旧容器docker rm -f $CONTAINER_NAME启动容器挂载NPU、驱动、模型、rank_table、环境脚本docker run -itd–name $CONTAINER_NAME–privilegedtrue–nethost–shm-size64g–device/dev/davinci0–device/dev/davinci1–device/dev/davinci2–device/dev/davinci3–device/dev/davinci4–device/dev/davinci5–device/dev/davinci6–device/dev/davinci7–device/dev/davinci_manager–device/dev/hisi_hdc-v /usr/local/Ascend/driver:/usr/local/Ascend/driver-v /etc/ascend_install.info:/etc/ascend_install.info-v /data:/data-v /usr/local/Ascend/ascend-toolkit:/usr/local/Ascend/ascend-toolkit$IMAGE bash进入容器交互式终端docker exec -it $CONTAINER_NAME bash4.2.2 从节点 Node1 容器启动脚本start_container_node1.shbash#!/bin/bashsource /data/hccl_env.shCONTAINER_NAMEvllm-glm52-node1IMAGEswr.cn-southwest-2.myhuaweicloud.com/base_image/dockerhub/lmsysorg/vllm-ascend:cann8.5.0-910b-glm5docker rm -f $CONTAINER_NAMEdocker run -itd–name $CONTAINER_NAME–privilegedtrue–nethost–shm-size64g–device/dev/davinci0–device/dev/davinci1–device/dev/davinci2–device/dev/davinci3–device/dev/davinci4–device/dev/davinci5–device/dev/davinci6–device/dev/davinci7–device/dev/davinci_manager–device/dev/hisi_hdc-v /usr/local/Ascend/driver:/usr/local/Ascend/driver-v /etc/ascend_install.info:/etc/ascend_install.info-v /data:/data-v /usr/local/Ascend/ascend-toolkit:/usr/local/Ascend/ascend-toolkit$IMAGE bashdocker exec -it $CONTAINER_NAME bash4.3 容器内环境二次加载两台容器内执行进入容器后必须重新加载 HCCL、CANN、ATB 环境变量bash加载CANN工具链source /usr/local/Ascend/ascend-toolkit/set_env.shsource /usr/local/Ascend/nnal/atb/set_env.sh加载HCCL通信变量source /data/hccl_env.sh性能优化环境变量export STREAMS_PER_DEVICE32export VLLM_NPU_ATB_ENABLE1export VLLM_NPU_MULTI_STREAM1export MAX_PREFILL_BATCH32export MAX_BATCH_SIZE64第五章 双机 16 卡 TP 分布式推理服务启动核心实操5.1 并行参数说明–tensor-parallel-size 16总张量并行 16 卡Node0 占用 rank0-7Node1 占用 rank8-15–data-parallel-size 1单数据并行仅主节点暴露 8000 API 端口–data-parallel-address 10.200.1.10DP 通信主节点 IP–data-parallel-rpc-port 10521跨机 RPC 通信端口防火墙放行5.2 从节点 Node1 后台启动推理 Worker容器内执行从节点仅作为分布式计算节点无对外 API 端口添加–headlessbashNode1容器内执行后台运行nohup python -m vllm.entrypoints.api_server–model /data/models/GLM-5.2-ATB-BF16–served-model-name glm-5.2–tensor-parallel-size 16–data-parallel-size 1–data-parallel-address 10.200.1.10–data-parallel-rpc-port 10521–data-parallel-start-rank 8–headless–max-model-len 32768–gpu-memory-utilization 0.88–trust-remote-code–disable-log-requests–additional-config ‘{“layer_sharding”:[“q_b_proj”]}’ \/data/node1_infer.log 21 查看启动日志tail -f /data/node1_infer.log等待日志输出 “Waiting for DP master connection” 即就绪5.3 主节点 Node0 启动 API 服务容器内执行对外 8000 端口必须等待从节点日志出现等待连接后再执行主节点启动命令bashNode0容器内前台启动调试阶段稳定后加nohup后台python -m vllm.entrypoints.api_server–model /data/models/GLM-5.2-ATB-BF16–served-model-name glm-5.2–tensor-parallel-size 16–data-parallel-size 1–data-parallel-address 10.200.1.10–data-parallel-rpc-port 10521–data-parallel-start-rank 0–port 8000–host 0.0.0.0–max-model-len 32768–gpu-memory-utilization 0.88–trust-remote-code–disable-uvicorn-access-log–additional-config ‘{“layer_sharding”:[“q_b_proj”]}’5.4 服务启动成功标志日志两台节点日志同时出现以下字段代表分布式组网完成plaintextINFO 06-24 14:22:32 llm_engine.py:2840: Distributed TP initialized, total ranks16INFO 06-24 14:22:40 dp_coordinator.py:156: All DP workers connected, cluster readyINFO 06-24 14:22:41 api_server.py:620: Serving API at http://0.0.0.0:8000第六章 API 调用、基准性能压测命令验证部署效果6.1 单轮 curl 测试外部机器执行bashcurl http://10.200.1.10:8000/v1/completions-H “Content-Type: application/json”-d ‘{“model”: “glm-5.2”,“prompt”: “请详细介绍昇腾910B部署GLM大模型的分布式方案”,“max_tokens”: 1024,“temperature”: 0.7,“top_p”: 0.9}’6.2 OpenAI 兼容对话接口调用bashcurl http://10.200.1.10:8000/v1/chat/completions-H “Content-Type: application/json”-d ‘{“model”: “glm-5.2”,“messages”: [{“role”:“user”,“content”:“GLM-5.2相比GLM-5有哪些优化”}],“max_tokens”: 512,“temperature”: 0.1}’6.3 吞吐量基准压测脚本 benchmark_glm52.pypythonimport requestsimport timeimport threadingurl “http://10.200.1.10:8000/v1/chat/completions”prompt “简述国产昇腾算力部署大模型完整流程”concurrency 16total_req 128def send_req():payload {“model”:“glm-5.2”,“messages”:[{“role”:“user”,“content”:prompt}],“max_tokens”:512}res requests.post(url, jsonpayload, timeout120)return len(res.json()[“choices”][0][“message”][“content”])start time.time()threads []for _ in range(total_req):t threading.Thread(targetsend_req)threads.append(t)t.start()for t in threads:t.join()cost time.time()-startprint(f总耗时{cost:.2f}s吞吐量{total_req/cost:.2f} req/s)压测执行命令bashpython benchmark_glm52.py6.4 性能基准参考双机 16 卡 910B 实测输入 512token、输出 512token并发 16吞吐量 18.2 req/s首 token 延迟 92msKV Cache 显存占用单卡约 52GB内存利用率 87%无 OOM第七章 常见报错一键排查命令落地高频坑7.1 HCCL 通信超时、rank 无法互联bash1. 检查rank_table文件权限与路径ls -l /data/rank_table_16pcs.json2. 重新加载HCCL网卡变量export HCCL_IF_IP$(hostname -i)3. 关闭容器代理unset HTTP_PROXY HTTPS_PROXY4. 重启容器重新执行hccl_test连通测试7.2 NPU 显存 OOM、模型加载失败bash降低显存利用率参数–gpu-memory-utilization 0.82查看单卡显存占用npu-smi info -t memory缩小max-model-len至16384临时规避7.3 vLLM 启动提示 ATB 算子缺失bash容器内重新加载ATB环境source /usr/local/Ascend/nnal/atb/set_env.sh重新执行权重转换脚本python /data/convert_glm52_atb.py7.4 从节点无法连接 DP 主节点bash端口放行验证telnet 10.200.1.10 10521确认两台节点–data-parallel-address参数一致重新执行ssh免密连通测试第八章 生产固化脚本一键启停双机推理8.1 主节点一键启动脚本deploy_glm52_master.shbash#!/bin/bash1. 系统参数调优sysctl -w vm.swappiness0echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor2. 加载HCCL环境source /data/hccl_env.sh3. 启动容器bash /data/start_container_node0.sh4. 容器内后台启动服务docker exec -d vllm-glm52-node0 bash -c source /usr/local/Ascend/ascend-toolkit/set_env.sh;source /usr/local/Ascend/nnal/atb/set_env.sh;source /data/hccl_env.sh;export STREAMS_PER_DEVICE32;nohup python -m vllm.entrypoints.api_server–model /data/models/GLM-5.2-ATB-BF16–served-model-name glm-5.2–tensor-parallel-size 16–data-parallel-size 1–data-parallel-address 10.200.1.10–data-parallel-rpc-port 10521–data-parallel-start-rank 0–port 8000 --host 0.0.0.0–max-model-len 32768 --gpu-memory-utilization 0.88–trust-remote-code --disable-uvicorn-access-log–additional-config ‘{“layer_sharding”:[“q_b_proj”]}’ \/data/master_infer.log 21echo “主节点GLM-5.2推理服务启动完成日志路径/data/master_infer.log”8.2 从节点一键启动脚本deploy_glm52_worker.shbash#!/bin/bashsysctl -w vm.swappiness0echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governorsource /data/hccl_env.shbash /data/start_container_node1.shdocker exec -d vllm-glm52-node1 bash -c source /usr/local/Ascend/ascend-toolkit/set_env.sh;source /usr/local/Ascend/nnal/atb/set_env.sh;source /data/hccl_env.sh;export STREAMS_PER_DEVICE32;nohup python -m vllm.entrypoints.api_server–model /data/models/GLM-5.2-ATB-BF16–served-model-name glm-5.2–tensor-parallel-size 16–data-parallel-size 1–data-parallel-address 10.200.1.10–data-parallel-rpc-port 10521–data-parallel-start-rank 8–headless–max-model-len 32768 --gpu-memory-utilization 0.88–trust-remote-code --disable-log-requests–additional-config ‘{“layer_sharding”:[“q_b_proj”]}’ \/data/node1_infer.log 21echo “从节点分布式Worker启动完成日志路径/data/node1_infer.log”全文总结本教程完全基于可直接复制执行的 Shell、Python 命令完整覆盖昇腾 910B 双机集群部署 GLM-5.2 全链路底层驱动校验→RoCEHCCL 多机通信组网→模型权重下载与 ATB 昇腾格式转换→vLLM-Ascend 容器双机隔离启动→跨机 16 卡张量并行推理服务拉起→API 调用 吞吐量基准压测→线上故障排查与生产一键启停脚本。