https://yangyang233333.github.io/tags/alltoall/ Recent content in AllToAll on Hellokitty's Blog Hugo zh-CN Wed, 02 Sep 2026 11:29:42 +0800 https://yangyang233333.github.io/posts/distributed-training-collective-communication/ Wed, 02 Sep 2026 11:29:42 +0800 https://yangyang233333.github.io/posts/distributed-training-collective-communication/ 从数据流、张量形状、通信量和训练场景出发，系统解释 Broadcast、Reduce、Gather、Scatter、AllReduce、AllGather、ReduceScatter 与 AllToAll，并给出可运行的 torch.distributed 示例。