Written by Luyang Zhao; Translated by Wenwen Dong, Yanjun Hu The previous article The Execution Process of a Tensor in a Deep Learning Framework has introduced the execution mechanism of a Tensor in OneFlow. In this article, we will take output = flow.relu(input) as an example to illustrate how an…

Written by Jinhui Yuan; Translated by Jiali Shen, Yushan Zhang In our previous blog: Combating Software System Complexity: Appropriate Abstraction Layer, we mentioned that the communication in a distributed deep learning framework is highly dependent on regular collective communication operations like all-reduce, reduce-scatter, all-gather, and so on. Therefore, it’s crucial…

Written by Zekang Zheng; Translated by Yanjun Hu, Jiali Shen PReLU is an activation function that is frequently used in InsightFace. It has two operating modes: PReLU(1) and PReLU(channels). For the latter, PReLU is equivalent to a binary broadcast operation. In this article, we are going to talk about optimizing…

Welcome to use OneFlow v0.7.0. We would love to hear your feedback! Highlights This release has the following highlights: Provides a Tensor that can be executed in multi-nodes multi-GPUs scenarios: Global Tensor. It is an easy-to-use solution for distributed execution. It makes it easier to implement various distributed parallel strategies and…

Written by Xiaoyu Zhang; Translated by Xiaozhen Liu, Chenyang Xu, Yakun Zhou This article focuses on what is happening behind the execution of a Tensor in the deep learning framework OneFlow. It takes the operator `oneflow.relu` as an example to introduce the Interpreter and VM mechanisms that need to be…

Written by Xiaoyu Zhang; Translated by Xiaozhen Liu, Chenyang Xu, Yakun Zhou This article introduces OneFlow’s operator AutoTest framework to analyze how OneFlow perform the operator alignment task when we develop operators. This AutoTest Framework can also be easily ported to other deep learning training frameworks. The code is here. 1. Traditional Operator Alignment …

Written by Jing Qiao, Chi, Yao; Translated by Xiaozhen Liu, Chenyang Xu, Yakun Zhou In the previous article, we introduced the basic concept of flow control and reviewed the history of credit-based flow control. In this article, we will focus on its present, analyze the core principle of credit-based flow…

Written by Jing Qiao, Chi, Yao; Translated by Xiaozhen Liu, Chenyang Xu, Yakun Zhou Backpressure mechanism, also known as credit-based flow control, is a classic scheme for network communication flow control problems. Its predecessor is the TCP sliding window. This idea is particularly simple and effective. As we will see…

Written by Xiaoyu Zhang; Translated by Xiaozhen Liu, Chenyang Xu, Yakun Zhou This article describes how OneFlow works with MLIR, how to add a graph-level Pass to OneFlow IR, how OneFlow Operations automatically become MLIR Operations, and why OneFlow IR can use MLIR to accelerate computations. For more details, please…