Publications

2021

  1. arXiv
    Communication-Efficient Separable Neural Network for Distributed Inference on Edge Devices
    Jun-Liang Lin, and Sheng-De Wang

    arXiv preprint arXiv:2111.02489

    PDF arXiv
    The inference of Neural Networks is usually restricted by the resources (e.g., computing power, memory, bandwidth) on edge devices. In addition to improving the hardware design and deploying efficient models, it is possible to aggregate the computing power of many devices to enable the machine learning models. In this paper, we proposed a novel method of exploiting model parallelism to separate a neural network for distributed inferences. To achieve a better balance between communication latency, computation latency, and performance, we adopt neural architecture search (NAS) to search for the best transmission policy and reduce the amount of communication. The best model we found decreases by 86.6% of the amount of data transmission compared to the baseline and does not impact performance much. Under proper specifications of devices and configurations of models, our experiments show that the inference of large neural networks on edge clusters can be distributed and accelerated, which provides a new solution for the deployment of intelligent applications in the internet of things (IoT).
  2. ICIP 2021
    The Maximum a Posterior Estimation of DARTS
    Jun-Liang Lin*, Yi-Lin Sung*, Cheng-Yao Hong*, Han-Hung Lee, and Tyng-Luh Liu

    2021 IEEE International Conference on Image Processing (ICIP)

    HTML PDF
    The DARTS approach manifests the advantages of relaxing the discrete problem of network architecture search (NAS) to the continuous domain such that network weights and architecture parameters can be optimized properly. However, it falls short in providing a justifiable and reliable solution for deciding the target architecture. In particular, the design choice of a certain operation at each layer/edge is determined without considering the distribution of operations over the overall architecture or even the neighboring layers. Our method explores such dependencies from the viewpoint of maximum a posterior (MAP) estimation. The consideration takes account of both local and global information by learning transition probabilities of network operations while enabling a greedy scheme to uncover a MAP estimate of optimal target architecture. The experiments show that our method achieves state-of-the-art results on popular benchmark datasets and also can be conveniently plugged into DARTS-related techniques to boost their performance. Our code is available at https://github.com/MAP-DARTS/MAP-DARTS.
  3. Bioinfo
    ezGeno: An Automatic Model Selection Package for Genomic Data Analysis
    Jun-Liang Lin*, Tsung-Ting Hsieh*, Yi-An Tung*, Xuan-Jun Chen, Yu-Chun Hsiao, Chia-Lin Yang, Tyng-Luh Liu, and Chien-Yu Chen

    Bioinformatics, Volume 38, Issue 1, Pages 30–37

    HTML
    To facilitate the process of tailor-making a deep neural network for exploring the dynamics of genomic DNA, we have developed a hands-on package called ezGeno. ezGeno automates the search process of various parameters and network structures and can be applied to any kind of 1D genomic data. Combinations of multiple abovementioned 1D features are also applicable. For the task of predicting TF binding using genomic sequences as the input, ezGeno can consistently return the best performing set of parameters and network structure, as well as highlight the important segments within the original sequences. For the task of predicting tissue-specific enhancer activity using both sequence and DNase feature data as the input, ezGeno also regularly outperforms the hand-designed models. Furthermore, we demonstrate that ezGeno is superior in efficiency and accuracy compared to the one-layer DeepBind model and AutoKeras, an open-source AutoML package. The ezGeno package can be freely accessed at https://github.com/ailabstw/ezGeno. Supplementary data are available at Bioinformatics online.