Benchmark Analysis of Representative Deep Neural Network Architectures (IEEE Access)

Paper

Please include the following reference in your paper if you mention the method:

Simone Bianco, Remi Cadene, Luigi Celona, and Paolo Napoletano. Benchmark Analysis of Representative Deep Neural Network Architectures. In IEEE Access, volume 6, issue 1, pp. 2169-3536, 2018.

@article{bianco2018dnnsbench,
 author = {Bianco, Simone and Cadene, Remi and Celona, Luigi and Napoletano, Paolo},
 year = {2018},
 title = {Benchmark Analysis of Representative Deep Neural Network Architectures},
 journal = {IEEE Access},
 volume = {6},
 pages = {64270-64277},
 doi = {10.1109/ACCESS.2018.2877890},
 ISSN = {2169-3536},
}

Abstract

This paper presents an in-depth analysis of the majority of the deep neural networks (DNNs) proposed in the state of the art for image recognition. For each DNN, multiple performance indices are observed, such as recognition accuracy, model complexity, computational complexity, memory usage, and inference time. The behavior of such performance indices and some combinations of them are analyzed and discussed. To measure the indices, we experiment the use of DNNs on two different computer architectures, a workstation equipped with a NVIDIA Titan X Pascal, and an embedded system based on a NVIDIA Jetson TX1 board. This experimentation allows a direct comparison between DNNs running on machines with very different computational capacities. This paper is useful for researchers to have a complete view of what solutions have been explored so far and in which research directions are worth exploring in the future, and for practitioners to select the DNN architecture(s) that better fit the resource constraints of practical deployments and applications. To complete this work, all the DNNs, as well as the software used for the analysis, are available online.

Performance indices

Top-1/Top-5 accuracy vs. computational complexity

Architectures

We collect 40 state-of-the-art Deep neural network architectures already trained on ImageNet-1k.

Acknowledgement

Thanks to the deep learning community and especially to the contributers of the PyTorch ecosystem.
Evaluation of Automatic Image Color Theme Extraction Methods This work has been partially supported by E4S: ENERGY FOR SAFETY Sistema integrato per la sicurezza della persona ed il risparmio energetico nelle applicazioni di Home & Building Automation, CUP: E48B17000310009 - Call “Smart Living”.

models-comparison.pytorch

Code for the paper Benchmark Analysis of Representative Deep Neural Network Architectures