Subsequently, there are only 3 parameters left for the BO agent to optimize. 1 and top-5 accuracy achieved by completely different methods when pruning 50% for ResNet56. The rollback scheme further improves the accuracy by 0.9%. Note that in (He et al. Our proposed layer clustering technique improves BO’s efficiency with 1.4% greater in top-1 accuracy. 2018), the writer claimed that AMC can achieve a 90.2% top1 accuracy in 400 epochs. The table additionally exhibits that the BO agent outperforms RL in efficiency by a big margin. For comparability, the best top1 accuracy of our proposed rollback method is as much as 93.14 %, which is significantly increased than the end in (He et al.
Consumer Electronics Show
Our experiments present that our rollback algorithm will further enhance the pruning accuracy, leading to more promising pruning policies which won’t be discovered in the naive BO and the clustering-based BO. The accuracy is estimated primarily based on a random subset of the training set, whose sizes are 5000 and 3000 for Cifar10 ILSVRC2012, respectively. To make a good comparison, we undertake the identical channel pruning scheme with the RL counterpart. In our experiments, we use GpyOpt (SheffieldML 2016) as the fundamental BO agent and implement the proposed strategies base on it. RL agent in (He et al. 2018) is also applied as a baseline. We conduct our experiments on a number of representative CNN model architectures, together with ResNet56 (He et al.
Furthermore, our technique achieves lower variance and shorter time than the RL-primarily based counterpart. There is a growing pattern to use CNNs in different eventualities corresponding to object detection, speech recognition, and many others. Nonetheless, the high performance of CNNs is on the expense of their giant mannequin dimension and high computing complexity, which have prevented it from having a broader usage. Convolutional neural networks (CNN) have gotten standard resulting from their excessive performance and universality.
RL agent is far greater than our proposed layer clustering and the rollback algorithm. After its convergence, the rollback scheme turns the design house again into a excessive-dimensional space and can further improve the accuracy of the pruned community. Thus, our methodology can also be far more efficient from the angle of wall clock time. In Fig. 3(a), we present the effectiveness of the proposed strategies in detail. Note that each one strategies take around 2300 seconds to complete 200 epochs, which signifies the time spent for each trial in BO and RL is shut and the time overhead of rollback is ignorable. Clearly, the layer clustering technique can considerably enhance the convergence of the BO agent.
