Due to this fact, there are only 3 parameters left for the BO agent to optimize. 1 and prime-5 accuracy achieved by different methods when pruning 50% for ResNet56. The rollback scheme additional improves the accuracy by 0.9%. Notice that in (He et al. Our proposed layer clustering methodology improves BO’s efficiency with 1.4% higher in top-1 accuracy. 2018), the author claimed that AMC can achieve a 90.2% top1 accuracy in 400 epochs. The table also reveals that the BO agent outperforms RL in effectivity by a big margin. For comparability, the very best top1 accuracy of our proposed rollback methodology is up to 93.14 %, which is considerably greater than the result in (He et al.

RL agent in (He et al.

Our experiments show that our rollback algorithm will additional enhance the pruning accuracy, leading to more promising pruning insurance policies which won’t be discovered within the naive BO and the clustering-based BO. The accuracy is estimated based mostly on a random subset of the training set, whose sizes are 5000 and 3000 for Cifar10 ILSVRC2012, respectively. To make a fair comparability, we adopt the identical channel pruning scheme with the RL counterpart. In our experiments, we use GpyOpt (SheffieldML 2016) as the essential BO agent and implement the proposed methods base on it. RL agent in (He et al. 2018) can also be carried out as a baseline. We conduct our experiments on several representative CNN – www.pipihosa.com/2018/12/11/4227700-t-shares/ – mannequin architectures, including ResNet56 (He et al.

Moreover, our technique achieves lower variance and shorter time than the RL-primarily based counterpart. There is a rising pattern to apply CNNs in different situations comparable to object detection, speech recognition, and so forth. Nonetheless, the high efficiency of CNNs is at the expense of their large mannequin dimension and excessive computing complexity, which have prevented it from having a broader utilization. Convolutional neural networks (CNN) are becoming common attributable to their high performance and universality.

RL agent is much larger than our proposed layer clustering and the rollback algorithm. After its convergence, the rollback scheme turns the design area again right into a excessive-dimensional area and might further improve the accuracy of the pruned community. Thus, our methodology is also way more environment friendly from the attitude of wall clock time. In Fig. 3(a), we show the effectiveness of the proposed strategies in detail. Note that all methods take around 2300 seconds to complete 200 epochs, which signifies the time spent for each trial in BO and RL is close and the time overhead of rollback is ignorable. Obviously, the layer clustering methodology can significantly increase the convergence of the BO agent.